Article complete

Meta's Breakthrough Language Model on Par with GPT-4 and Bard in Performance

One of the notable aspects of LIMA’s performance is its ability to generalise well to unseen tasks that were not part of its training data.

A robotic figure stands before a checkered finish line, with a large blue infinity symbol above it.

Key takeaways

  • LIMA, developed by Meta and academic partners, is a 65 billion parameter language model that exhibits strong performance comparable to GPT-4 and Bard.
  • The model demonstrates impressive generalisation abilities, effectively handling unseen tasks outside of its training data.
  • LIMA excels in learning specific response formats with minimal training examples, allowing it to tackle complex queries.
  • Responses generated by LIMA rival those of human-tuned models, showcasing its advanced capabilities.

Takeaways drafted with AI from the article text.

Researchers for Meta AI, alongside Carnegie Mellon University, University of Southern California and Tel Aviv University, today unveiled LIMA, a 65 billion parameter LLaMa language model fine-tuned with the standard supervised loss on only 1,000 carefully curated prompts and responses, without any reinsurance learning or human preference modelling. 

Check out the research paper: LIMA: Less Is More for Alignment

The researchers said that LIMA has shown strong capabilities in learning specific response formats with minimal training examples. They said that it can effectively handle complex queries, ranging from planning travel itineraries to speculating about alternate history. 

Interestingly, one of the notable aspects of this new language model’s performance is its ability to generalise well to unseen tasks that were not part of its training data. In other words, LIMA can apply its learned knowledge to new and unfamiliar tasks, demonstrating a level of flexibility and adaptability, shared by the researchers. 

In comparison to human-controlled responses, specifically GPT-4, Bard, and DaVinci-003 (trained with human feedback), the responses generated by LIMA were quite impressive. For instance, in 43% of cases, responses from LIMA were either equivalent to or preferred over GPT-4. In the case of Bard, LIMA was preferred in 58% of cases. In the case of DaVinci-003, the preference rose to almost 65%.

It is interesting to note that with only limited instruction tuning data, models like LIMA can generate high-quality output. 

A few weeks back, Meta also released MEGABYTE, a scalable architecture for modelling long sequences. This new technique has outperformed existing byte-level models across a range of tasks and modalities, allowing large models of sequences of over 1 million tokens.

Published Updated

Spotted an error? Tell our editors. Read our editorial standards and corrections.

More in this story

ChatGPT · 5 stories
  1. You are here: Meta's Breakthrough Language Model on Par with GPT-4 and Bard in Performance
  2. OpenAI Looks to Sell its Existing Shares At Record High Valuation
  3. OpenAI Enhances ChatGPT’s Voice Modality Feature
  4. OpenAI Dismisses Senior Policy Executive After ‘Adult Mode’ Criticism: Report
  5. ChatGPT Ads Hit $1 Billion ARR in Under 200 Days
About the author

Journalist

Mohit writes about AI in simple, explainable, and often funny words. He's especially passionate about chatting with those building AI for Bharat, with the occasional detour into AGI.

All stories
Belamy · Daily

The day’s AI news, before your first meeting.