Article complete

Meta Partners with Microsoft to Release Llama-2

The release is now made available as an open-source platform for research as well as commercial use.

A llama wearing sunglasses with yellow lenses against a bright yellow background, with "LLAMA" text behind it.

Key takeaways

  • Meta and Microsoft have announced the release of Llama 2, an advanced open-source language model.
  • Llama 2 will be accessible on Microsoft Azure and Amazon SageMaker, catering to diverse development environments.
  • The new model includes performance enhancements and improved safety measures over its predecessor.
  • A fine-tuned variant, Llama 2-chat, is specifically designed for optimised dialogue applications.

Takeaways drafted with AI from the article text.

In a groundbreaking announcement today at Microsoft Inspire, Meta and Microsoft announced the release of Llama 2, the cutting-edge iteration of their renowned open source LLM, LLaMa. 

The model will also be available on Microsoft Azure platform catalog and Amazon SageMaker. The developers can also access the model on Windows platforms such as Subsystem for Linux (WSL), Windows terminal, Microsoft Visual Studio and VS Code.

The release is now made available as an open-source platform for research as well as commercial use.

Building upon the success of the original LLaMA release earlier this year, which was hailed by the open source community, Llama 2 introduces a host of enhancements, bolstering its performance and safety measures. This also includes a fine-tuned model called Llama 2-chat, built for optimised dialogue use cases. 

https://twitter.com/ylecun/status/1681336284453781505

The highlight of this release is the introduction of the 7B, 13B, and 70B pre-trained and fine-tuned parameter models. Representing a substantial leap forward from its predecessor, Llama 2 incorporates a staggering 40% increase in pre-trained data, harnesses larger context data for training, and leverages GQA (Generalised Question-Answering) to achieve superior inference capabilities for the larger model.

Along with Microsoft, Meta also partnered with Amazon, HuggingFace, NVIDIA, Qualcomm, IBM, Zoom, Dropbox and a number of academic leaders from around the world to release the model highlighting the importance of open source software. 

The paper highlights how the model is trained on publicly available data sources. Interestingly, the company also highlights that this does not include data from Meta’s products or services. “We made an effort to remove data from certain sites known to contain a high volume of personal information about private individuals,” said the researchers. 

Read: The Biggest Winner from Threads: LLaMA

The fine-tuned Llama 2-Chat models have undergone meticulous optimisation for dialogue purposes. Employing a combination of supervised fine-tuning and multiple RLHF (Reinforcement Learning from Human Feedback) methods, including rejection sampling and PPO (Proximal Policy Optimization), the Llama 2 models have been enriched with over 1 million annotation points. This approach ensures enhanced interactions and responses in dialogues, setting new benchmarks in the field.

https://twitter.com/Mascobot/status/1681336872759603200

With rigorous evaluation against human and academic benchmarks across both closed and open alternatives, the Llama 2 model demonstrates an unparalleled level of competitiveness. Researchers and developers can now harness its formidable capabilities for a wide array of applications and projects, revolutionising the future of language processing and understanding.

Meta's release of Llama 2 marks a significant step towards advancing the boundaries of AI research and its accessibility to the wider community. By making Llama 2 open source and available for commercial use, Meta fosters collaboration, inspiring further innovation, and opening the doors to new possibilities in the realm of generative AI. 

Published Updated

Spotted an error? Tell our editors. Read our editorial standards and corrections.

More in this story

LLaMA · 5 stories
  1. Meta Launches New LLM LLaMA which Outperforms GPT-3 at a Fraction of the Size
  2. You are here: Meta Partners with Microsoft to Release Llama-2
  3. Meta AI's MuAViC Sets New Benchmark for Highly Accurate Speech Translation
  4. Llama 3 Might Not be Open Source
  5. Meta AI Unveils Husky, a Unified, Open-Source Language Agent
About the author

Journalist

Mohit writes about AI in simple, explainable, and often funny words. He's especially passionate about chatting with those building AI for Bharat, with the occasional detour into AGI.

All stories
Belamy · Daily

The day’s AI news, before your first meeting.