Meta open sources OPT-66B

This completes a full release of logbooks detailing the development of all the OPT models.

On June 23, 2022, Meta announced the release of the Open Pretrained Transformer(OPT-66B), the largest unrestricted open-sourced model to date. The tech giant has also released the logbooks used for training all their baselines: 125M through 66B. This completes a full release of logbooks detailing the development of all the OPT models and marks the first time in the AI industry that such extensive notes are released with the models and associated paper.

Meta created a buzz when it launched OPT-175B, the large language model that could match OpenAI’s GPT-3 with only 1/7th of the carbon footprint to develop. Meta was one of the first companies to release a language model of such scale on a non-commercial licence and gave access to academic researchers, those affiliated with organisations in the government, civil society, academia and industry research laboratories around the world.

During the launch of OPT-175B, Meta announced it would be releasing the baseline models as well as notes and logbooks of the whole development and training process. As promised Meta has now released baselines  – OPT-125M, OPT-350M, OPT-1.3B, OPT-2.7B, OPT-6.7B, OPT-13B, OPT-30B, OPT-66B on GitHub

AIM Daily XO

Join our editors every weekday evening as they steer you through the most significant news of the day, introduce you to fresh perspectives, and provide unexpected moments of joy
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

For more details, click here.

Sign up for The Deep Learning Podcast

by Vijayalakshmi Anandan

The Deep Learning Curve is a technology-based podcast hosted by Vijayalakshmi Anandan - Video Presenter and Podcaster at Analytics India Magazine. This podcast is the narrator's journey of curiosity and discovery in the world of technology.

Kartik Wali
A writer by passion, Kartik strives to get a deep understanding of AI, Data analytics and its implementation on all walks of life. As a Senior Technology Journalist, Kartik looks forward to writing about the latest technological trends that transform the way of life!

Our Upcoming Events

27-28th Apr, 2023 I Bangalore
Data Engineering Summit (DES) 2023

23 Jun, 2023 | Bangalore
MachineCon India 2023

21 Jul, 2023 | New York
MachineCon USA 2023

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox
MOST POPULAR

The Great Indian IT Reshuffling

While both the top guns of TCS and Tech Mahindra are reflecting rather positive signs to the media, the reason behind the resignations is far more grave.

OpenAI, a Data Scavenging Company for Microsoft

While it might be true that the investment was for furthering AI research, this partnership is also providing Microsoft with one of the greatest assets of this digital age, data​​, and—perhaps to make it worse—that data might be yours.