Meta’s Llama 3 400 Bn Might Not be Open Source

Finally, the predictions are coming true. Meta is most likely to not open source its Llama 3 with a 400 billion parameter size model. 

Or rather, an AI insider who goes by the name Jimmy Apples revealed this. 

“Meta plans to not open the weights for its 400B model,” said Jimmy Apples on X, raising lots of eyebrows.  

To this, Meta AI chief said: “Patience, my blue friend. It's still being tuned,” to one of the users on X who asked, “Yann, please comment on meta not releasing llama-405B. Is this FUD? Or have you guys changed your stance on OSS?”

In April, Meta released two Llama 3 models: an 8-billion-parameter model and a 70-billion-parameter model. Coincidently, Llama 3 has become the talk of the town for the Indian developer ecosystem, to say the least. (see below) 

https://www.youtube.com/watch?v=RAoTisOaJSQ

Meta announced that they are currently training a 400-billion-parameter model, which will be released over the coming months. The new model will feature enhanced capabilities such as multimodality, the ability to converse in multiple languages, an extended context window, and overall stronger performance.

Llama 3 (8B and 70B) offers enhanced reasoning and coding capabilities, with its training process three times more efficient than its predecessor. The model is now available on GitHub and Hugging Face, allowing developers to integrate it into their projects. 

Check out Wild Llama 3 use cases here. 

Zuckerberg also said, in his podcast with Lex Fridman last year, that Meta might have to reconsider open-sourcing the next iteration of Llama, Llama 3. “Right now, the priority is building that into a bunch of consumer products,” said Zuckerberg. 

At the same time, Meta Chief has been a proponent of open-source AI and loves what the community has been doing with Llama 2. The developer community that AIM spoke to hopes this news turns out false. 

“We would need a process to red team this, and make it safe. My hope is that we would be able to open source the next version when it is ready to, but we are not close to doing that this month. It’s a thing that we are still early in work now,” said Zuckerberg, in September last year.