Llama 3 Might Not be Open Source

Meta AI, and everyone in its AI team, including Mark Zuckerberg and Yann LeCun, have been one of the biggest proponents of open source. It was indeed leading the open source LLM race all this while with LLaMA and then the pseudo open source, Llama 2. But since then, a lot of models have been coming up in the open source community that are outperforming Llama 2 on various benchmarks.

Now, it might be time for tech-giant to realise how to finally use its AI technology to earn some bucks. Why should it compete in the open source market? Zuckerberg, in his latest podcast with Lex Fridman in the metaverse, said that Meta might have to reconsider if it is going to open source the next iteration of Llama, which is Llama 3. “Right now, the priority is building that into a bunch of consumer products,” said Zuckerberg. 

At the Meta Connect 2023, the team announced AI in a lot of consumer products such as AI chatbots and characters on WhatsApp and Instagram, based on celebrities, and a slew of other integrations. There is no doubt that open sourcing models and integrating them into its products has helped Meta move faster, but now there might be a chance of going the other way. 

Meta learns how to use Llama

Zuckerberg adds that Meta trained Llama 2 and released it as an open source, but it is not a consumer product, but just an AI infrastructure. Though Zuckerberg is all about open sourcing AI and loves what the community has been doing with Llama 2, when it comes to Llama 3, he said that the debate with Fridman was very helpful for open sourcing Llama 2, and the same would be needed for Llama 3. 

“We would need a process to red team this, and make it safe. My hope is that we would be able to open source the next version when it is ready to, but we are not close to doing that this month. It’s a thing that we are still early in work now,” said Zuckerberg. This time, Llama 3 might even be better than GPT-4.

Thus, making Llama 3 safe sounds like a great bet, but it is somewhat in contrast to what Zuckerberg said in the last podcast with Fridman. “The stage we are in right now, the equities balance strongly in my view towards doing this more openly.” He believed that if companies think that they have gotten close to what they believe is “superintelligence”, it makes sense for them to discuss and think through a lot more. 

“We want a lot more researchers working on it and for a lot of these reasons open source has a lot of advantages.” He explained how open source software is more secure because there are a lot of people criticising it, finding holes in it, and thus in the end making it safe. “I think this is how we will achieve more aligned models, by making it open source and letting people work on it,” said Zuckerberg.

There were still discussions if Llama 2 is in the works or not, but then it was actually open sourced by Meta. Yann LeCun also said that open sourcing is the best way to make progress in the LLM field, and thus promotes every company who goes this way. 

But, there is always a doubt about how to make money out of open source AI. For example, Mistral AI, a startup founded by former DeepMind and Meta employees, is also a great proponent of open source. Just this week, the team released an open source model. But in 2024, the team aims to raise more fundings in 2024, to build a consumer product, apart from the open source ones. 

Similarly, Meta’s competitors like Google or OpenAI have not released their models to the public and kept it closed. But earlier, when OpenAI released GPT-2, it was open source. The company decided to pivot to the closed source model approach after GPT-3. Probably because it realised that the revenue model with open source is not that fool proof. 

Stepping up the game

With Zuckerberg’s latest comments, it seems like Meta might steer a little away from being the champion of open source and finally learn how to monetise it within its consumer products. Which is indeed fair. The company has been making strides in the metaverse and with its headsets, where it is implementing AI. 

Meta’s Llama 2 is arguably still “an order of magnitude behind what OpenAI and Google are doing,” said Zuckerberg. But still, many companies have been adopting the open source model and earning a few bucks for themselves. It would only make sense now for Meta to keep Llama 3 for itself, and use it for consumer products, at least for a little bit, just to catch up on the four years. Even if that means, ditching its true nature.