Stability AI is taking the open source community by storm. After the open source Stable Diffusion, the company released its LLM called Stable LM. And now, it ups the game with a smaller version — Stable LM 3B, ideal for running on-edge for smart devices.
With 3 billion parameters and pre-trained on 1 trillion tokens of English and code datasets for 4 epochs, Stable LM 3B is designed to run efficiently on portable digital devices like handhelds and laptops, or even the upcoming headsets, a departure from the industry norm of models such as Llama 2 or Falcon. In contrast to larger models, it demands fewer resources and comes with reduced operating costs, making it accessible to a wide range of users.
This model is made available under the open-source CC-By-SA 4.0 licence, which means it is still experimental. That means it is not yet commercially available freely, without an agreement with Stability AI. Though this kills the open source approach of Stability AI, it is still outperforming other larger models on various benchmarks.
Sounds like Apple
One of the focuses of Stability AI with this release is affordability and sustainability by using less power. Despite its modest scale, Stable LM 3B is highly competitive, outperforming both previous 3B parameter language models and some leading open-source models with 7B parameters.
Furthermore, Stable LM 3B is adaptable. While it serves as a general-purpose language model, it can be fine-tuned for specific applications, such as aiding in programming tasks. Stability AI warns that developers should be aware that Stable LM 3B serves as a base model, requiring adjustments to ensure safe and optimal performance in specific applications, such as chat interfaces.
Depending on their use case, developers must evaluate and fine-tune the model prior to deployment. “Our instruction fine-tuned model is currently undergoing safety testing and will be released in the near future,” reads the blog.
Bringing Apple into the picture, the recent event was a testament to how much the company is dedicated to sustainability and making their products carbon neutral. Additionally, the company also wants to bring all of the AI capabilities on-device. Both of these ideas are what Stability AI is also focusing on.
Furthermore, there have been rumours that Apple might even approach Stability AI for a buyout. According to many reports, Emad Mostaque, the founder of Stability AI, is in talks with the founder of OTOY, a company that provides decentralised GPU-based rendering solutions for Apple.
If Stability AI continues doing what it is doing right now, Apple’s offer might be the best that the open source company can hope for. It is clearly in development.
What about Meta AI?
Is Stability AI a problem for Meta? Reportedly, many of the companies have been ditching OpenAI’s proprietary models for Llama 2, and fine-tuning it on their own data. The open source champion has been on the forefront of many AI developments. But what if the companies find Stability AI’s offerings better than Meta’s?
Stable LM 3B is also based on a similar architecture as Llama’s decoder-only transformer architecture. Moreover, the dataset used for training is Falcon-refined web and RedPajama dataset, which is the open source repository for Llama.
Moreover, Meta is also possibly heading the closed source way. In the recent podcast with Lex Fridman, Mark Zuckerberg said that he is still on the fence on deciding whether to open source the next iteration of their LLM Llama 3. This is similar to how all companies that started out as open source driven, such as OpenAI, become closed source. Mistral AI also has plans for the same.
Developers love open source. That is why Meta with its open source LLaMA and Llama 2 have been getting all the praise. But with Stable LM 3B, the tables might turn against Meta and in the favour of Stability AI.
