Stability AI Open Sources GPT-like Models, StableLM

Stability AI, the parent company of Stable Diffusion, has announced StableLM, a suite of open-source large language models available in “alpha” on GitHub and Hugging Face. Under the permissive license, developers can freely access, inspect, and utilize the models for both commercial and research purposes.
Check out the GitHub repository here.

Much like ChatGPT, the suite is designed to efficiently generate text and code. Despite its modest size of 3 and 7 billion parameters (15 to 65B parameter models en route) in contrast to GPT-3's 175 billion. All thanks to a larger version of the open-source dataset known as the Pile. The AI company has decided to stay mum on whether these models suffer from toxicity and hallucinations. Given Pile's profanity-laced content, it's a possibility.

Alongside, the release also includes instructionally fine-tuned research models, utilizing a combination of open-source datasets such as Stanford’s Alpaca, GPT4All, Databricks’ Dolly, ShareGPT, and HH. These models, designed exclusively for research purposes, are made available under a noncommercial CC BY-NC-SA 4.0 license, aligning with Alpaca’ license.

Last year, the Emad Mostaque run company made its text-to-image AI available through a public demo, a software beta, and a full download of the model, allowing users to experiment with the tool and come up with various integrations. A similar trend can be expected with StableLM as Meta’s open-source LLaMa language model has been broadly adapted by the community within a few weeks of its release. 

Stability AI's release has been met with criticism by some researchers, who worry about the models’ potential in nefarious activities. However, Stability AI stands firm in their belief that transparency and accessibility are key to advance technology and it sees open-sourcing as the right path forward.

Read: 14 Open Source LLMs You Need to Know