StableLM Might Be What Stability AI Needs To Survive

Earlier this week, Stability AI, the creators of Stable Diffusion, released a family of large language models they called StableLM. This model, currently in its alpha version, is available in 3 billion and 7 billion parameter versions, with the company promising the future release of 15 and 65 billion parameter versions.

While this model is sure to provide strong competition to OpenAI’s GPT suite of models, it also demonstrates the power of an open-source strategy. StableLM oozes open-source from the ground up, whether it’s the models’ datasets or even crowd-sourced reinforcement learning with human feedback (RLHF) efforts. 

However, despite the good intentions, the UK-based startup is struggling below the surface. A report earlier this month detailed the company’s financial trouble, burning through millions of dollars of funding to give back to the open-source community. Could StableLM be a saving grace for the troubled company? 

Stability AI’s love letter to open-source

Emad Mostaque, the CEO of Stability AI, has been quite vocal in the past about championing open-source. He has gone on record stating that Stability AI will be the ‘largest supporter of open-source innovation via individuals, collaboratives and academia’, calling Stability AI the RedHat to open-source’s Linux. 

StableLM, was also released with the same intentions of being transparent, accessible, and supportive. Building on the open-source history established by the launch of Stable Diffusion, the new set of models has also been released under a non-commercial CC BY-NC-SA 4.0 licence, which allows for usage in both commercial and research use cases. 

Even StableLM’s datasets come from a set of 5 open-source datasets for conversational agents, namely those used for Alpaca, GPT4All, Dolly, ShareGPT, and HH. As part of the StableLM launch, the company has also pledged to contribute to LAION’s Open Assistant project, creating an open-source dataset for AI assistants. 

Stability AI’s past offering, Stable Diffusion, has become the go-to model for the open-source community’s image generation needs. The Stable Diffusion GitHub repo has been forked over 2,400 times and has garnered mentions in papers by researchers from Microsoft and Google. In addition to this, many members of the open-source community have fine-tuned the model for their own projects such as Scribble Diffusion, Amazing AI, Riffusion, and Unstable Diffusion. 

Even though Stability AI’s models have added a lot of value to the open-source community, this has not translated into hard cash in the company’s coffers. However, this might change with the launch of StableLM, and by extension, Stability’s business model. 

Seeking stability on shaky ground

A report recently emerged on the state of Stability AI’s finances, and it does not paint a pretty picture. To provide some history, a majority of the company’s funding came from Emad Mostaque, and other investors like Eros Investments. To add to this, late last year, the company raised £89 million, or close to $100 million in seed funding.

According to people familiar with the matter, the company has burned through a significant chunk of this seed funding, as developing state-of-the-art AI models is not a cheap undertaking. Mostaque has previously stated that the training cost of Stable Diffusion is close to $600,000, as it took over 150,000 GPU hours on AWS servers. A second funding round that would value the company at $4 billion has also left 2 VCs with cold feet due to Mostaque's leadership style. 

In addition to this, the divisive CEO has also spoken about his plans to take Stability AI public in the next few years. He said that an IPO wasn't the easiest thing and that a company needs to have “amazing revenue, amazing margins, distribution” to do so. 

Until the company goes for an IPO, it seems Mostaque has a plan in place. He stated, 

“Dozens of major companies want models they own based on their own data and want to pay us to do it…It's a pretty good model. Stable models are benchmark open-source of every modality based on open data. We also build custom models for largest cos & govts [sic]”.

This can be taken to mean that Stability AI will look to keep their models open-source but charge for the service of fine-tuning or training models on proprietary data pools. This will allow Stability to stay open-source while helping them avoid competition altogether. 

There are other moving parts that are taking shape, characterised by Stability AI's deal with AWS. As part of this partnership, Stability AI made Stable models available on Amazon Bedrock, a set of foundation models made available to AWS customers. The company also struck a partnership with AWS before this, making them their preferred cloud service provider. 

Even though the company is currently on shaky ground when it comes to finances, striking partnerships currently seems to be one of the ways to keep itself afloat. In addition to this, pursuing the training of models on private data pools will also ensure a steady inflow of money, allowing Stability AI to keep up with the Microsoft-OpenAI combo while staying committed to open-source.