Since the open-source LLM boom began with the LLaMA leak, many other parties have wanted to follow the trend. Last week alone, two coding LLMs were released, suggesting the beginning of a new trend. Now that LLMs with capabilities close to GitHub Copilot are in the hands of the open-source community, developers have mixed feelings.

Even as the release of LLaMA spurred the creation of a bevy of open-source LLMs, it seems that these new coding LLMs will do the same for auto-coders. Both BigCode’s StarCoder and Replit’s Code V1 offer an open-source alternative to Copilot’s proprietary LLM based on GPT-4, opening them up to tinkering and product integration.

Is open source the future of AI?

Recently, a researcher from Google stated in a leaked document that neither OpenAI nor Google has a moat when it comes to LLMs, giving the win to the open-source community. He stated:

“We should not expect to be able to catch up [to open source]. The modern internet runs on open source for a reason. Open source has some significant advantages that we cannot replicate.”

This is largely true, especially when considering the impact that LLaMA had on the open-source ecosystem. The list of innovations brought to the model is long and numerous, but the key takeaway is that all of these improvements were made by volunteer AI enthusiasts wishing to build a better product.

Even though Meta stood to gain the least from the LLaMA leak, the improvements made by the community effectively netted them a planet’s worth of free labor. This is, by far, the biggest value proposition of open source, as it allows volunteers to contribute to even the biggest projects.

Until now, the biggest barrier for entry for the open-source community when it comes to LLMs was the training cost, which routinely went up into the millions for especially big models. However, developers were able to cut down the training cost for LLaMA down to $300 and even optimize it to the point where it could run on a Raspberry Pi.

Regarding auto-coders, programmers and developers were in the pre-LLaMA state of affairs before the launch of this week’s LLMs. They either had to use a proprietary, closed source solution, like OpenAI’s GPT-4, GitHub Copilot, or Tabnine, or fine-tune an existing open-source LLM at a large personal cost.

Both of these approaches were not conducive to harnessing the potential of open source, hampered as they were by restrictive licenses and usage rights. However, with the launch of StarCoder and CodeV1, there is an opportunity for auto-coding-powered projects to be released onto the market.

Innovation waiting to happen

Even though both these models were released under open-source licenses (CC by SA for CodeV1 and OpenRAIL-M for Starcoder), it seems as though they are serving different purposes in the open-source world. Out of the two, StarCoder is arguably built from the ground up for the open-source community, as both the model and a 6.4TB dataset of source code were open-sourced at the same time.

The model was also found to be better in terms of quality than Replit’s Code V1, which seems to have focused on being cheap to train and run. On the HumanEval benchmark, StarCoder was able to reach a score of 40.8%, while Code V1 could only reach 30.5%. Moreover, StarCoder can do more than just predict code, it can also help programmers review code and solve issues using code metadata.

One drawback of StarCoder is its hardware requirements, which demand at least 32GB of GPU memory in 16-bit mode. However, if LLaMA is any indication, the open-source community only needs a few more weeks to optimize this model to run even on phones and laptops.

Replit’s CodeV1, on the other hand, seems to be a valuable addition to their pre-existing software ecosystem. CodeV1 appears to be another step in their strategy to democratize access to an AI-powered software stack. This way, instead of locking in developers to a single ecosystem like GitHub Copilot is trying to do, it can target a much larger and diverse market of developers who want freedom in the way they use AI.

User runnerup on the Hacker News forum had this to say about CodeV1: “Things like GitHub Copilot don’t allow me to use their AI against my codebase the way that I want to. So if Replit can find more innovative, boundary-pushing ways of integrating LLMs, they won’t necessarily need the highest quality LLMs to produce a superior user experience.”

Most importantly, it does not matter that these models are either difficult to run or inaccurate, as the open-source community can easily remedy these issues. The volunteers working on LLaMA and its associated projects have shown that the community can make models both easy to run and more accurate through efforts such as crowd-sourced RLHF. However, one thing is certain: the innovation is just beginning for these open-source models.