Recently, Google-backed DeepMind announced the launch of its generator model, AlphaCode, on GitHub, where it has made the dataset and code available. 

Click here to view.

With this latest announcement, the company has also included extensive tests on the platform to ensure the programmes that pass these tests are correct—a critical feature current datasets lack. 

Earlier this year, AlphaCode had made waves through its potential to beat computer programmers by analysing the algorithm and generating complex programmes. 

Simplifying computer programming 

The developers at DeepMind tested the potential of AlphaCode by testing it in competitive programming websites where human developers are given programming problems and ranked on the basis of their results.

One of them was a competitive coding competition on Codeforces, a popular platform for hosting coding competitions. A selection of ten varied test problems from different stages of development was given to AlphaCode. 

The AI tool achieved an estimated rank within the top 54 percentile of participants that attended the contest, thus proving that AlphaCode’s code generation system has achieved results at a competitive level. 

AlphaCode vs Codex 

AlphaCode is a transformer-based language model that consists of 41.4 billion parameters. It is a language model four times the size of GitHub Copilot’s language model Codex that parses 12 billion parameters only. The architecture of AlphaCode is based on three parts:

• Data: The AI tool is fed data by public GitHub repositories. 
• Learning: The tool then trains on the datasets and calibrates them to the task’s requirements (e.g., competitive programming at Codeforces).
• Sampling and evaluation: Here, the AI tool performs large-scale sampling of variations of programmes for each problem. Then, through the process of filter and cluster, the programmes are ranked into a small subset of ten solutions that are submitted for external assessment.

AlphaCode’s AI system is pre-trained in various programming languages that include C++, C#, Go, Java, JavaScript, Lua, PHP, TypeScript, Ruby, Scala, Rust and Python. This dataset consists of approximately 715 GB of codes along with their descriptions.

Through AlphaCode, DeepMind has been able to fill the gap that is lacking in AI models like Codex, which is problem-solving skills. AlphaCode has not only been trained to “understand” natural language but also to design complex programmes and algorithms and implement them in code. 

AI expert Alberto Romero said in an article that the company created five sizes of AlphaCode models, which included parameters spanning 300M, 1B, 3B, 9B, and 41B. All these are named AlphaCode, but the one that the organisation refers to in their communications is an ensemble of the 9B and 41B models combined with clustering. 

Romero further said that they built models of different sizes to compare the effects of scale, training times, and compute efficiency, among other factors  He also said that the model tends to program better in Python than C++ and generates a similar amount of dead code to humans.