Key takeaways
- DeepMind's AlphaDev claims to have developed a sorting algorithm that sorts data up to three times faster than traditional methods, but this assertion is met with skepticism from experts.
- Critics argue that the algorithms presented by AlphaDev are not genuinely novel, suggesting that the findings were exaggerated to enhance DeepMind's reputation amid competition from other tech companies.
- The development of AlphaDev mirrors DeepMind's previous work on protein folding, employing AI to tackle complex problems in computer science, specifically algorithm optimisation.
- The approach taken by DeepMind involved structuring the algorithm discovery as a game, building on their experience with AI agents in gaming contexts.
Earlier this week, Google DeepMind created a new algorithm based on AlphaZero. Termed AlphaDev, this algorithm can reportedly create algorithms that can sort data up to 3 times faster than human-written algorithms. While this has been heralded as one of the biggest advancements in machine learning, it is actually a classic case of making a mountain out of a molehill.
Delving deeper into the purported findings, we found that this story was spun in an attempt to make DeepMind look more relevant. In reality, these ‘new sorting algorithms’ were actually not novel in any way and were cherry-picked to make the findings look bigger than they actually were. This also comes at a time when DeepMind is falling behind other tech giants in AI innovation.
AlphaDev explained
Just like DeepMind researchers solved protein folding, they set out to tackle one of the biggest problems in computer science: sorting algorithms. These algorithms are used trillions of times across various computer systems, with DeepMind choosing to take on the task of optimising them. The think tank approached this problem by deploying AI to ‘go beyond the current state of the art by discovering hitherto unknown routines’.
To do so, DeepMind structured the task of discovering the sorting algorithm as a single-player game, which allowed them to deploy their past experiences designing AI agents to play games. Using this approach, DeepMind created a new AI agent they call AlphaDev, which was then put to the task to discover new algorithms.
AlphaDev is a spiritual successor to the AlphaZero algorithm, but instead of choosing possible moves in a board game, it chooses instructions to add to a sorting procedure. What’s more, the agent carried out these operations at the assembly level, similar to programmers who write code in C++, the language of choice for sorting algorithms.
At every step of the process, AlphaDev is given a few choices — comparing values, moving values between different locations, or jumping between different parts of the program. Once it picks its choice, it tries sorting a list. Based on the success of its sorting algorithm, it receives a reward. This process continues until it sorts all the lists perfectly.
According to DeepMind, the agent’s best results performed up to 71% better than human-written algorithms. DeepMind has also gone on record stating that they were ‘a bit shocked’ to see the results and want to ‘tackle the whole stack’ with AlphaZero-style algorithms.
However, it is important to note this colossal time saving dropped to 1-2% when dealing with larger values due to the unoptimised code present in other parts of the program. While experts in the field acknowledged the ability of the algorithm to generate code, they also pointed out that the paper did not bring any new innovation to the table.
Not that great
In discussions all over the Internet, AlphaDev has been universally panned as one of the biggest non-news in AI this week. orlp, a Hacker News user, stated that the main reason that they were able to net a 70% improvement on a certain libc++ algorithm was due to the fact that the library has not been actively developed for the last 10 years. Moreover, DeepMind’s improvements only worked due to a fundamental issue with the library itself not having an efficient implementation of a branchless sorting network.
Other users pointed out that this take was ‘overly cynical’, and that the actual improvement was the fact that the algorithm was able to create a new sorting algorithm by itself. However, orlp noted that while the algorithm did generate good code automatically, it was nowhere close to being revolutionary or an improvement on the state of the art.
In fact, AlphaDev’s strengths lie in code assembly, finding missing optimisations, and balancing between correctness and distribution for hashing algorithms. The algorithm did not find an entirely new approach to sorting, but instead micro-optimised the code. However, programs known as super optimisers already exist. These optimisers find the most optimal programs, usually by the brute force method.
While DeepMind has struck upon something new by deploying AlphaZero to the coding process, the impact of this has largely been magnified. All AlphaDev did was find a faster micro-optimisation for a particular library running on a specific CPU, far removed from the various deployment scenarios found in the world of computing. AlphaDev might be capable in its field, but it’s not the earth-shaking discovery that it was made out to be.
