While We Grapple With Geometry, Google DeepMind's AI Model Beats Math Olympiad Gold Medalists

Google’s AI lab, DeepMind, has unveiled a new AI model, AlphaGeometry2, which they claim outperforms some of the top minds who have won a gold medal in the International Mathematical Olympiad. 

Last year, it hit the silver medal mark, and this year, we have a gold.

The research paper claims to have an overall solving rate of 84% for all geometry problems over the last 25 years.

DeepMind initially published the first iteration of the AI model back in January 2024 with a 54% solve rate. Looking back, it sounds like good progress with a year of new development. With AlphaGeometry2, the model now tackles locus-type theorems, linear equations, and non-constructive problem statements. 

The AI model is built as a neuro-symbolic system that combines a language model with a symbolic engine to tackle challenging geometry problems. Under the hood, it leverages the Gemini architecture with an increased model size and a diverse dataset.

DeepMind’s model was trained on algorithmically generated synthetic data. The method starts by sampling a random diagram and using the symbolic engine to deduce all possible facts from it. It avoids using human-crafted problems and instead starts from random diagrams.

As per the research paper, AlphaGeometry2 translates geometry problems in natural language. The paper mentions, “To do this, we utilise Gemini Team Gemini (2024) to translate problems from natural language into the AlphaGeometry language and implement a new automated diagram generation algorithm.”

The paper mentions the approach of setting the stage for test results, “There are a total of 45 geometry problems in the 2000-2024 International Math Olympiad (IMO), which we translate into 50 AlphaGeometry problems (we call this set IMO-AG-50). Some problems are split into two due to specifics of our formalisation.” 

In addition, it also sheds light on how effective it was, where the model solved 42 out of 50 of all 2000-2024 International Math Olympiad (IMO) geometry problems, thus surpassing an average gold medallist for the first time.

The model was also pitched against other models like OpenAI’s o1, but as you can see in the table above, AlphaGeometry2 solved most of the questions.

To arrive at a conclusion, the paper highlights, “Our geometry experts and International Math Olympiad (IMO) medallists consider many AlphaGeometry solutions to exhibit superhuman creativity.” They also add, “Despite good initial results, we think the auto-formalisation can be further improved with more formalisation examples and supervised fine-tuning.”

With models like AlphaGeometry2, AI is also getting into the high school math competition, which is an intriguing development.