Google’s New AI Model Outperforms DeepSeek-V3, OpenAI's o3-mini

Google on Wednesday announced Gemma 3, the next iteration in the Gemma family of open-weight models. It is a successor to the Gemma 2 model released last year. 

The small model comes in a range of parameter sizes – 1B, 4B, 12B and 27B. The model also supports a longer context window of 128K tokens. It can analyse videos, images, and text, supports 35 languages out of the box, and provides pre-trained support for 140 languages. 

In the Chatbot Arena, Gemma 3 27B outperformed DeepSeek-V3, OpenAI’s o3-mini and Meta’s Llama 3-405B model. Models in Chatbot Arena are evaluated against each other through side-by-side evaluations by humans. 

Moreover, Gemma 3 27B scored 67.5% and 42.4 across standard benchmarks like MMLU-Pro, GPQA Diamond, respectively. The model performs well compared to other small models in the competition. 

Claude 3.5 Haiku scored 63% on the MMLU-Pro benchmark and 41% on GPQA Diamond, while OpenAI’s GPT-4o Mini achieved 65% and 43% on the same tests, respectively. Meta’s Llama 3.3 70B outperformed both, with 71% in MMLU-Pro and 50% in GPQA Diamond, making it the strongest contender among these models.

However, Gemma-3’s key superpower seems to be efficient compute usage. Google said that Gemma 327B achieved the scores with a single NVIDIA H100 GPU, whereas other models necessitated up to 32 GPUs. 

Source: Google

The company also revealed that the architecture of the model was modified to reduce the KV-cache memory, which tends to increase with longer context. 

Google has published a detailed technical report outlining the techniques used to build the model, its performance and other specifications. Gemma 3 can be accessed via various methods. Google is offering the model on the web using the Google AI Studio, via the default chatbot or the API, and it is also available on the Google GenAI SDK. 

Besides, the model can be downloaded for local deployment on Hugging Face, Ollama, and Kaggle.

Along with Gemma 3, Google has also launched ShieldGemma 2, a 4B parameter image safety checker built on Gemma 3’s foundation. This provides safety labels for harmful images which involve dangerous, sexually explicit, and violent content.