Article complete

Google Gemini 1.5 Crushes ChatGPT and Claude with Largest-Ever 1 Mn Token Context Window

It can process vast amounts of information in one go, including 1 hour of video, 11 hours of audio, codebases with over 30,000 lines of code, or over 700,000 words.

A stylized illustration of a smiling man in a suit with the Google logo behind him.

Key takeaways

  • Google Gemini 1.5 introduces a groundbreaking 1-million-token context window, the largest in NLP history.
  • The model can process extensive data, including 1 hour of video and 700,000 words in one go.
  • A limited group of developers can access Gemini 1.5's advanced features via AI Studio and Vertex AI.
  • Gemini 1.5 significantly outperforms competitors like ChatGPT and Claude in context processing capabilities.
  • Future tests indicate potential for processing up to 10 million tokens, expanding possibilities for AI applications.

Takeaways drafted with AI from the article text.

“We’ve been able to significantly increase the amount of information our models can process — running up to 1 million tokens consistently, achieving the longest context window of any large-scale foundation model yet,” reads the blog, co-authored by Google chief Sundar Pichai and Google DeepMind chief Demis Hassabis, comparing it with existing models like ChatGPT and Claude.

Gemini 1.5 Pro comes with a standard 128,000 token context window. But starting today, a limited group of developers and enterprise customers can try it in a private preview with a context window of up to 1 million tokens via AI Studio and Vertex AI. 

It can process vast amounts of information in one go, including 1 hour of video, 11 hours of audio, codebases with over 30,000 lines of code, or over 700,000 words. In their research, Google also successfully tested up to 10 million tokens.

Gemini 1.5 is built upon Transformer and MoE architecture. While a traditional Transformer functions as one large neural network, MoE models are divided into smaller "expert” neural networks.

Gemini 1.5 Pro's capabilities span various modalities, from analysing lengthy transcripts of historical events, such as those from Apollo 11's mission, to understanding and reasoning about a silent movie. The model's proficiency in processing extensive code further establishes its relevance in complex problem-solving tasks, showcasing its adaptability and efficiency.

Gemini 1.5 Pro's performance in the Needle In A Haystack (NIAH) evaluation stands out, where it excels at locating specific facts within long blocks of text, achieving a remarkable 99% success rate. Its ability to learn in context, demonstrated in the Machine Translation from One Book (MTOB) benchmark, solidifies Gemini 1.5 Pro as a frontrunner in adaptive learning.

This new development comes after Google released the first version of Gemini Ultra just last week. Recently, Google added generative AI features to Chrome as well. Google has introduced the “Help me Write” feature across all websites. By right-clicking on any text box, users can access the feature, prompting Google’s AI to inquire about their writing requirements and subsequently generate an initial draft.

While Google is focusing on improving its AI models, OpenAI is reportedly working on a web search product to challenge Google. Additionally, OpenAI is working on its next LLM, GPT-5, which is expected to be smarter than ever, according to OpenAI CEO Sam Altman.

Published Updated

Spotted an error? Tell our editors. Read our editorial standards and corrections.

More in this story

ChatGPT · 5 stories
  1. Coca-Cola Gets Marketing Makeover with OpenAI and Bain & Co's Partnership
  2. Google’s Gemini Takes The Center Stage at MLDS 2024
  3. OpenAI Rolls Out ChatGPT “Memory” Function to Select Users
  4. You are here: Google Gemini 1.5 Crushes ChatGPT and Claude with Largest-Ever 1 Mn Token Context Window
  5. OpenAI Hires JioStar CEO Kiran Mani to Lead Asia-Pacific Operations
About the author

Tech Journalist

Siddharth is a media graduate who loves to explore tech through journalism and putting forward ideas worth pondering about in the era of artificial intelligence.

All stories
Belamy · Daily

The day’s AI news, before your first meeting.