Article complete

Indian Startup Sarvam AI Launches Hindi LLM, OpenHathi

The model demonstrates robust performance across various Hindi tasks, comparable to, if not surpassing, GPT-3.5.

A cartoon elephant sits on the back of a pink llama against a light green background.

Key takeaways

  • Sarvam AI launches OpenHathi, the first Hindi LLM, showcasing performance comparable to GPT-3.5.
  • Model developed using Llama2-7B, optimized for cost-effective deployment in Indic languages.
  • OpenHathi demonstrates superior performance in Hindi tasks, surpassing GPT-3.5 evaluations.
  • Collaboration with AI4Bharat and KissanAI enhances model training with real-world conversational data.
  • Model supports both native and Romanised Hindi scripts, broadening accessibility and usability.

Takeaways drafted with AI from the article text.

Indian AI startup Sarvam AI has released OpenHathi-Hi-v0.1, the first Hindi LLM in the OpenHathi series. Developed on a budget-friendly platform, the model, an extension of Llama2-7B, boasts GPT-3.5-like performance for Indic languages.

OpenHathi, featuring a 48K-token extension of Llama2-7B’s tokenizer, underwent a two-phase training process. The first phase focused on embedding alignment, where Hindi embeddings were randomly initialized and aligned, followed by bilingual language modelling to teach the model cross-lingual attention across tokens.

The model demonstrates robust performance across various Hindi tasks, comparable to, if not surpassing, GPT-3.5, while maintaining English proficiency. Sarvam AI's evaluation includes non-academic, real-world tasks alongside standard Natural Language Generation (NLG) tasks. Evaluations against GPT-3.5 generation with GPT-4 as the judge revealed superior performance in Hindi, both in native and Romanised scripts.

Developed in collaboration with academic partners at AI4Bharat, who contributed language resources and benchmarks, and fine-tuned in partnership with KissanAI, the model leverages conversational data from a bot interacting with farmers in multiple languages. 

Stemming from the founders' background in AI research and digital infrastructure development, the startup aims to cater to India's unique needs, prioritising Generative AI integration for diverse Indian languages and fostering collaborations with enterprises for domain-specific AI model development using their data.

You can find the base model here. 

Published

Spotted an error? Tell our editors. Read our editorial standards and corrections.

About the author

Tech Journalist

Siddharth is a media graduate who loves to explore tech through journalism and putting forward ideas worth pondering about in the era of artificial intelligence.

All stories
Belamy · Daily

The day’s AI news, before your first meeting.