Google I/O 2024 wrapped up last week, featuring a range of exciting announcements. During the latter part of the keynote, Google showcased a short video highlighting the power of Gemma in building Indic LLMs.
The search giant introduced Navarasa, a Gemma 7B/2B instruction-tuned model supporting 15 Indian languages and English, developed by Telugu LLM Labs, founded by Ravi Theja Desetty and Ramsri Goutham Golla.
The model was trained on E2E Networks Limited using NVIDIA A100 GPUs which took approximately 44 hours to train the 7 billion model and 18 hours for the 2 billion model.
“When a technology is developed for a particular culture, it won’t be able to solve and understand the nuances of a country like India,” said Harsh Dhand, head of APAC research partnerships at Google.
AIM reached out to Theja and Golla to learn more about their experience. “The Google marketing team invited us to Mysore and booked a five-star hotel, reserving almost all the rooms,” shared Golla, adding that they filmed for about two or three days.
“It was an interesting experience. All those big camera rigs everywhere felt like a movie shoot, with all the big lamps and the makeup crew. They made us feel like celebrities for a day,” he said.
“Google reached out to me after seeing my blog post about Navarasa," said Theja, adding that he had no idea that the video would be shown during the keynote. "We really didn’t expect it to be showcased at the main event of Google I/O. In fact, Ramsri slept through that day," he chuckled.
Hyderabad-based Golla studied and worked in the US for almost eight years before returning to India in 2018. He describes himself as a builder/engineer and loves creating SaaS apps. Golla has successfully developed two AI SaaS apps, with a combined ARR of $100K. Additionally, he takes AI courses on Udemy as well as his own platform.
On the other hand, Theja works as a developer advocate engineer at Llama Index. Prior to this, he served as a senior ML engineer at Glance, where he worked on recommendation systems and GenAI applications.
What is Navarasa?
Navarasa 2.0 is a Gemma 7B/ 2B SFT (Supervised Fine-Tuned model) using Gemma 7B/ 2B base models.
The model's generative capabilities have been enhanced to cover a total of 15 Indian languages. This expansion was achieved by translating the alpaca-cleaned-filtered dataset into six additional Indian languages: Marathi, Urdu, Konkani, Assamese, Nepali, and Sindhi.
The 15 languages supported by Navarasa 2.0 besides English are: Hindi, Telugu, Tamil, Kannada, Malayalam, Marathi, Gujarati, Bengali, Punjabi, Odia, Urdu, Konkani, Assamese, Nepali, and Sindhi.
The model is also available on IndicChat, a playground for open-source Indic LLMs. Built on Hugging Face’s HuggingChat, this platform hosts Indic AI models for users to chat and test. “Many people started using Navarasa through IndicChat, as it is hosted there with a chat interface. A considerable number of users accessed it from there,” said Golla.
Speaking of the advantage Gemma has over Llama 2 and 3, Golla said, “A fundamental difference between Llama and Gemma is that with Llama, you had to fine-tune a model for each individual language. However, with Gemma, you only need to fine-tune once and can combine all the datasets created for Llama.”
“On a higher level, the Gemma tokeniser includes tokens for most Indian languages, providing strong representations for these tokens. In contrast, the Llama3 tokeniser supports only a few languages, and its quality of support is not as robust,” said Theja.
OpenAI’s Love for India is Unmatched
OpenAI's latest model GPT-4o has made considerable improvements when it comes to Indian languages. It supports over 50 languages and has notably optimised token usage for Indian languages, reducing Gujarati by 4.4 times, Telugu by 3.5 times, Tamil by 3.3 times, and Marathi and Hindi by 2.9 times.
“One of the biggest updates GPT-4o got is a brand new tokeniser and an extended vocabulary size of 2,00,019 tokens as compared to 1,00,277 tokens in GPT-4. This means we have much better support for many new languages and yes some of them are Indian,” said Abhishek Upperwal, founder of Socket AI labs, which recently launched Pragna-1B, India’s first open-source multilingual model.
“OpenAI is paying attention to Indic languages! My mom has been struggling to learn English from Telugu - now I can make a simple wrapper using the new GPT-4o model,” wrote a user on X.
“OpenAI has done well even for Indic languages in terms of tokenisation efficiency and cost etc,” said Golla. He added that in the past, he had to switch between Anthropic and OpenAI for his SaaS app development, as Anthropic was deemed more efficient with Indic languages compared to GPT-4.
Theja had a different perspective, saying that during the Spring Update, OpenAI didn’t demonstrate how GPT-4o would sound in Indian languages. “If you've tried the audio feature in the app for Indian languages, it doesn't sound good at all. I can't even understand what it's saying,” he said.
Indian AI startup Sarvam AI is currently working on an Indic Voice LLM, while Hanooman and Krutrim do not include a voice modality yet. However, they are expected to add this feature soon.
Your reaction
Discussion
What's your take on this story?
No comments yet — be the first to share your view.