Building AI for Bharat and democratising it comes with a steep challenge. India is home to over a hundred major languages and thousands of dialects that current frontier models fail to even recognise, let alone understand accurately.
But IISc researchers want to change that. They have been working on getting AI to listen to Indians in languages that most AI systems have never learned.
The Indian Institute of Science’s SPIRE Lab, in collaboration with non-profit ARTPARK and with support from Google, has released SraVaani 1.0, an open-source speech recognition model covering 65 Indian languages and dialects.
More than 40 of these languages and dialects have no competing speech recognition system with official support, according to the researchers.
On Garo, a language spoken in Meghalaya, SraVaani recorded a word error rate of 9.5% on one benchmark, compared with 69.4% for Gemini 3 Flash.
But benchmarks can only tell so much when there is little to compare against. The bigger story is how IISc got here.
Project Vaani, which forms the data foundation for SraVaani, has collected more than 31,000 hours of speech from around 156,000 speakers across 165 districts. The larger ambition is to take that corpus beyond 150,000 hours, with data from nearly 1 million people across all 773 districts in India.
This is not the kind of problem frontier AI labs typically rush towards. And that is precisely why IISc is doing it.
Prof Prasanta Kumar Ghosh, who leads Project Vaani at IISc, has spent more than two decades working in speech and language technology. While working on healthcare applications with institutions including NIMHANS and St John’s Medical College, he realised that the models worked reasonably well for English. However, they struggled to understand patients from rural backgrounds who spoke with different accents and dialects.
"We [as an ecosystem] are focusing more on a non-Indian language—English," Ghosh says in an interview with AIM. "Who will build technology for languages of ours?"
That became the problem to solve.
Project Vaani began building the underlying speech resources, following earlier work such as RESPIN (Recognising Speech in Indian Languages), a project funded by the Bill & Melinda Gates Foundation in 2020.
The approach was very different from simply asking people to read sentences into a microphone.
Project Vaani uses what the researchers describe as a geocentric data collection approach. Teams go into villages and communities and collect image-prompted speech. Speakers are shown an image and asked to describe whatever comes to mind.
That sounds simple. It isn't.
Recruiting speakers for low-resource languages is difficult. Finding people who can accurately transcribe those languages is even harder. In some cases, the language may not even have a standardised script.
But that’s where images matter.
An image can make it easier for someone to speak naturally. If the image is also representative of a particular region, it can capture references to local festivals, food, culture, and everyday life.
So the resulting dataset is not simply speech and text. It also carries information about the environment and culture in which that speech exists.
That multimodal data became important when the team started building SraVaani.
The model was trained in three stages. First, it was trained on the complete Vaani speech corpus. The second stage included 11.8 million Vaani audio-image pairs to exploit the multimodal nature of the dataset. The final stage involved fine-tuning on transcribed speech from Vaani and other Indian speech datasets, including IndicVoices, RESPIN, SPRING-INX, SPICOR, and SYSPIN.
The result is a multilingual model that can work across 65 languages and dialects, including 20 scheduled languages and 45 regional languages and dialects. And the researchers have open-sourced it under an MIT licence.
That last part is important.
Nihar Desai, who heads the GenAI programme at ARTPARK, does not see SraVaani as a finished product. He sees it as infrastructure that others can build on.
"I think as academia our objective is to open source data sets and models so that others can build on top of it to make it better," he shares.
This is where an academic lab can sometimes do something a commercial AI company cannot.
Google, OpenAI, and other frontier labs have enormous amounts of compute and data. Indian AI companies are also increasingly building speech and language models. But there is little commercial incentive to spend years collecting speech in languages where the immediate market is small.
That is where IISc’s approach becomes interesting.
Desai argues that the goal is not to compete with Gemini or other frontier models.
"We would perhaps go to problems that, let's say, Gemini would perhaps not immediately go towards," he says. "We go to languages which are underserved."
And that could give smaller Indian startups something they otherwise would not have: a starting point.
How Can It be Used
Already, the team is receiving interest from startups and organisations working in these areas. MWire Labs, for instance, has been developing speech technology for Garo, while Shun Labs has built a speech model covering 55 languages using Vaani.
There is also interest in applications around education, healthcare, and counselling.
Desai points to an early exploration by iSaksham, which is looking at using SraVaani for a voice agent that coaches and counsels students in rural Bihar.
But SraVaani is not production-ready.
The researchers understand that the real test begins outside the benchmark. "The major one with language-wise switching, their dialectal variation, the domain-specific words and phrases," notes Sujith Pulikodan, Lead for Speech Technology at ARTPARK.
A farmer's field, a hospital, a government office, and a classroom are very different acoustic environments. A general-purpose speech model can recognise ordinary words reasonably well and still fall apart when someone starts mixing languages or uses specialised vocabulary.
The team also acknowledges that some languages remain far more underserved than others.
Tulu, Beary, and several languages across the Northeast still have limited data. The solution is not another clever architecture. It is more data, better transcription, and more iterations.
That is perhaps the most interesting thing about Project Vaani—going to places which do not have any convenient dataset.
That's also why the university-led approach matters. The team had to persuade people to give their voices, recruit speakers, find transcribers, and in some cases create the skills needed to work with languages that had barely any digital resources.
Google supported the project with compute and data-related resources, while also helping disseminate the dataset and model.
But the researchers are looking for more partners, particularly organisations that can become what Desai calls "strategic adopters"—groups willing to work patiently with the model and turn it into real applications.
For now, SraVaani is a starting point.
Its success will not ultimately be measured by whether it can beat Gemini on a benchmark. It will be measured by whether someone in a village in Meghalaya, a student in Bihar or a speaker of Tulu in Karnataka can eventually talk to an AI system without first having to change the language they speak.
Your reaction
Discussion
What's your take on this story?
No comments yet — be the first to share your view.