Professor Prasanta Kumar Ghosh from IISc Bangalore has figured out a unique way to collect speech data in different languages and dialects. Travelling to 80 districts in the first phase, showing local people a picture, asking them to describe it and then recording it has given the Google funded Project Vaani around 16,000 hours of speech data. 

The team is open-sourcing the Vaani corpus and is transcribing around 10% of the data manually. The aim is to collect 150,000 hours of speech data from 773 districts of India. “India is not one language and not even several dialects, it’s a continuum of languages, which requires a lot of research and development,” said Ghosh, in an exclusive interview with AIM. 

Apart from this, Ghosh and his research team are also working on RESPIN and SYSPIN, which are both projects related to text-to-speech and speech-to-text models in Hindi, Bengali, Marathi, Telugu, Bhojpuri, Kannada, Magadhi, Chhattisgarhi, and Maithili. 

Funded by Bill and Melinda Gates Foundation, RESPIN stands for Recognising Speech in Indian languages, which is an automatic speech recognition (ASR) project for converting speech to text. Currently, the project is focused on agriculture and finance and the outcome is going to be available as open source. 

SYSPIN on the other hand is an initiative to build a text-to-speech model, which synthesises speech in nine Indian languages – in one male and one female voice. “The mission, going forward, is to cover every language variation of the country,” said Ghosh, highlighting the importance of covering every dialect of every single language. 

AI should solve problems

Ghosh’s interest in Indian languages and AI was born out of his motivation to solve healthcare problems in the country. For several years, he has been working to help people suffering from Dysarthria, a neurological disorder similar to Parkinson’s where people lose their voice. He has been working in collaboration with NIMHANS in Bangalore. 

“We record people with impairments and are building technology that can understand them. Maybe humans can’t, but AI would be able to,” said Ghosh. He said that while collecting the data, he realised that there is no corpus even to build technologies for healthy people. “And then the first thing that came to my mind was the idea to build a good foundational model for them.” 

While doing that, Ghosh realised the importance of making it in Indic languages so that the technology is able to help people who live in different parts of our country but speak in different languages and dialects. 

Apart from healthcare, Ghosh is also focused on using AI for education in India on his project called English Gyani. Funded by the Ministry of Science and Technology, it is also called the IMPRINT project. “We’re building tools to help young jobseekers learn spoken English and comprehension skills right from their homes,” said Ghosh. 

Similar to Duolingo, Ghosh said that the whole application can be controlled through voice, something that his team is working on. Citing examples of people who pronounce the same words differently, he said that it is important to correct people with voice feedback for proper pronunciation and learning. “This will help people get better jobs as well,” he added. 

Focus on responsible AI

“Since AI is almost everywhere now, I think it is important for Indian institutions and researchers to also focus on explainable and responsible AI,” said Ghosh. “It’s not a vertical or horizontal phenomenon anymore, it’s connecting everything,” he added. 

He added that since most of the focus currently is on building models for healthcare, it becomes essential for researchers to focus on how they are collecting data and how these models operate safely. “It’s not just about building a model and making it work, we should also make sure that we are not introducing biases or unnecessarily depriving the model of important information,” said Ghosh. 

Talking about the open source Indic language models, Ghosh said that it is essential to build them since there are a lot of problems that English models cannot solve. “If you go to a district in Rajasthan, none of them speak English or Hindi. But we want to solve their problems, that is what we are aiming to do with our projects,” Ghosh concluded.