AI firms scaling voice AI in India face an enormous wall of linguistic complexity.
Over 85% of Indians are not fluent in English, and nearly three-quarters of new internet users actively seek content in their native languages, per Gnani.ai. For voice AI to function at scale across the country, it must work in the languages people actually speak.
Yet the gap between that ambition and reality is stark.
A logistics firm working with conversational voice AI-focused HRtech startup Hunar.ai in Pune piloted a voice bot for driver onboarding and found that while English speakers faced no issues, rural drivers using accented Hindi or regional dialects triggered persistent recognition errors. Misinterpretations caused frustrating menu loops, with 70% of calls escalating to human agents.
This is not an isolated case. It reflects a wider failure mode in deploying voice AI across a country as linguistically layered as India.
Four Problems
Raoul Nanavati, Co-founder and CEO of voice AI startup Navana.ai, identifies four distinct challenges that global providers consistently miss.
The first is linguistic diversity: India has thousands of dialects within its 22 official languages, and a model that cannot distinguish Bihari Hindi from Rajasthani Hindi will fail in the field.
The second is code-switching. "Real Indian speech mixes languages mid-sentence constantly: Tamil-English, Hindi-English, Marathi-Hindi. No monolingual model handles this well,” he tells AIM.
“The third is environmental noise: rural and semi-urban callers phone from noisy environments on low-quality microphones, and lab-trained models collapse in the field," he adds.
The fourth is domain vocabulary—phrases like "RD khatam ho gayi" or crop insurance scheme names in local dialect simply do not exist in general training corpora, and domain-specific understanding requires targeted data collection from actual users in actual contexts.
Nagasanthosh Josyula, Co-founder of Tablesprint, which recently launched voice AI platform Fernor, frames the same problem in operational terms. Code-switching—conversationally switching between languages—is how most Indians actually speak. Still, most ASR (automatic speech recognition) systems, trained on one language at a time, may miss out on critical information.
"For businesses, this makes it difficult to deliver consistent, scalable voice experiences across regions without deeply localised models," Josyula says.
“So what we do is to start off with, we don't train models because as of today what we see is we don't need that to do, because there are organisations who provide these kinds of voices, speech‑to‑text, text‑to‑speech. What we do is we have partnered with a lot of these providers like 11 Labs, Sarvam,” says Sai Krishna, Director of Engineering, TestMu.ai.
“Add our own guardrails, evaluation layer, personas, noise environments, and switching logic on top. This dynamically switches providers/models based on language and dialect (e.g., Azure or Sarvam for regional Indian languages; 11 Labs for very natural voices). Domain (e.g., Deepgram’s healthcare/pharma models when needed, healthcare-tuned LLMs when the domain is medical). We emphasise that no single provider fits all use cases, so they maintain multi-partner integrations and choose per-region/per-domain,” he adds
Even when transcription succeeds, the next layer can fail.
A speaker from rural Rajasthan using Hindi will have tone patterns and vowel sounds quite different from Delhi Hindi. An ASR model trained on urban data assigns low confidence scores to that audio, leading to wrong transcription. In a lead qualification flow, that could mean a prospect's budget or city is misheard, a valid lead is dropped, or a cold lead lands on a sales representative's desk.
And even when the voice agent transcribes speech correctly, it can still misread what the caller actually means, because certain phrases carry different meanings depending on the region. A model untrained on those nuances will interpret them at face value. The downstream consequences range from lost leads and wrong bookings to frustrated customers.
"Many of our customers were incorrectly classified as needing a loan due to false positives. In one analysed call, background noise from a cow was mistakenly interpreted as customer acceptance, leading the agent to assume the person required a loan," says Srinivasan Sekar, Director of Engineering, TestMu.ai
How Indian Startups are Closing the Gap
A handful of Indian startups are beginning to address these challenges in meaningful ways.
Gnani.ai, which processes over 30 million voice interactions daily across more than 12 languages and serves over 200 enterprise clients, recently introduced Vachana TTS, a text-to-speech model capable of replicating human voices across 12 Indian languages using less than 10 seconds of reference audio, retaining key voice attributes such as tone, pitch, and speaking style.
Fernor AI's pipeline approach is instructive. Rather than relying on off-the-shelf ASR, the platform fine-tunes its speech-to-text layer specifically for Indian accents, code-mixed speech, and domain vocabulary.
As Josyula notes, a generic ASR model might transcribe "EMI" as "Emmy", and such errors break all downstream logic. Beyond transcription, Fernor trains task-specific NLU models per use case rather than relying on a general LLM for everythingidentify to enable lead qualification bots to identify hesitation or buying signals with precision that a generic model cannot match.
For open-ended conversation turns, large foundation models step in to handle objections and edge cases that scripted flows cannot cover. The platform also maintains separate TTS voice profiles for each language and region—calibrated to sound natural.
"A bad voice kills engagement, especially in trust-sensitive industries like insurance or healthcare," Josyula notes.
Advanced deployments are also building audio evaluation agents—secondary models that run alongside the main conversation, listening to both sides of the call and analysing tone, signs of frustration, filler words, interruptions, pacing, and talk-to-listen ratios. These systems build context graphs that map topics, emotional states, and intents over time, feeding back into learning and allowing the system to adapt behaviour for different regions, pitching opportunities differently, or emphasising job stability versus incentives depending on local needs.
From Response to Resolution
The industry is also moving beyond measuring bot responses, determining whether it completed the task.
"First-generation bots answered questions. The measure of success was 'did it respond?' That's not an enterprise outcome. The agent paradigm measures completion: did the loan application get submitted, did the policy renew, did the EMI get collected? That shift from response to resolution is what makes voice AI genuinely valuable rather than a cost optics exercise," Nanavati explains.
The infrastructure required to deliver that resolution is fundamentally more demanding. ASR, TTS, dialogue management, core system integrations, authentication, compliance guardrails, and human handoff logic all need to work together under two seconds of latency.
Varun Viswanath, Agent Engineer at Prodigal, a startup building AI agents for loan servicing and debt collections, offers a window into what a production-grade pipeline looks like at scale.
The company uses Deepgram's Nova and Flux models for speech-to-text, purpose-built for real-world, noisy, conversational audio, combined with Anthropic's Claude Sonnet for contextual reasoning and instruction-following in complex, multi-state conversations, and ElevenLabs on the output side for fine-grained control over pacing, tone, and accent. On economics, Viswanath is direct. A voice bot is paid only for the hours it actually works. This structural difference drives a 60–65% improvement in margin on the calling cost base.
"The companies seeing the strongest financial impact and efficiency improvements are not the ones that deployed Voice AI as a cost-cutting measure. They're the ones that redeployed human capacity toward higher-value work at the same time and measured the ROI on both," he says.
The Structural Opportunity
Nanavati sees the opportunity ahead as structural rather than incremental. "India's formal financial system is expanding into populations that have never interacted with a bank digitally. They speak 22-plus languages and aren't comfortable with apps or forms, but they can speak. Voice AI is the only scalable bridge. That's not a niche. That's the core of India's next decade of financial inclusion."
Current deployments already hint at the scale of what is possible. One in four borrowers in loan collection deployments commit to repayment on the call itself, per Navana.ai. "That's a behaviour change that's hard to ignore," he says.
The next evolution will be full agentic workflows: multi-step processes that a voice agent runs autonomously across systems, without a human in the loop. For that to work in India, at the scale India demands, the language barrier must be solved first.
Your reaction
Discussion
What's your take on this story?
No comments yet — be the first to share your view.