Is India’s AI Infrastructure Ready for Agents?

Everyone is chasing better AI models. Ritesh Dhoot, EVP of Engineering at Neysa, believes that’s the wrong focus. At MLDS 2026, India’s biggest AI agentic conference organised by AIM, he pointed to infrastructure as the real constraint holding back AI’s next phase.

India’s AI momentum is undeniable. With billions in committed investments and nearly half of enterprises already running live models, the country is moving fast. But Dhoot warns that speed without structural readiness could backfire.

“We are all talking about models,” he said. “What we are not asking is, do we have the infrastructure in India which will support our AI?”

Referring to commitments made during the India AI Impact Summit, he said this is not theoretical capital. “This is money in motion. It’s operational,” he added, pointing to the urgency of aligning infrastructure with ambition.

Chatbots to Autonomous Agents

AI has moved beyond simple interactions to autonomous agents, and the growth is exponential.

Dhoot explained that while a single AI interaction today may involve around 2,000 tokens, agent-based workflows can demand up to 60,000 tokens per task. “That’s a 30x increase.”

And that’s just the beginning. Enterprises are expected to deploy hundreds of concurrent agents to handle compliance, customer support, data transformation, and more, each operating with massive context windows.

“The scale is humongous,” Dhoot remarked. “We are no longer building for 5x or 10x growth. We are already looking at 30x today.”

Infrastructure Evolution

Dhoot made his case, arguing that every major leap in the past succeeded only after the underlying systems were rebuilt.

“We moved from telegraph and telephony to the internet. That transition required a complete infrastructure overhaul, including TCP IP and packet switching. Everything changed,” he said. “AI is following the same curve, just in a much more condensed timeframe.”

From simple chat interfaces to multi-agent ecosystems, AI’s evolution is outpacing the infrastructure it depends on. 

The 5 Bottlenecks That Could Break AI

Dhoot outlined key challenges that could limit AI’s scalability if left unaddressed.

Compute and latency remain critical. “If each inference takes 800 milliseconds and you run 30-50 of them, you’re looking at 40 seconds for an output,” he explained. “That’s expensive automation.”

Memory is another constraint. With hundreds of agents running simultaneously and context windows stretching into millions of tokens, the demand for high-bandwidth memory is staggering. “You’re talking about terabytes of memory,” he stressed.

Security, too, is evolving. “We’ve moved from SQL injection to prompt injection,” he said, highlighting the need for AI-native security frameworks.

At the same time, cost unpredictability and lack of observability further complicate deployment. Without visibility into how agents behave, enterprises risk runaway costs and inefficiencies.

Finally, data sovereignty remains non-negotiable, especially in sectors like BFSI and healthcare. “Data sanctity and regulatory compliance will define how AI scales in India,” he added.

Rethinking the Stack

Dhoot introduced the AI infrastructure framework developed by Neysa, describing it as a three-layered architecture designed for agentic AI.

At the top sits compute, followed by a high-speed communication fabric, and at the base, a robust memory layer. Together, these components function like a brain, a synapse, and working memory.

“You cannot have a fast brain with a slow synapse,” Dhoot explained. “And without sufficient memory, you cannot store state. All three must work together.”

He also highlighted features like an AI firewall for customisable security controls and an observability layer that provides deep visibility into system behaviour.

A Call to Rebuild for the Future

Dhoot made it clear that legacy infrastructure will not support the future of AI.

“Whatever has been designed in the past is not going to scale for the future,” he said. “You have to start migrating your workloads.”

As enterprises rush to build AI-driven systems, he urged them to pause and assess whether their infrastructure can support their ambitions.

Dhoot believes that trust in AI infrastructure providers will become increasingly important. “The pace of change today is fundamentally different,” he said. “You need systems that are reliable, secure, and built for scale.”

AI agents may be the face of the next technological wave, but as Dhoot argues, they are only as strong as the systems beneath them. 

And in India’s race to lead in AI, that distinction could make all the difference.