The GPU shortage has become one of India’s biggest hurdles in building AI infrastructure. Even with around 62,000 GPUs under the IndiaAI Mission ecosystem, the country’s compute capacity remains far smaller than SpaceX’s 3,25,000-GPU cluster.
However, hyperscale networking software company Arrcus’ CEO Shekar Ayyar believes the solution is not simply adding more GPUs but making every GPU work harder through intelligent networking.
In an exclusive interview with AIM, Ayyar says that “GPUs are just the tip of the iceberg” and that much of the action takes place at the networking layer. He believes smarter networking can help countries like India make better use of their AI infrastructure, where GPUs, power, and budgets remain constrained.
Arrcus operates at the networking layer, allowing data centres to intelligently direct workloads instead of treating every AI request equally.
Its flagship network operating system, ArcOS, determines whether a workload truly requires a GPU or whether it can be handled by a data processing unit (DPU) or another accelerator. That routing decision alone, Ayyar says, can improve GPU utilisation by as much as 30–40%.
Apart from improving GPU utilisation, ArcOS allows enterprises and cloud providers to build and manage large-scale networks across data centres, edge locations, and cloud environments. It can automatically route outbound traffic through lower-cost cloud exit points, helping customers reduce expensive egress charges.
Built on ArcOS, the Arrcus Connected Edge (ACE) platform supports both AI training and inference workloads. For AI training, the company provides Ethernet-based networking for large GPU clusters, which Ayyar says can serve as an alternative to NVIDIA’s NVLink.
Inference Delivery
For inference, AINF (AI Inference Networking Fabric) is the company’s latest offering. The platform intelligently routes AI requests across edge, cloud, and data centre infrastructure based on factors such as latency, GPU availability, and data sovereignty. It also integrates with inference frameworks including NVIDIA Dynamo, vLLM, SGLang, and NVIDIA Triton.
Ayyar compares it with earlier-era Content Delivery Networks (CDNs) that ensured websites and streaming services remained available globally. AI, he believes, demands a fundamentally different networking model.
“The world is moving from content delivery to inference delivery. Therefore, networks need to go from Content Delivery Networks to Inference Delivery Networks.”
Ayyar adds that organisations should begin preparing for inference-centric AI infrastructure rather than waiting for demand to force the transition.
Optimising GPUs
Ayyar believes India’s constraints could ultimately become an advantage. “India has always been a ‘do more with less’ country,” he says.
He believes many operators are currently hyper-focused on following Western infrastructure strategies by acquiring GPUs first and worrying about efficiency later. “I still don’t see yet a lot of the leading operators and data centre providers really taking this to heart and starting to innovate their infrastructure to be a ‘do more with less’ architecture.”
Ayyar believes India should design infrastructure around local realities, including traffic patterns, edge deployments, and cost sensitivity.
India’s AI Infra Market
In March earlier this year, Arrcus partnered with Singapore- and Gurugram-headquartered connectivity provider Lightstorm, which Ayyar describes as one of the company’s key strategic relationships in India to enable high-performance connectivity for distributed AI inference and training workloads.
The company is also working with several other organisations, including large financial institutions, deploying its technology in both conventional and AI data centres.
Ayyar believes India’s biggest opportunities lie across AI-ready data centres, new AI cloud providers, and telecom operators modernising their networks. He identifies traditional telecom companies such as Jio, Airtel, and Vodafone Idea as key targets for Arrcus’s expansion.
Competing Against Incumbents
On how Arrcus differentiates itself from established networking vendors such as Cisco and Juniper, Ayyar says flexibility and programmability are the company’s biggest strengths.
Rather than replacing existing infrastructure wholesale, Arrcus targets organisations looking to introduce new revenue-generating services such as AI-as-a-service, multi-cloud networking, VPNs, security services, or network slicing. He argues that programmable infrastructure allows telecom operators to monetise networks that have otherwise struggled to generate returns.
Moreover, Arrcus also promises lower infrastructure costs.
Ayyar says customers can reduce networking costs by 30–40% compared with traditional networking deployments while simultaneously requiring fewer hardware components because more intelligence is built into the operating system.
While many AI companies are struggling to reduce token costs through model optimisation, Arrcus approaches the problem from the infrastructure layer.
“We’re not actually going in and optimising the LLM itself,” Ayyar says. The company analyses bandwidth availability, compute utilisation, network topology, and power consumption to optimise the infrastructure supporting those models.
Your reaction