The question most enterprises today are grappling with is where production AI should run once experimentation ends.
Public cloud remains the fastest path to prototyping AI applications. But for organisations operating in banking, financial services and insurance (BFSI), government and other regulated sectors, production deployments are increasingly pulling infrastructure decisions back towards private cloud, driven by governance, security and the need for predictable infrastructure costs.
Broadcom’s latest Private Cloud Outlook 2026 found that 56% of enterprises globally are already running or planning production AI inference on private cloud, while public cloud usage for production inference fell to 41%, down 15 percentage points year over year.
The report also found that 62% of IT leaders are highly concerned about generative AI infrastructure costs, while 36% say AI is creating new requirements around privacy, security and risk management.
Why Indian Enterprises Need Private Cloud More Than Ever
India’s regulated sectors are moving in the same direction.
The Reserve Bank of India’s FREE-AI Committee found that around 21% of surveyed banks, NBFCs, and financial institutions are already developing or deploying AI systems, with adoption concentrated among large public and private-sector lenders.
At the same time, India’s Digital Personal Data Protection Act has heightened scrutiny of how enterprises process personal data, while the RBI has proposed AI governance guidelines that require banks to establish board-approved AI risk management frameworks, independently validate AI and machine learning models, and strengthen cybersecurity controls for customer-facing generative AI systems.
Against this backdrop, Broadcom is positioning VMware Cloud Foundation (VCF) as the infrastructure layer for production AI, especially with its latest VCF 9.1 iteration.
In an interaction with AIM, Dilpreet Bindra, Senior Director, Engineering, VMware Cloud Foundation at Broadcom, says conversations with Indian customers consistently revolve around one priority: keeping AI close to enterprise data.
“Each one of these enterprises needs to work through its own logic around how it's building out its AI story,” Bindra adds. “Proprietary data is their business, so bringing it anywhere close to the public cloud would be extremely detrimental to their overall business.”
Increasingly, organisations are adopting hybrid architectures, keeping sensitive AI workloads and proprietary data on private infrastructure while continuing to use public cloud for development, burst compute and less regulated applications.
According to Bindra, enterprises are increasingly looking beyond AI experimentation and focusing on operationalising models without exposing sensitive business information or losing control over infrastructure.
Broadcom argues VMware Cloud Foundation 9.1 is designed for exactly that transition. The latest release combines virtual machines, Kubernetes, AI workloads and security into a unified private cloud platform while supporting CPUs and GPUs from AMD, Intel and NVIDIA.
The company says the platform can reduce server costs by up to 40%, lower the total cost of ownership by up to 39%, reduce Kubernetes operational costs by up to 46%, and deliver cluster upgrades four times faster.
The platform has become central to Broadcom’s infrastructure software strategy since its $69 billion acquisition of VMware in November 2023.
Infrastructure Software now contributes roughly one-third of Broadcom’s overall revenue, although the company does not disclose VMware- or VCF-specific revenue.
Broadcom also says more than 90% of VMware’s top 10,000 enterprise customers have purchased VMware Cloud Foundation.
Private Cloud to Also Help With Tokenomics
Bindra points to four themes that repeatedly emerged regarding why enterprises are moving production AI to private infrastructure: protecting proprietary data, achieving predictable infrastructure costs, strengthening governance of AI systems, and improving utilisation of increasingly expensive compute resources.
He believes governance will become even more important as enterprises deploy AI agents and autonomous workflows inside business operations. He also states that organisations need infrastructure-level controls to ensure AI agents and large language models operate within approved policies, rather than relying solely on application-level safeguards.
For Indian enterprises, particularly in the BFSI sector, he argued that the business case extends beyond compliance.
Infrastructure ownership provides what Bindra describes as “investment protection”, allowing organisations to build AI capabilities without continuously worrying about token consumption or unpredictable cloud costs.
“You can do complex tasks in your environment without worrying too much about the overall tokenomics, unlike using cloud-based LLMs,” he says, adding that dedicated infrastructure allows enterprises to focus on adapting models to their own data rather than managing variable infrastructure bills.
Resource efficiency is another major focus of VCF 9.1.
One of the platform’s key additions is intelligent memory tiering, which allows enterprises to extend memory capacity with NVMe storage. Frequently accessed memory pages remain in DRAM, while less active pages are moved to NVMe, reducing the need for expensive memory upgrades.
“Because NVMe is so fast, even when the system needs to retrieve data from it, the application barely notices. We’ve been able to maintain application performance while extending memory capacity this way,” Bindra explains.
He argues that extending existing server capabilities, rather than purchasing additional hardware, becomes increasingly valuable as memory prices and AI infrastructure costs continue to rise.
GPU utilisation is another area Broadcom is targeting.
According to Bindra, the company has worked closely with NVIDIA to support both virtual GPU partitioning for smaller AI workloads and full GPU passthrough for large-scale inference deployments. The objective is to improve utilisation across experimentation, model development and production inference without leaving expensive accelerator hardware idle.
The Competition Around Enterprise AI Infrastructure
Despite the renewed momentum around private cloud, Broadcom is entering an increasingly competitive market.
Vendors, including Nutanix, Red Hat OpenShift Virtualisation, Microsoft Azure Local, HPE GreenLake, Dell APEX Cloud Platform and Platform9, are all positioning their platforms as the foundation for enterprise AI infrastructure. Most are competing on operational simplicity, Kubernetes integration, hardware flexibility and total cost of ownership.
Broadcom also faces scrutiny from existing VMware customers. Gartner Peer Insights reviews consistently praise VCF’s stability, integrated management and automation capabilities, but recurring criticisms include licensing costs, customer support and the complexity of upgrades in large deployments.
Within the VMware community, administrators have also pointed to the growing infrastructure footprint required by newer VCF services and continued concerns around licensing following Broadcom’s acquisition of VMware.
Bindra, however, believes enterprises are increasingly willing to make that investment because they expect the same cloud-native experience inside their own datacentres that developers receive from hyperscale cloud providers.
Delivering that, he says, requires platform engineering teams capable of integrating compute, storage, networking and Kubernetes into a unified operating environment.
Your reaction
Discussion
What's your take on this story?
No comments yet — be the first to share your view.