AI’s biggest bottleneck is no longer the model. It’s the infrastructure needed to run it. As enterprises race to deploy AI applications, cloud bills are soaring, data centres are struggling to keep pace, and inference costs continue to climb.
The scale of investment reflects the challenge. Goldman Sachs estimates that global spending on AI compute, data centres, and power infrastructure will reach $7.6 trillion between 2026 and 2031.
Meanwhile, Microsoft, Amazon, Alphabet, and Meta are expected to spend a combined $725 billion on capital expenditure this year alone, much of it directed toward AI infrastructure.
Rather than building ever-larger data centres, some technology companies are pursuing a different strategy: moving AI workloads closer to where developers work.
At its Build 2026 conference, Microsoft made a strong case for local AI computing, unveiling the Surface Laptop Ultra and Surface RTX Spark Dev Box, both powered by NVIDIA's RTX Spark platform and designed to run advanced AI workloads directly on the desktop.
The chip maker claims that RTX Spark chip combines CPU, GPU and AI processing in a single package, supported by a unified memory architecture, delivering up to one petaflop of AI performance. The platform combines NVIDIA’s Blackwell RTX architecture with Arm-based CPU cores and up to 128 GB of unified memory, allowing AI workloads to run directly on Windows devices.
Microsoft is also bringing new AI models to Windows. During his keynote, Satya Nadella announced AION 1.0 Instruct, which he described as a great reasoning model, alongside AION 1.0 Plan, a planning model for agentic applications.
“You can give it tools access and build fully agentic applications without having to run a trip to the cloud,” Nadella said.
He further emphasised that the idea of unmetered intelligence running at the edge is becoming increasingly relevant.
Microsoft also announced that Windows will be available on NVIDIA’s DGX Station systems.
Nadella said the system is capable of running a one-trillion-parameter AI model locally, a level of computing power that was once reserved for large-scale AI infrastructure.
“It is close to what we perhaps had when we built GPT-3, one of the first supercomputers,” he said. “It is crazy to think we have come this far, where you can have a data centre on your desktop.”
The device ships with a developer-optimised version of Windows 11, pre-loaded with Python, Node, Visual Studio Code, and GitHub Copilot.
Ashish Banerjee, Senior Principal Analyst at Gartner, tells AIM that local inference is cheaper only when used heavily. Otherwise, the cloud remains the more cost-effective option.
“Shifting from per-token pricing to fixed hardware cost works for high-frequency, predictable workloads. That’s the bet behind devices like Microsoft’s RTX Spark Dev Box,” he says.
Banerjee adds that once you factor in hardware costs, utilisation, maintenance, and model limitations, cloud remains superior for bursty workloads and for anything requiring frontier models.
He argues that local AI is best suited for high-volume, everyday tasks where good-enough performance is sufficient, while cloud-based AI is better for workloads that require the most advanced capabilities and flexible scaling.
Meanwhile, beyond hardware, NVIDIA is also investing in software.
It is bringing its OpenShell framework to Windows, built on new security and containment capabilities introduced by Microsoft. OpenShell is NVIDIA’s framework for connecting AI agents with applications and system tools.
The companies said AI agents such as Hermes Agent and OpenClaw will run on Windows using OpenShell, allowing users to perform tasks locally instead of relying on cloud services.
Similarly, Google has also launched Gemma 4 12B, a new open-weight multimodal model designed to run locally on consumer hardware while supporting text, image and audio inputs through a single unified architecture.
While local inference is becoming practical, the cloud is unlikely to disappear. Training frontier models and running the most demanding reasoning workloads will still require large-scale AI infrastructure, making hybrid architectures the more likely future.
Banerjee said local AI won’t fix the data centre crunch; it will just shift the load to local devices for simple tasks. The main demand drivers—frontier models, enterprise inference, and workflows—still require the cloud.
“Data centre energy demand is still set to surge, not decline.”
Perplexity Is Making the Same Bet
Microsoft is not alone in this thinking. Perplexity AI made a closely related announcement at Computex 2026 in Taipei, also on June 2, when CEO Aravind Srinivas took the stage alongside Intel CEO Lip-Bu Tan to unveil what the company calls the first hybrid local-server inference orchestrator.
“We believe that the future is going to be a hybrid of server and local for agentic inference. The data centre is coming to your laptop. People are tired of token maxing and paying half a billion dollars a month,” said Srinivas in an interview with CNBC.
The system, which is coming to Perplexity’s Personal Computer product in July, automatically decides in real time which parts of an AI task should run on a user’s local device and which should be routed to more powerful cloud models. It does this mid-task, without the user having to configure anything. Sensitive or simpler tasks stay on the device, while complex reasoning and large retrieval jobs are routed to the cloud.
“The race for local compute is on. As chips advance, only Perplexity has the agentic harness and applied inference engineering for truly seamless orchestration,” the company said in its announcement.
The company added that the approach could help reduce reliance on centralised AI infrastructure by shifting sensitive and routine workloads to users’ devices, while reserving cloud resources for more demanding tasks.
“Perplexity’s approach – dynamically deciding what runs locally versus in the cloud – reflects where the market is going. The real battleground will be orchestration: policy-aware, context-aware, and model-aware routing across device, edge, and cloud,” says Banerjee.
The timing is not accidental. Cloud AI inference is expensive. Running large models continuously through third-party APIs at the scale that modern software demands adds up fast.
Even for smaller developers, unpredictable token costs are a genuine constraint on how aggressively they can build with AI. GitHub Copilot’s pricing change is the latest example of that.
NVIDIA CEO Jensen Huang said at GTC that Spark marks the start of a long-term reinvention, imagining a future with an AI supercomputer in every home running countless assistants.
The industry spent the past decade moving computing workloads into the cloud. That trend may be reversing now. As models become more efficient and hardware more capable, the next battleground may not be the data centre alone, but the devices sitting on users’ desks.
Your reaction
Discussion
What's your take on this story?
No comments yet — be the first to share your view.