Inside the Mission to Boost AMD's AI GPUs

On paper, AMD’s AI chips often rival or beat NVIDIA’s, but in reality, they have historically been held back by a fragmented software ecosystem that requires an army of elite engineers to optimise. 

Instead of replicating NVIDIA’s parallel computing platform CUDA and fighting the software war alone, AMD is leaning on highly specialised startups like MangoBoost and TensorWave. These partners are doing the heavy lifting to unleash AMD's silicon and make it enterprise-ready, giving the market a genuine alternative to NVIDIA.

Optimising AMD’s Performance

Take, for instance, MangoBoost, which focuses on optimising the performance of AMD's AI GPUs. The company is backed by AMD and works closely with it to build a full-stack solution.

Jangwoo Kim, the CEO of MangoBoost, stated at a software workshop hosted by AMD that the semiconductor giant’s customers are suffering from software, even though he claimed they were better than NVIDIA GPUs on paper, and his company is attacking these problems head-on. 

For one, he stated that properly engineering and optimising an AI software stack requires the expertise of hundreds of top-tier system software engineers. Most enterprise and operational companies lack these highly specialised resources, preventing them from making the necessary performance fixes. 

Image credits: MangoBoost

Also, customers often piece together open-source software that fails to deliver optimised performance. The market is flooded with too many disjointed software products, leaving companies confused about what to use or how to integrate them into a seamless, push-button “full stack” solution.

“We’re showing that we can outperform competitors using AMD GPUs, and we have a strong chance as AMD GPUs have larger memory and are actually better in certain specs, and come with very aggressive pricing,” Kim told AIM on the sidelines of the workshop in Singapore. “I don’t see any reason to avoid buying AMD GPUs, and our solutions together actually show real-world performance gains.”

While there are plenty of specialised open-source tools available today—such as vLLM and SGLang for inference, Megatron and Primus for training, or Optuna and Ray for tuning—these toolkits are heavily fragmented and require extensive manual integration. 

Kim explains that MangoBoost’s LLMBoost solves this fragmentation challenge by acting as an intelligent add-on software layer designed specifically to boost this existing open-source ecosystem. Rather than relying on slow, manual brute-force optimisation, LLMBoost provides full-stack automated tuning that maximises GPU performance instantly.

Furthermore, LLMBoost bridges the massive gap between high-level application APIs and physical infrastructure by solving the enterprise-readiness problem. 

Where open-source tools often lack reliability and support, LLMBoost delivers commercial support, bug fixes, and regular updates. It introduces critical enhancements over baseline tools, including smart inference features (like KV-cache and context-awareness), enhanced telemetry, and smart collective communication over base communication kernels like ROCm/RCCL. 

To eliminate deployment friction, MangoBoost packages all this heterogeneous GPU optimisation into a containerised full-stack software package that is completely turnkey and easy to deploy at top performance.

On AMD’s latest MI355X GPUs, LLMBoost demonstrates clear performance improvements over standard vLLM baselines. 

Local multi-GPU benchmark tests running the DeepSeek-v3.2 and Qwen3-Next models show notable increases in generation throughput. The software also significantly lowers the time-to-first-token (TTFT) latency, reducing the delay before the models begin generating text.

Image credits: MangoBoost

In training environments, LLMBoost's network communication optimisations help close the hardware gap. When scaling AMD MI300X GPUs across multiple nodes for complex training workloads, the resulting runtime trendlines match and slightly edge out NVIDIA's H100 systems.

Beyond these specific models, MangoBoost’s automated tuning replaces manual parameter configuration, which generally improves baseline speeds across both training and inference. The software also maintains linear scaling across multiple servers and demonstrates measurable improvements in power efficiency under capped wattage budgets.

Beyond just software, MangoBoost develops its own AI infrastructure hardware, including custom network accelerators and DPUs (data processing units), which provide a synergistic performance boost when co-designed with the LLMBoost stack.

Having said that, while AMD's ROCm serves as the essential foundational software layer for its Instinct GPUs, it has historically faced an uphill battle for adoption against NVIDIA's deeply entrenched CUDA ecosystem. “They just love CUDA,” Kim quipped.

To shift this, Kim proposed AMD prove its capabilities through concrete results rather than just spec sheets. “We have to change it together,” he said, explaining that as companies like MangoBoost consistently demonstrate real-world performance gains, the broader market will become more open to adopting the ROCm environment. 

Beyond just building tools on top of the stack, Kim confirmed that MangoBoost is collaborating directly with AMD's teams on low-level kernel optimisations to improve ROCm at scale. These attempts aim to accelerate communication libraries like RCCL to enable faster, more efficient multi-node scaling.

Addressing the current state of the AI hardware market, he stressed the need to establish a viable alternative to NVIDIA. “We should make it happen, otherwise one company will overwork, and we don’t like that,” Kim stated. 

He admitted that selling alternative hardware and software solutions still encounters resistance because there are “so many NVIDIA fans,” but affirmed that MangoBoost is highly aware of the ecosystem's bottlenecks and is working closely with AMD to fix them.

AMD Exclusivity

At the workshop, MangoBoost was joined by TensorWave, an exclusive neocloud provider for AMD GPUs. Founded in 2023, TensorWave provides access to the latest AMD hardware and has built a massive cluster of over 8,000 GPUs for AI training. The company raised a $100 million in Series A in 2025, co-led by AMD Ventures.

According to TensorWave co-founder Piotr Tomasik, the market was broadly underestimating AMD's MI300, while enterprises were simultaneously desperate for an alternative to a single-vendor hardware ecosystem. “We set out to be like the CoreWeave of AMD, purpose-built, focused on making ROCm production-ready at scale,” Tomasik stated in a fireside chat.

While hyperscalers were already quietly diversifying their hardware, Tomasik noted the broader neocloud ecosystem lacked a dedicated specialist. “Every neocloud was chasing the same allocation and sort of building similar businesses with the same chips. And what was missing was anyone having a conviction in a second source or platform,” he explained. 

To fill this gap, TensorWave differentiated itself through absolute AMD exclusivity, deep co-engineering for early hardware access, and heavy investments in building out a custom software stack. Tomasik’s confidence in AMD stemmed from a strategic market need, a strong relational partnership, and undeniable technical advantages, specifically the high-bandwidth memory (HBM) capacity of the MI300 and MI355 series. 

This confidence was quickly validated when TensorWave became the first public cloud to offer the MI300, and subsequently the MI355, to rapid customer adoption.

Addressing software, Tomasik acknowledged AMD's past hurdles but emphasised the platform's rapid maturation. “Two years ago, ROCm was probably where CUDA was ten years ago,” he admitted, “but it's come an incredible way in those two years.” 

Today, tools like vLLM and SGLang have first-class support, and PyTorch parity is a reality for enterprise workloads. He praised AMD's aggressive focus on developer experience, noting that getting an MI300x to work used to require manual, error-prone compilation, whereas today it requires only a single command.

With AI training and inference budgets exploding, profit margins are directly tied to infrastructure efficiency. “AMD's advantage is the memory capacity and bandwidth per dollar, which on the right workloads translates directly to lower cost per token,” said Tomasik. 

To maximise these economic benefits, TensorWave eliminates the need for code porting by providing a custom software stack tuned explicitly for AMD silicon. Furthermore, it offers reserved, long-term “take-or-pay" contracts, ensuring clients secure predictable pricing and realise genuine TCO savings without the traditional friction of migrating away from NVIDIA.

Having said that, for AMD, the biggest mountain to climb is its software, especially as the company gears up for the production of the next-generation MI400 series. In 2023, it acquired Nod.ai and appointed Anush Elangovan as the Senior Vice President and General Manager of AMD AI Software. 

With ROCm 7.0 and beyond, the company has expanded its open-source ecosystem, delivering day-zero compatibility with major frameworks like PyTorch, integrating unified Triton kernels, accelerating its update cycle to a rapid six-week cadence, and actively taking in developer feedback.