IBM and Together AI Sign $240 Mn Deal to Build NVIDIA B300 Inference Cluster

IBM and Together AI have signed a multi-year $240 million agreement to deploy a large cluster of NVIDIA HGX B300 systems on IBM Cloud, with the infrastructure expected to be available in the first quarter of 2027.

Together AI will use the cluster to provide inference for open-source AI models. The deployment will be the first dedicated large-scale inference cluster on IBM Cloud using NVIDIA HGX B300 systems and NVIDIA Spectrum-X Ethernet networking.

The companies said the infrastructure will help Together AI increase inference capacity while reducing the cost of serving AI models to enterprise customers.

“Enterprises want the performance of the best frontier models without the closed-model price tag, and that only works if the infrastructure underneath is fast and reliable at scale,” said Vipul Ved Prakash, CEO of Together AI.

“Working alongside IBM with NVIDIA gives us that foundation. This cluster lets us bring production-grade inference to more companies, faster, and it's a big step in our push to make open-source AI the obvious choice for enterprises,” he added.

Together AI said its inference business now processes 400 trillion tokens a month. The company raised $800 million in a Series C round at an $8.3 billion valuation, with the funding going towards its AI Native Cloud.

Its platform covers inference, model training, fine-tuning and agentic workflows. The company has positioned open-source models as an alternative to closed AI systems, with developers able to build applications using modular model and infrastructure stacks.

Together AI selected IBM and NVIDIA based on their infrastructure roadmaps and ability to provide GPU capacity at the pace required for its expansion. The companies also pointed to the cost of serving tokens as a key consideration.

The cluster will use NVIDIA HGX B300 systems connected through NVIDIA Spectrum-X Ethernet networking. NVIDIA said its B300 platform can deliver 30 times more AI factory output than prior generations.

IBM said the infrastructure will also give Together AI a path into the enterprise market by combining its inference platform with IBM Cloud infrastructure.

“Enterprises are in a race to adopt agentic AI at scale to drive real business outcomes,” said Alan Peacock, General Manager of IBM Cloud. “IBM and NVIDIA are delivering scalable, economical, enterprise-grade AI infrastructure that can help Together AI accelerate innovation for the next generation of AI infrastructure.”

The deal also expands an existing collaboration between IBM and NVIDIA. The companies have been working across GPU-native data analytics, unstructured data extraction, on-premises and cloud infrastructure, and consulting services.

Dion Harris, Senior Director of HPC and AI Infrastructure Solutions at NVIDIA, said the infrastructure would help enterprises deploy open-source AI for real-time services.

“AI factories are becoming essential enterprise infrastructure—like electricity and telecommunications—turning compute and data into intelligence,” Harris said.