AMD ROCm 6 is Here, OpenAI Adds Support

AMD at the Advancing AI event has showcased significant advancements in its software part of its Instinct data centre accelerators, emphasising an open, proven, and ready AI software platform for the market. The company introduced the latest iteration of its parallel computing framework, ROCm 6, optimised for a comprehensive software stack for AMD Instinct, particularly catering to large language models in generative AI.

Key features of ROCm 6, which is an alternative to NVIDIA’s CUDA, include support for new data types, advanced graph and kernel optimizations, optimised libraries, and state-of-the-art attention algorithms. Notably, the performance boost is remarkable, with an approximately 8x increase in overall latency for text generation compared to ROCm 5 running on the MI250.

In a collaborative effort with three emerging AI startups – Databricks, Essential AI, and Lamini – AMD showcased how they leverage the AMD Instinct M1300X accelerators and the open ROCm 6 software stack to deliver differentiated AI solutions for enterprise customers. 

Read: AMD Focuses on Software Ahead of MI300X Release

Furthermore, OpenAI is aligning with AMD by adding support for AMD Instinct accelerators to Triton 3.0. This move provides out-of-the-box support for AMD accelerators, enabling developers to work at a higher level of abstraction on AMD hardware within the growing ecosystem.

The ROCm Software and Ecosystem Partners announcement underscores AMD's commitment to contributing to the open-source community. ROCm 6 represents a significant step forward, offering open-art libraries and supporting various key features for generative AI, including FlashAttention, HIPGraph, and vLLM. 

Read: AMD’s Attempt to Break NVIDIA’s CUDA

This positions AMD uniquely to leverage widely used AI software models, algorithms, and frameworks, fostering innovation and simplifying deployment for enterprises.

In addition to the software advancements, AMD continues its strategic investments in software through Mipsology, acquisitions like Nod.AI, and partnerships such as Lamini, which enables LLM training on AMD ROCm with zero code changes. 

The unveiled developments mark a pivotal moment in the AI landscape, with AMD's robust software platform poised to unlock the true potential of generative AI, driving the industry forward into a new era of innovation.