Key takeaways
- AMD has introduced ROCm 6, enhancing its software stack for Instinct data centre accelerators, particularly for generative AI.
- Key features of ROCm 6 include support for new data types and advanced optimisations, aiming to compete with NVIDIA's CUDA.
- Performance improvements in ROCm 6 demonstrate significant latency reductions for text generation tasks compared to previous versions.
- Collaboration with AI startups showcases the practical applications of AMD Instinct accelerators and the ROCm 6 software stack.
- OpenAI's support for AMD's accelerators in Triton 3.0 highlights a growing partnership between the two companies.
AMD at the Advancing AI event has showcased significant advancements in its software part of its Instinct data centre accelerators, emphasising an open, proven, and ready AI software platform for the market. The company introduced the latest iteration of its parallel computing framework, ROCm 6, optimised for a comprehensive software stack for AMD Instinct, particularly catering to large language models in generative AI.
Key features of ROCm 6, which is an alternative to NVIDIA’s CUDA, include support for new data types, advanced graph and kernel optimizations, optimised libraries, and state-of-the-art attention algorithms. Notably, the performance boost is remarkable, with an approximately 8x increase in overall latency for text generation compared to ROCm 5 running on the MI250.
In a collaborative effort with three emerging AI startups – Databricks, Essential AI, and Lamini – AMD showcased how they leverage the AMD Instinct M1300X accelerators and the open ROCm 6 software stack to deliver differentiated AI solutions for enterprise customers.
Furthermore, OpenAI is aligning with AMD by adding support for AMD Instinct accelerators to Triton 3.0. This move provides out-of-the-box support for AMD accelerators, enabling developers to work at a higher level of abstraction on AMD hardware within the growing ecosystem.
The ROCm Software and Ecosystem Partners announcement underscores AMD's commitment to contributing to the open-source community. ROCm 6 represents a significant step forward, offering open-art libraries and supporting various key features for generative AI, including FlashAttention, HIPGraph, and vLLM.
Read: AMD’s Attempt to Break NVIDIA’s CUDA
This positions AMD uniquely to leverage widely used AI software models, algorithms, and frameworks, fostering innovation and simplifying deployment for enterprises.
In addition to the software advancements, AMD continues its strategic investments in software through Mipsology, acquisitions like Nod.AI, and partnerships such as Lamini, which enables LLM training on AMD ROCm with zero code changes.
The unveiled developments mark a pivotal moment in the AI landscape, with AMD's robust software platform poised to unlock the true potential of generative AI, driving the industry forward into a new era of innovation.
