AMD Launches MI300X AI Accelerator, Takes On NVIDIA H100

AMD, going for a slice of the AI pie, has added another AI chip to its arsenal. This GPU, titled the MI300X, aims to tap a part of the AI accelerator market that NVIDIA is enjoying dominance over. The chip is purpose-built for AI tasks, and comes with up to 192GB of memory, perfect for big models like LLMs.

Due to its large amount of memory, large models can easily be deployed on the chip. The chip is also built from the ground up to enable generative AI workloads. Most state-of-the-art models today will fit into the MI300X's 192GB of HBM3 (high-bandwidth memory), with AMD CEO Lisa Su stating, 

"With all of that additional memory capacity, we actually have an advantage for large language models because we can run larger models directly in memory. What that does is...it actually reduces the number of GPUs you need, significantly speeding up the performance, especially for inference."

While this announcement comes off the back of NVIDIA's market capitalisation reaching $1 trillion, it is an important step for AMD to get a handle on the growing AI market. AMD also showed off the capabilities of the chip by running Falcon 40B on the chip at the reveal, with Su stating that this is the "first time an LLM of this size can be run entirely in memory".

During the keynote, AMD also released an update to RocM, a competing platform to NVIDIA's CUDA language. This is an important step forward for AMD, as CUDA has been one of NVIDIA's biggest moats, especially in AI-accelerated workloads. Moreover, the large amount of memory bandwidth available on the chip will allow companies to buy lesser GPUs, making AMD an interesting value proposition for smaller companies with light to medium AI workloads. 

While a price has not yet been revealed, AMD has positioned the MI300X to directly compete with NVIDIA's H100 chips for a bigger slice of the AI compute market. However, NVIDIA has a more comprehensive lineup of products for the AI age, including its latest chips made specifically for generative AI tasks. With a lack of big customers on its platform, AMD has an uphill battle to fight against NVIDIA but has fired the first volley.