NVIDIA Debunks AMD’s Claim of Being the Fastest GPU

After AMD announced at its Advancing AI event that its MI300X is faster than NVIDIA’s H100 for inference, NVIDIA has rebuffed AMD's claim. 

According to a blog post by NVIDIA, contrary to AMD's presentation, the company contends that the H100 GPU, when benchmarked appropriately with optimised software, outpaces the MI300X by a substantial margin. The results, obtained using software predating AMD's presentation, demonstrated a performance that was twice as fast at a batch size of 1. 

Going even further, when applying the standard 2.5-second latency used by AMD, NVIDIA emerges as the undisputed leader, surpassing the MI300 by an astonishing 14-fold.

NVIDIA attributes the disparity in results to AMD's failure to utilise NVIDIA's optimised software, specifically TensorRT-LLM. 

AMD employed alternative software that lacked support for Hopper’s Transformer Engine and missed critical optimizations present in TensorRT-LLM. NVIDIA emphasised that its software, freely available on GitHub, is designed to enhance performance on NVIDIA hardware, a factor overlooked by AMD.

The implications of this revelation extend beyond the realm of benchmark numbers. NVIDIA reassures its investors that the company's leadership in the GPU market remains unshaken. The blog post underscores the importance of maximising GPU performance through the latest inference software, a critical factor in reducing costs and fostering broader adoption of AI.

As the AI hardware market evolves rapidly, NVIDIA remains at the forefront, continuously optimising its CUDA ecosystem. The H100, while currently holding the AI performance crown, is soon to be succeeded by the H200.

Meanwhile, AMD's MI300X, positioned as the good alternative to NVIDIA’s AI chip, faces challenges in optimising software to unlock its full potential. The competitive landscape is dynamic, with AMD acknowledging the need for further advancements in its ROCm software, which were announced at the event.