At re:Invent in Las Vegas, Amazon Web Services (AWS) announced two new AI chips –AWS Graviton4 , AWS Trainium2. The new chips aim to provide advancements in price performance and energy efficiency for a wide range of customer workloads, including machine learning training and generative AI applications. 

Graviton4 offers up to 30% better compute performance, 50% more cores, and 75% more memory bandwidth than Graviton3. Trainium2 delivers up to 4x faster training than its first generation, with deployment capability in EC2 UltraClusters of up to 100,000 chips.

(Source: Business Wire)

David Brown, VP of Compute and Networking at AWS said that Graviton4 marks the fourth generation they have delivered in just five years, and is the most powerful and energy-efficient chip ever built. “Silicon underpins every customer workload, making it a critical area of innovation for AWS,” he added. 

AWS chief Adam Selipsky said that it has more than 50K customers for Graviton, and its other cloud providers are still just talking about making them, and are yet to deliver first server processors. At Ignite 2023, Microsoft recently launched Azure Maia 100 AI Accelerator, its first in-house custom AI system on a chip. 

Some of its customers leveraging AWS chips include Anthropic, Databricks, Datadog, Epic, Honeycomb, SAP and others. Naveen Rao, VP of generative AI at Databricks said that AWS Trainium gave them the scale and high performance needed to train our Mosaic MPT models, and at a low cost. 

“AWS Graviton4 instances are the fastest EC2 instances we’ve ever tested, and they are delivering outstanding performance across our most competitive and latency-sensitive workloads,” said Roman Visintine, lead cloud engineer at Epic Games. 

Juergen Mueller, CTO of SAP SE said that as part of the migration process of SAP HANA Cloud to AWS Graviton-based Amazon EC2 instances, we have already seen up to 35% better price performance for analytical workloads.

Graviron4-powered R8g instances are available today in preview, with general availability planned in the coming months. Check out here. Trainium2  is said to be available in Amazon Ec2 Trn2 instances Check it out here.