AWS to Soon Roll Out Trainium 2 Chip to Scale Anthropic’s Claude

Amazon is planning to roll out its latest AI chip, Trainium 2, in the coming month, most likely to support targeting AI model training at scale, the Financial Times reported. 

Already tested by partners such as Anthropic, Databricks, and Deutsche Telekom, Trainium 2 is part of Amazon’s larger strategy to optimise its data centre performance while reducing costs for Amazon Web Services (AWS) customers.

“The price of cloud computing tends to be much larger for machine learning and AI,” explained AWS vice president Dave Brown, adding that savings of 40% on large AI workloads can significantly impact customer choices. 

AWS chief Andy Jassy said, “The second version of Trainium, Trainium 2, will start to ramp up in the next few weeks, and I think it’s going to be very compelling for customers on a price-performance basis.”

Read: GenAI Boom Bleeds Users, Fills AWS, Azure, GCP’s Coffers 

The report further said that Amazon’s other AI chip, Inferentia, is reported to save customers approximately 40% on costs for generating responses from AI models. 

Recently, reports surfaced that AWS plans to invest more in the AI startup Anthropic. However, the company has set one condition, requiring Anthropic to use a large number of servers powered by AI chips developed in-house by Amazon.

Meanwhile, NVIDIA is building next level of computing capabilities for its customers, including OpenAI and others, where they are looking to build next level of test-time computing chips for o1-like models. 

In a recent podcast with No Priors, NVIDIA chief Jensen Huang shared that one of the major challenges NVIDIA is currently facing in computing is inference time scaling, which involves generating tokens at incredibly low latency.

Huang explained that, in the future, AI systems will need to perform tasks like tree search, chain of thought, and mental simulations, reflecting on their own answers. The model would prompt itself and generate text internally, all while responding in real-time, ideally within a second. This approach subtly points to the capabilities of the o1 system.

Meanwhile, OpenAI plans to partner with TSMC and Broadcom to launch its first in-house AI chip by 2026. This move comes after the startup began exploring a new method to scale up its models, particularly o1, using the test-time compute approach.

AWS on AI Chip Expansion Mode 

AWS recently announced a $110 million investment to support university-led research in generative AI through its new “Build on Trainium” program. This initiative provides compute hours and AWS Trainium credits, giving researchers access to Trainium UltraClusters for large-scale AI research, covering topics from AI architecture to machine learning (ML) library development.

The Build on Trainium program aims to advance AI research by offering access to up to 40,000 Trainium chips, facilitating work on distributed systems, algorithmic improvements, and AI accelerator performance. AWS developed Trainium as a specialised chip for deep learning and inference, enabling high-performance AI experiments previously limited by budget constraints.

As part of the program, AWS will conduct rounds of Amazon Research Awards calls for proposals. Selected institutions receive AWS Trainium credits and access to resources for exploring innovations in AI. 

Participants include prominent research institutions, such as Carnegie Mellon University (CMU) and the University of California at Berkeley, focusing on ML systems and compiler optimisations.

Build on Trainium also offers training and resources to grant recipients. AWS provides technical education and connects researchers with the Neuron Data Science community, fostering knowledge sharing among AWS specialists, startups, and the Generative AI Innovation Center.