AWS says Falcon40B Large Language Model Trained on Sagemaker

Amazon Web Services (AWS) has revealed that Technology Innovation Institute (TII), a leading global scientific research center in Abu Dhabi, trained its top-performing, open source Falcon 40B model on AWS. 

According to the institute, FalconLM is the most powerful open-source language model to date. Two variants of FalconLM, Falcon 40B Instruct and Falcon 40B top the Hugging Face OpenLLM Leaderboard with Meta’s LLaMA at third. 

To drive better performance and cost efficiencies throughout the development process, numerous customers, including Stability AI, AI 21 Labs, Hugging Face, and LG AI rely on Amazon SageMaker, AWS’ end-to-end machine ML service, to build, train, and deploy their LLMs. That is why TII turned to Amazon SageMaker to build its Falcon 40B model, AWS said.

Because SageMaker is a fully managed service, TII could focus on developing custom training mechanisms and optimisations instead of managing its ML infrastructure. To minimise training costs and reduce time to market, TII pursued several optimisations, including writing a custom matrix multiplication, to accelerate training speed. Throughout the training process, AWS also worked closely with TII to enhance resiliency using SageMaker, ensuring that training ran smoothly and reducing interruptions that required developer attention.

Recently, AWS also announced the general availability of Amazon Security Lake, a service that automatically centralises an organisation’s security data from across their AWS environments, leading SaaS providers, on-premises environments, and cloud sources into a purpose-built data lake, so customers can act on security data faster and simplify security data management across hybrid and multi-cloud environments. 

Amazon Security Lake is generally available today in US East (Ohio), US East (N. Virginia), US West (Oregon), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), Europe (Frankfurt), Europe (Ireland), Europe (London), and South America (São Paulo) with availability in additional AWS Regions coming soon.