Article complete

Microsoft, ETH Zurich Introduce SliceGPT For Compressing LLMs

The research demonstrates how SliceGPT efficiently reduces the size of language models by eliminating up to 25% of model parameters.

A stylized graphic featuring clouds and a Microsoft logo with multiple computer cursor icons.

Key takeaways

  • SliceGPT reduces language model size by eliminating up to 25% of parameters while maintaining performance.
  • Achieve significant efficiency with LLAMA2-70B, requiring only 64% of total compute for inference.
  • Utilize fewer GPUs for improved speed and efficiency without additional code optimization.
  • Innovate with computational invariance in transformer networks to inspire future advancements in model efficiency.
  • Address resource constraints of large language models through effective post-training sparsification techniques.

Takeaways drafted with AI from the article text.

Researchers from Microsoft and ETH Zurich have unveiled SliceGPT, a post-training sparsification scheme aimed at addressing the substantial computational and memory costs associated with large language models. 

The research demonstrates how SliceGPT efficiently reduces the size of language models by eliminating up to 25% of model parameters, including embeddings, while maintaining impressive zero-shot task performance.

The study focuses on three prominent models: LLAMA2-70B, OPT 66B, and Phi-2. For LLAMA2-70B, SliceGPT achieves a remarkable reduction to 64% of the dense model's total compute for inference on 24GB consumer GPUs. 

Similarly, on 40GB A100 GPUs, SliceGPT decreases total compute to 66% for LLAMA2-70B. The research also showcases that SliceGPT requires fewer GPUs to operate, resulting in enhanced speed and efficiency without the need for additional code optimisation.

The key innovation behind SliceGPT lies in its ability to replace each weight matrix with a smaller, dense matrix, effectively reducing the embedding dimension of the network. 

The sparsification technique offers a compelling solution to resource constraints associated with large language models, showcasing its potential to pave the way for future advancements in reducing memory and computation demands for pre-trained models.

Microsoft's SliceGPT not only proves its efficiency in model size reduction but also introduces a new perspective—computational invariance in transformer networks. This insight is expected to inspire and unlock new avenues for further developments in the realm of pre-trained models.

Published Updated

Spotted an error? Tell our editors. Read our editorial standards and corrections.

More in this story

Microsoft · 5 stories
  1. Google Takes On Microsoft with PaLM API & MakerSuite Launch
  2. TCS Seeks a Boost in AI Partnership with Microsoft
  3. You are here: Microsoft, ETH Zurich Introduce SliceGPT For Compressing LLMs
  4. Microsoft Researchers Introduce AllHands, An LLM Framework for Large-Scale Feedback Analysis
  5. Microsoft & Mistral Announce Multibillion-Dollar AI Infrastructure Partnership in Europe
About the author

Journalist

Mohit writes about AI in simple, explainable, and often funny words. He's especially passionate about chatting with those building AI for Bharat, with the occasional detour into AGI.

All stories
Belamy · Daily

The day’s AI news, before your first meeting.