Article complete

China Open Sources TinyGPT-V, Outperforms Larger MLLMs

The model is built on top of Microsoft Phi-2.

A stylized figure runs over a trap with a keyhole emblem and decorative clouds against a red background.

Key takeaways

  • Launch TinyGPT-V, a pioneering AI model outperforming larger MLLMs with reduced computational demands.
  • Achieve superior performance compared to 13 billion and 7 billion parameter models like Flamingo and MiniGPT-4.
  • Utilize only a 24GB GPU for training and an 8GB GPU or CPU for inference.
  • Leverage Microsoft Phi-2 and integrate vision modules, ensuring high performance with minimal resource requirements.
  • Implement a unique quantisation process for seamless deployment on devices with limited capacity.

Takeaways drafted with AI from the article text.

Researchers from China from Anhui Polytechnic University, Nanyang Technological University, and Lehigh University have unveiled TinyGPT-V. This AI model combines remarkable performance with reduced computational demands, marking a paradigm shift in the development of cost-effective and efficient MLLMs.

Compared to other MLLM such as Flamingo, MiniGPT-4, the model achieves better performance than 13 billion and 7 billion models. It is built on top of Microsoft’s Phi-2.

TinyGPT-V distinguishes itself by requiring only a 24GB GPU for training and an 8GB GPU or CPU for inference, addressing the computational efficiency challenges faced by its predecessors. 

Leveraging the Phi-2 model as its language backbone and integrating pre-trained vision modules from BLIP-2 or CLIP, TinyGPT-V strikes a unique balance between high performance and minimized resource requirements.

The architecture of TinyGPT-V incorporates a distinctive quantisation process, allowing for seamless local deployment and inference tasks on devices with an 8GB capacity. This feature makes TinyGPT-V an ideal choice for real-world scenarios where deploying large-scale models is often impractical. 

Linear projection layers embedded in the model facilitate the efficient integration of visual features into the language model, bridging the gap between image-based information and language comprehension.

Notable benchmarks attest to TinyGPT-V's outstanding capabilities. In the Visual-Spatial Reasoning (VSR) zero-shot task, TinyGPT-V outshone models with significantly larger parameter counts, showcasing its prowess in handling complex multimodal tasks efficiently. 

Benchmarks such as GQA, IconVQ, VizWiz, and the Hateful Memes dataset further underscore the model's versatility and computational efficiency, making it a compelling option for a wide range of real-world applications.

Published Updated

Spotted an error? Tell our editors. Read our editorial standards and corrections.

More in this story

Open Source AI · 5 stories
  1. Google & FB’s Latest Open Source Contributions To Machine Learning
  2. CRED open sources its UI design system NeoPop
  3. You are here: China Open Sources TinyGPT-V, Outperforms Larger MLLMs
  4. Snowflake Open Sources Arctic, Family of Embedding Models for RAG
  5. Meet the ‘Chinese Claude Code’ Looking to Create the Next DeepSeek Moment
About the author

Journalist

Mohit writes about AI in simple, explainable, and often funny words. He's especially passionate about chatting with those building AI for Bharat, with the occasional detour into AGI.

All stories
Belamy · Daily

The day’s AI news, before your first meeting.