Google AI Introduces Flan-T5, A New Open-Source Language Model

Researchers have publicly released Flan-T5 checkpoints, which achieve strong few-shot performance compared to the much larger model of PaLM 62B
Listen to this story

Fine-tuning language models on a group of datasets phrased as instructions have been prominent in improving generalisation and model performance on unseen tasks. In an effort to take this advancement ahead, Google AI has released a new open-source language model – Flan-T5, which is capable of solving around 1800+ varied tasks. 

The first author of the paper ‘Scaling Instruction-Finetuned Language Models’, Hyung Won Chung, broke the news in a Twitter thread:

Source: Twitter

AIM Daily XO

Join our editors every weekday evening as they steer you through the most significant news of the day, introduce you to fresh perspectives, and provide unexpected moments of joy
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

The paper primarily explores instruction finetuning of areas such as scaling the number of tasks and the model size, and chain-of-thought data. The paper reads, “We find that instruction finetuning with the above aspects dramatically improves performance on a variety of model classes (PaLM, T5, U-PaLM), prompting setups (zero-shot, few-shot, CoT), and evaluation benchmarks (MMLU, BBH, TyDiQA, MGSM, open-ended generation).”

Source: Twitter

Download our Mobile App

The team has publicly released Flan-T5 checkpoints, which achieve strong few-shot performance

compared to the much larger model of PaLM 62B. Moreover, instruction finetuning is a general method utilised to improve the performance and usability of pretrained language models. With Flan-T5, researchers claim that the new model will lead to improved prompting and multi-step reasoning abilities.
To know more about Flan-T5, read the whole paper here.

Sign up for The Deep Learning Podcast

by Vijayalakshmi Anandan

The Deep Learning Curve is a technology-based podcast hosted by Vijayalakshmi Anandan - Video Presenter and Podcaster at Analytics India Magazine. This podcast is the narrator's journey of curiosity and discovery in the world of technology.

Bhuvana Kamath
I am fascinated by technology and AI’s implementation in today’s dynamic world. Being a technophile, I am keen on exploring the ever-evolving trends around applied science and innovation.

Our Upcoming Events

24th Mar, 2023 | Webinar
Women-in-Tech: Are you ready for the Techade

27-28th Apr, 2023 I Bangalore
Data Engineering Summit (DES) 2023

23 Jun, 2023 | Bangalore
MachineCon India 2023 [AI100 Awards]

21 Jul, 2023 | New York
MachineCon USA 2023 [AI100 Awards]

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox