Meet GPT-JT, the Closest Open Source Alternative to GPT-3

This new model has been built using recently published open-source techniques and datasets
Meet GPT-JT, the Closest Open Source Alternative to GPT-3
Listen to this story

Recently, Together, an open source community led by researchers and technocrats, released a new version of GPT-JT trained on six billion parameters. This new model has been built using recently published open source techniques and datasets. It has been trained with its decentralised approach on the Together Research Computer, a local, in-house network developed by the team.  

Unlike GPT-3, which is available on-demand/request, GPT-JT is now available as open source

Click here to access the code and datasets. 


GPT-JT came very close to GPT-3’s text-davinci-002 (175B) in terms of performance on classification benchmarks such as RAFT (following holistic evaluation of language models (HELM) protocol). 

Read: OpenAI Turns to Davinci to Make GPT-3 Better 

Together claimed that this new model is a variant of the previous GPT-J (6B), and performs well on text classification and other tasks. Banking on the power of open source AI, the team said that this would not have been possible without the open source works published by several organisations, including EleutherAI (GPT-J-6B, GPT-NeoX), Google Research (UL2, CoT), Natural-Instructions (NI) dataset by AllenAI, BigScience’s Public Pool of Prompts (P3) dataset, Ought (RAFT), and Stanford CRFM (HELM). 

Open. Scalable. Together 

Founded in 2022, Together is a decentralised cloud for artificial intelligence that enables researchers, developers, and companies to leverage and improve AI with an intuitive platform combining data, models, and computation. The team said that their GPT-JT model is inspired by lo-fi and ProxSkip, developed by Ludwig Schmidt, Mitchell Wortsman, Peter Richtárik, and others. The company has hosted this model through HuggingFace, which is the hub for the open-source AI ecosystem. 

Check out the live demo here

Download our Mobile App

Aparna Iyer
Aparna Iyer has covered various sectors spanning education, wildlife, culture and law for close to a decade. She now writes on technology and is keen to unearth its capability for public good.

Subscribe to our newsletter

Join our editors every weekday evening as they steer you through the most significant news of the day.
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

Our Upcoming Events

15th June | Online

Building LLM powered applications using LangChain

17th June | Online

Mastering LangChain: A Hands-on Workshop for Building Generative AI Applications

Jun 23, 2023 | Bangalore

MachineCon 2023 India

26th June | Online

Accelerating inference for every workload with TensorRT

MachineCon 2023 USA

Jul 21, 2023 | New York

Cypher 2023

Oct 11-13, 2023 | Bangalore

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox

Is Sam Altman a Hypocrite? 

While on the one hand, Altman is advocating for the international community to build strong AI regulations, he is also worried when someone finally decides to regulate it