Advertisement

Is EleutherAI Closely Following OpenAI’s Route?

An AI open source research group started by hackers and backed by AI players such as StabilityAI, CoreWeave, and others transitions to a non-profit research institute.
Listen to this story

EleutherAI, which began as an open-source AI research group on a Discord server in July 2020 by a group of hackers—namely, Connor Leahy, Sid Black, and Leo Gao—has announced that it is becoming a non-profit research institute. This step is seen as an encouraging move towards promoting open-source AI platforms and EleutherAI’s Discord server will continue to serve as an unrestricted platform for users. Stella Rose Biderman, an AI researcher at EleutherAI, has referred to it as a “research institute with open doors”. In 2023, the institute plans to focus more on alignment and interpretability projects.

https://twitter.com/GaryMarcus/status/1631328776486531072

Connor Leahy, the founder of EleutherAI, has announced that he and, his friend and co-founder, Sid Black, will step down as organisers and leaders of the group to focus on his AI alignment research startup, ‘Conjecture‘. Two other researchers with the institute, Louis Castricato and Tanishq Abraham, have also left to pursue their individual projects. Castricato has started ‘CarperAI’, which focuses on preference learning and RLHF, and Abraham has launched ‘MedARC’, which focuses on biomedical applications of AI technologies.

EleutherAI said that until now contributors were mostly trying to balance their regular forty-hour workweek and work on AI technology on the side. With the move of becoming a non-profit research institute, over twenty of their regular contributors will now be working full-time on their research activities.  

EleutherAI has OpenAI connection

In June 2020, OpenAI launched GPT-3 which got the ML community buzzing. During that time, on Shawn Presser’s discord server, Connor Leahy took up a paper to challenge the scaling of neural networks and thus led to an active community that focused on promoting open source AI. 

Connor used his access to Google’s TRC TPU Research Cloud to work with fellow hackers to see how far they could go with the research. TRC enables researchers to apply for access to clusters of more than 1000 Cloud TPU devices. Once accepted, researchers can access Cloud TPUs at no extra charge.

EleutherAI has built 825 GB of language modelling dataset called ‘The Pile’, which is curated from datasets including arXiv, GitHub, Wikipedia, StackExchange and HackerNews. They then built GPT-J which is a 6-billion-parameter model trained on The Pile. 

In the last one-and-a-half years, EleutherAI members have authored 28 papers, trained dozens of models, and released 10 codebases. 

Backup Partners

By transforming into a not-for-profit research institute, EleutherAI will continue to thrive on the vast domain expertise in the platform. The company has support from some of the major players in the AI market. Stability AI, Hugging Face, CoreWeave, Canva, Google TRC, Nat Friedman (former CEO of GitHub) and Lambda Labs are among the prominent collaborators for the platform. 

‘CoreWeave’, a specialised cloud service provider and part of NVIDIA Preferred Cloud Services Provider network, are the GPU providers for EleutherAI to train GPT-3 language models. CoreWeave was not only keen about open source code but also about “breaking Microsoft’s monopoly”. Built on CoreWeave GPU, EleutherAI launched its largest LLM GPT-NeoX-20B

EleutherAI, along with CompVis LMU, Runway and LAION, helped StabilityAI create their text-to-image AI system, ‘Stable Diffusion’. Since then, StabilityAI has given a portion of compute from its AWS cluster for EleutherAI’s research. 

Gary Marcus has always been a vocal supporter of EleutherAI and their vision to provide open source software. 

“Non-profit” AI future

The company insists that EleutherAI will continue to work the way it did by focusing on fostering an open source AI community. However, going by the recent example of OpenAI’s functioning, a once non-profit organisation that was quick to go down the route of profitability by shedding the tag and becoming a “capped profit” structure, where investors can get profits which are capped at 100 times their investment value. 

Such a transition from a non-profit tag is likely to happen when a company’s funds dry up and that is something EleutherAI will need to be careful about. Considering the kind of companies backing the research institute, the problem may not arise. However, those same companies are commercially motivated ventures and whether they will ultimately affect the business goals of EleutherAI is something that only time will tell. 

Download our Mobile App

Vandana Nair
As a rare breed of engineering, MBA, and journalism graduate, I bring a unique combination of technical know-how, business acumen, and storytelling skills to the table. My insatiable curiosity for all things startups, businesses, and AI technologies ensure that I'll always bring a fresh and insightful perspective to my reporting.

Subscribe to our newsletter

Join our editors every weekday evening as they steer you through the most significant news of the day.
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

Our Upcoming Events

15th June | Bangalore

Future Ready | Lead the AI Era Summit

15th June | Online

Building LLM powered applications using LangChain

17th June | Online

Mastering LangChain: A Hands-on Workshop for Building Generative AI Applications

20th June | Bangalore

Women in Data Science (WiDS) by Intuit India

Jun 23, 2023 | Bangalore

MachineCon 2023 India

26th June | Online

Accelerating inference for every workload with TensorRT

MachineCon 2023 USA

Jul 21, 2023 | New York

Cypher 2023

Oct 11-13, 2023 | Bangalore

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox
MOST POPULAR

Can Apple Save Meta?

The iPhone kicked off the smartphone revolution and saved countless companies. Could the Pro Reality headset do the same for Meta?