Article complete

Yann LeCun Advices Students Getting Into AI Space to ‘Not Work on LLMs’

“I'm working on the next generation AI systems myself, not on LLMs,” said Yann LeCun.

Yann LeCun Advices Students Getting Into AI Space to ‘Not Work on LLMs’

Key takeaways

  • Avoid working on large language models (LLMs); focus on next-generation AI systems instead.
  • LeCun emphasizes that LLM development is dominated by large companies, limiting student contributions.
  • Explore alternative architectures beyond Transformers to innovate in AI development.
  • Advocate for open-source solutions to democratize AI technology and reduce corporate control.

Takeaways drafted with AI from the article text.

At VivaTech, the annual technology conference for startups, under way in Paris, Meta AI chief Yann LeCun, advised students looking to work in the AI ecosystem, to not work on LLMs.

“If you are a student interested in building the next generation of AI systems, don't work on LLMs. This is in the hands of large companies, there's nothing you can bring to the table,” said LeCun at the conference.  

He also said that people should develop next-generation AI systems that overcome the limitations of large language models.

Moving Away from LLMs

Interestingly, the discussion on alternatives of LLM-based models has been ongoing for a while now. Recently, Mufeed VH, the young creator of Devika, a Devin alternative, spoke about how people should move away from Transformer models and start building new architectures. 

“Everyone’s doing the same thing, but if we focus on different architectures, like RMKV [an RNN architecture], it would be really good,” said Mufeed, who goes on to explain the unlimited context window and inference for that particular architecture. 

He also believes that with this approach, it is even possible to build something nearly as impressive as GPT-4. 

Moving away from LLMs is something LeCun has been prominently advocating. He believes in taking the control away from the hands of a few. This is another reason why he pushes for open-source too. 

“Eventually all our interactions with the digital world will be mediated by AI assistants,” he said, and urged platforms to not allow a small number of AI assistants to control the entire digital world. 

“This will be extremely dangerous for diversity of thought, for democracy, for just about everything”, he said.

But, LLMs are Only Advancing

While LeCun might be against LLMs, the Transformer training models are evolving. Dan Hou, an AI/ML advisor, spoke about GPT-4o and emphasised on its training model. 

When text was believed to be the basis for all sophisticated models, GPT-4o was designed to understand video and audio natively. This impacts the volume of data that future versions can be trained on. 

“How much smarter can AI get? With a natively multi-modal architecture, I suspect the answer is much, much better,” said Hou. 

Furthermore, Sam Altman, in a recent interview also spoke about how data wouldn’t be a problem anymore, thereby addressing the concerns of training LLMs. 

Source: X

Published Updated

Corrections: AIM corrects errors of fact and notes the change on the story. Spotted one? Tell the editors. Read our editorial standards.

More in this story

Meta · 5 stories
  1. Meet the AI Pioneers Who Won The 2022 Princess of Asturias Award
  2. Stop Questioning OpenAI’s Open-Source Policy
  3. Meta Needs You in Its Generative AI Gambit
  4. You are here: Yann LeCun Advices Students Getting Into AI Space to ‘Not Work on LLMs’
  5. In a Relief to Meta, Law Tribunal Stays WhatsApp Data Sharing Ban
About the author

Contributor

All stories
Belamy · Daily

The day’s AI news, before your first meeting.