|
Listen to this story
|
Large Language Models (LLMs) and chatbots have ruled the internet since last year. After Microsoft-backed OpenAI released ChatGPT, powered by GPT-3.5, a prudent Google took some careful consideration and time, partially releasing its experimental chatbot Bard only recently. Not to be left behind, Chinese tech giant, Baidu, too is planning to compete in the race with its Ernie Bot, built on large language model ERNIE 3.0, expected to launch in March this year. There is more to come in the coming months.
But, one tech player seems to be shying away from developing LLM bots is Meta. Surprisingly, it is perhaps the only tech player still not hopping on to the LLM bandwagon, and that may be because of its multiple failed attempts in the past.
In June 2022, Meta had open-sourced its LLM OPT-66B. It was one of the first ones to release a chatbot built on LLM – BlenderBot 3 was all the hype when it first came out in August 2022. Though it was only released for the US, Twitter and Reddit were awash with screenshots of people’s interesting conversations with the AI conversationalist.
Within a day or two, the successful bot took a downward turn towards an AI-disaster. BlenderBot began to spew racist remarks and even questioned Mark Zuckerberg’s business strategies, labelling them ‘unethical’. No remorse at all. This, unfortunately, led to people losing trust in the chatbot, and Meta had to ultimately see a decline in users.
Not that it stopped the company from venturing into another LLM-based model. This time with a little more specific use case, Galactica, was built for scientific research and released just two weeks before ChatGPT’s launch. Fed on thousands of research papers, the model was expected to assist researchers and scientists further their discovery. But again, the model started doling out hallucinating results, and had to be shut down.
Undeterred, the company showed astonishing potential in creating rivals to ChatGPT or the recent Bard, but things seemed to have suddenly stopped their end lately. So what changed?
According to Yann LeCun, the company’s AI head, instead of nose diving into the current ‘AI boom’ with chatbots, the company wants to perfect the text models and actually make something that people can trust with accuracy. And this is what is the biggest criticism of the recent chatbots, according to the Meta AI head.
Also, another failed attempt from Meta would leave a permanent scar. For instance, Google’s recent blunder during the promotion of LaMDA powered chatbot Bard at the AI ‘Live in Paris’ event, cost the company $100 billion dollars. Meta cannot take that risk.
Meta’s place in the AI race
Meta has been an AI leader for a long time. Since 2013, Zuckerberg has pumped in billions of dollars to remain the leader in the AI race. Hiring LeCun was the next significant move, which bode well for the company and led to Zuckerberg pouring in even more into the AI field. But now, in the era of generative AI, more specifically LLMs, the company is hesitant to indulge more.
Most of the innovation from Meta’s AI team since 2013 went into their ad business and removing misinformation from the platform. Now the company is struggling to convert their AI models and algorithms into products. When they first tried it with their facial recognition system, they eventually had to shut it down due to privacy issues.
LeCun pointed out that the company has had a long run of being blamed for spreading false information. Now with the recent BlenderBot and Galactica blunders, the company wants to be extra sure with what they put out in the public, and not get blamed for the same mistakes again. Maybe small companies like OpenAI had nothing to lose when releasing experimental technology in the world. Same cannot be said for Meta.
A researcher at Google AI Labs and also the Chinese giant Baidu, Andrew Ng, had said, “People tend to hold large tech companies to a high standard.” This gives more creative and experimental freedom to smaller companies. Maybe that is why Microsoft invested in a separate company to keep the risk away from them. It’s all about playing it safe.
When BlenderBot was launched, LeCun said that it stretched the state of the art. But the company was very concerned about making sure that the chatbot does not spew misinformation. Now, LeCun says there is “nothing innovative” about ChatGPT, and the technology is built on what was earlier offered by Meta and Google. At the same time, he admits that language models are not useless as well.
Zuckerberg and LeCun are both currently betting big on generative AI, and have told the investors that AI is going to be the focus when providing ad services and products to other businesses, and yet are themselves shying away from LLMs.
The CEO’s long-running dream of the metaverse is still on, and his AI chief also subscribes to the fact that generative AI can be best implemented in the metaverse. But when it comes to LLMs and chatbots, the company seems to be observing from a distance, looking out for the mistakes that Google, Microsoft, or OpenAI are making. LeCun has been actively pointing them out while also being at the receiving end of a lot of bashing because of Galactica and BlenderBot.
