Google, Meta, Why NO ChatGPT?

OpenAI has been ahead of the game with generative AI, especially large language models (LLMs), with ChatGPT, and the trend since then has only been gaining more momentum. In 2019, Satya Nadella invested $1 billion into OpenAI even before they released GPT-3 and is planning to pump additional $10 billion. Microsoft has been supporting the development of generative models through OpenAI and since they made ChatGPT publicly available, people are questioning why the other tech giants​​—Google and Meta—are behind.

Recently, founder of the chatbot, ‘Replika’, Eugenia Kuyda said in an interview, “I feel like the person who was a week early arriving at the airport for a flight—and now the flight is boarding.” Kuyda also said that she refused a lot of funding offers from investors for the business.

Ethical Concerns

The money is pouring in, techniques and tools for building are available, academic papers and softwares are available on the internet, but a lot of other giants like Google and Meta are a little bit reluctant on making their models publicly available, if not building them in the first place. 

Agreeing with The New York Times article about the trend of generative AI, Yann LeCun went on to Twitter and said that Google and Meta are not releasing their generative technologies built with LLMs to the wider public citing ethical reasons because the models often produced misinformation and toxic content.

It is not the case that the two giants, Google and Meta, have made no strides in making text generating or conversational models. Meta built ‘Galactica’, fed on 48 million scientific papers and aimed to assist scientific research. The model was used for research by many scientists but had to shut down three months after its launch as it was ‘hallucinating’ results.

In the same thread, Grady Booch brought up Galactica and asked if LeCun had the same opinion about it. When Galactica was released, it was unethically claimed to be capable of publishing white papers, without providing any evidence for it.

LeCun points out the problem with ChatGPT-like models in his Linkedin post and explains that these models compensate for their weak reasoning capabilities by their large ‘memory’ capacity. “It is that answering people’s questions incorrectly doesn’t help anyone.” 

Read: Why is No One Taking Generative AI Seriously?

Closed Door Approach

Microsoft took an early bet with OpenAI, and released ChatGPT for the public. Since then, OpenAI has been on top of everyone’s mind and got everyone excited about the popularity of AI. However, though Google and Meta are not making their products openly available for the public, they still have plans and are investing in the sector.

At the AI@ ‘22 conference in November, Douglas Eck, Principal Scientist at Google Research, showed glimpses of the path they are taking towards generative models and LLMs. The company has been developing tools like Learning to Code, Wordcraft, DreamBooth, and AudioLM.

Google also announced that it is building a language model that would support a 1000 languages, and also unveiling a model with 400 languages as a first step. Meta also open sourced their OPT-175B, language model with 175 billion parameters, which is just a billion less than that of GPT-3. 

“Prophecies of Doom”

LeCun perfectly sums up the problem. On the same Twitter thread, when users pointed out that LeCun is forgetting about the failure of Galactica when criticising ChatGPT, he said that it is precisely the reason that big companies avoid public use of their products. Before actually causing any harm, people welcome them with “ridiculous knee-jerk prophecies of doom”. 

LeCun said that releasing projects as if they are products to the public is a risky step as people keep bashing the imperfections. Similar case happened with Google’s LaMDA when it was touted as sentient by one of the developers and people started criticising AI with the “end the humanity” narrative while some started calling it a hoax. 

OpenAI has released ChatGPT as a service and product for everyone to use. But most of the ideas of generative models came out of Google Brain since 2011 and the company also made big bids with DeepMind in 2014. Same is the case with Facebook and FAIR since 2013. Both companies have LLMs and generative models. They just chose not to make them publicly available, though FAIR has many of its projects open source. 

Also Read: Is DeepMind Deliberately Avoiding ‘Shallow’ Generative AI?