2022 — The Year Where AI Went Wrong

The year of 2022 ended with a lot of updates and anticipation around the generative AI, which made routine headlines globally. And while the folks around the tech industry were optimistic about AI being the future of the world, with the notion that AI will have the capabilities to enable automation, predictive analytics, and personalised experiences that will unlock new opportunities and business models; AI researchers in totality are not particularly happy about it. 

In the recent AGI debate, where many of the global AI researchers were present, Gary Marcus said that the common point that nearly all panellists found worrisome was the near-future of Artificial Intelligence. 

LLMs and Cognition

When it comes to Artificial Intelligence, ChatGPT has recently taken over the internet. Before that, the auto image generating models like Midjourney and Stable Diffusion were the talk of the town. The 94-year-old Noam Chomsky, however, believes that no matter how many models come with updated data and parameters, the fundamental flaw in the LLMs can never be remedied. 

According to Noam Chomsky, "The problem is quite general and important. The media are running major thought pieces about the miraculous achievements of GPT-3 and its descendants, most recently ChatGPT, and comparable ones in other domains, and their import concerning fundamental questions about human nature."

Chomsky further elaborated that large language models have been shown to have several flaws and, while they may improve with more data and more parameters, there is a fundamental flaw that will always persist. He explained that due to their design, these systems cannot distinguish between possible and impossible languages.

Would the current approach to artificial intelligence ever be able to tell us anything about what makes the human mind what it is? Chomsky believes that the more the systems are improved, the deeper the failure becomes. 

As we continue to delve into the realm of artificial intelligence, we must ask ourselves: Will our current approach ever truly unveil the mysteries of the human mind? Some, like Chomsky, argue that the more we strive to improve upon our AI systems, the further we stray from understanding the intricacies of the human brain. With each advancement, we may uncover new insights but Chomsky posits that the deeper we go, the more apparent the failure of our current approach becomes. The question remains: How do we bridge the gap between artificial intelligence and the inimitable nature of the human mind?

“They are telling us nothing about language and thought, about cognition generally, or about what it is to be human. We understand this very well in other domains. No one would pay attention to a theory of elementary particles that didn't at least distinguish between possible and impossible ones.”

Scaling is not better!

As GPT 4 is round the corner, which is expected to be built on 100 trillion parameters, one has to wonder how much is too much. Is scaling too fast, always beneficial? Dileep George, DeepMind researcher, doesn't think so. 

George provided two examples of significant dates in aviation history—the first aeroplane in 1903 and the first non-stop transatlantic flight in 1919—and asked others to guess the year of the Hindenburg Disaster. He pointed out that the answer, 1937, was likely surprising to many because it occurred much later despite the rapid advancements made in aviation prior to that time.

According to George, it is important to ensure that AI has a foundational understanding of the world before attempting to scale it up. This is because scaling up current AI programmes without addressing the fundamental differences between them and human-like intelligence may not lead to the desired outcomes. Instead, we should focus on bridging these differences in order to achieve true artificial intelligence.

Dark Matter

While the advancement in the field of AI in the year of 2022 is notable, do we really know how deep we can go? AI is making mistakes and will continue to make mistakes in upcoming years but what is the main reason behind it? 

Yejin Choi, the Brett Helsel Professor at the University of Washington, warns that artificial intelligence may continue to make mistakes on uncommon or unforeseen cases due to our limited understanding of the inner workings of these networks. She compares this to the concept of dark matter in physics—something that scientists are aware of but cannot fully measure or comprehend. It seems that the true nature of language and intelligence may remain somewhat of a mystery, akin to the mysterious dark matter of the universe.

Regardless, it may be worth considering that all these researchers, along with many others, were concerned about the near future of AI and if it makes sense. Will the AI community in 2023 be able to come together and work on the flaws that the researchers discussed in the recent AI debate?