Last week, Twitter turned into a battleground in the discussion around the significance of symbolic models in AI versus deep learning. Melanie Mitchell, author and Davis Professor at the Santa Fe Institute, posted a Twitter thread speaking about how the main ideas under artificial intelligence were transforming with time. Mitchell notes that AI was defined as a study of intelligence from the context of symbolic systems and problem-solving. On the other hand, continuous systems, pattern recognition, learning and neural networks were believed to be in the domain of cybernetics. Mitchell points out that these terms have become vastly what constitutes AI now.
Chief AI Scientist at Meta, Yann LeCun, who laid the foundations for deep learning and convolutional neural networks (CNNs), responded to the thread initially stating that it was the media that started referring to deep learning as AI, not him.
Sign up for your weekly dose of what's up in emerging technology.
LeCun then retweeted a response and sent a flurry of tweets saying that rather than making assumptions, people should conduct thorough research to show that symbolic models worked as well as deep learning. LeCun further clarified himself, saying that he wasn’t being a “bully to people” or “asking them to shut up.” Instead, he was welcoming diverse ideas as long as they were substantiated with proof.
Gary Marcus, author and NYU Professor Emeritus, replied to LeCun’s tweet asking him to address the argument directly. Marcus himself has long been a believer in neuro-symbolic AI, which combines deep learning neural networks with symbolic reasoning techniques.
History of symbols in AI
Mitchell referred to a research paper by Allen Newell published in 1982 called ‘Intellectual Issues in the History of Artificial Intelligence’ based on the historical theories that were the foundation of AI.
Between 1955 and 1965, the fundamental difference between AI and computer science is that while computers were generally fed instructions in numbers, AI scientists input symbols into computers. Symbols could encode all instructions, including numbers. Therefore, computers became symbol manipulators instead of number manipulators. Symbolic manipulation formed the basis of the separation between image and text processing in AI.
There was also a split between those who followed symbolic systems and continuous systems. Those who followed the continuous system adopted differential equations, statistics, probability systems and inhibitory networks. They became the cyberneticians and engineers who were involved in pattern recognition. Those who followed symbolic systems were the AI community.
This divide led to a deeper differentiation between the two sides. Those who worked, keeping in mind the continuous systems framework, usually worked on different types of pattern recognition, like character recognition, speech recognition and visual-pattern recognition. This side eventually also ended up focusing more on learning than problem-solving. Rosenblatt’s Perceptron is an example of this side.
On the other hand, the side that worked in the symbolic systems framework, was focused on problem-solving tasks like games, puzzle-solving and theorems. This difference between the two became more significant as the AI community deemed pattern recognition tasks more trivial as compared to problem-solving tasks. But this led to a myth that while it was easier to automate man’s higher reasoning functions, it was harder to automate the functions that humans shared with other animals. Consequently, pattern recognition work became the basis of AI, and problem-solving became an additional bonus.
There was another rift that was created between performance and learning due to this issue. While AI created performance systems that performed intelligent tasks, cybernetics and pattern recognition work were focused on building systems that learned. There was another subcategory that originated, called self-organising systems, that seemed similar to cybernetics. But self-organising systems were involved in problems of learning instead of problems of recognition.
The AI war
Symbolic AI is easier to understand as compared to neural networks and has higher explainability. Symbolic AI may not be able to predict the price of oil in the next month, but neural networks can. However, a neural network cannot explain how it got the outcome. In symbolic AI, the computer first needs to understand the universe of a problem, specifying its objects and rules, until the computer can make its own assumptions on the basis of these rules.
After a few years of research, it was found that neural networks were more helpful when it came to looking for uncertain answers. Several experts in recent years have supported the idea of a hybrid system or neuro-symbolic AI that can combine recognition (e.g., the number of objects shown in an image and their colour) and reasoning (e.g., are the surfaces of the objects metallic).
Neuro-symbolic systems resolve several obstacles that arise with data in deep nets to a great extent. Not only is it expensive to acquire training data, but it is also difficult. Even a slightly more noisy image doesn’t work in a deep neural net. Also, because neural nets have low explainability, it is not an easy task to determine why the system produced a certain result. Deep nets are also not equipped to answer questions that are more abstract.
In the words of Gary Marcus, “Let us invent a new breed of AI systems that mix an awareness of the past with values that represent the future that we aspire to. Our focus should be on figuring out how to build AI that can represent and reason about *values*, rather than simply perpetuating past data.”