Can symbolic models make a comeback in AI?

The split led to the myth that while it was easier to automate man’s higher reasoning functions, it was harder to automate the functions that humans shared with other animals.

Last week, Twitter turned into a battleground in the discussion around the significance of symbolic models in AI versus deep learning. Melanie Mitchell, author and Davis Professor at the Santa Fe Institute, posted a Twitter thread speaking about how the main ideas under artificial intelligence were transforming with time. Mitchell notes that AI was defined as a study of intelligence from the context of symbolic systems and problem-solving. On the other hand, continuous systems, pattern recognition, learning and neural networks were believed to be in the domain of cybernetics. Mitchell points out that these terms have become vastly what constitutes AI now. 

Chief AI Scientist at Meta, Yann LeCun, who laid the foundations for deep learning and convolutional neural networks (CNNs), responded to the thread initially stating that it was the media that started referring to deep learning as AI, not him. 

LeCun then retweeted a response and sent a flurry of tweets saying that rather than making assumptions, people should conduct thorough research to show that symbolic models worked as well as deep learning. LeCun further clarified himself, saying that he wasn’t being a “bully to people” or “asking them to shut up.” Instead, he was welcoming diverse ideas as long as they were substantiated with proof.


Sign up for your weekly dose of what's up in emerging technology.

Gary Marcus, author and NYU Professor Emeritus, replied to LeCun’s tweet asking him to address the argument directly. Marcus himself has long been a believer in neuro-symbolic AI, which combines deep learning neural networks with symbolic reasoning techniques.

History of symbols in AI

Mitchell referred to a research paper by Allen Newell published in 1982 called ‘Intellectual Issues in the History of Artificial Intelligence’ based on the historical theories that were the foundation of AI. 

Between 1955 and 1965, the fundamental difference between AI and computer science is that while computers were generally fed instructions in numbers, AI scientists input symbols into computers. Symbols could encode all instructions, including numbers. Therefore, computers became symbol manipulators instead of number manipulators. Symbolic manipulation formed the basis of the separation between image and text processing in AI.

There was also a split between those who followed symbolic systems and continuous systems. Those who followed the continuous system adopted differential equations, statistics, probability systems and inhibitory networks. They became the cyberneticians and engineers who were involved in pattern recognition. Those who followed symbolic systems were the AI community. 

This divide led to a deeper differentiation between the two sides. Those who worked, keeping in mind the continuous systems framework, usually worked on different types of pattern recognition, like character recognition, speech recognition and visual-pattern recognition. This side eventually also ended up focusing more on learning than problem-solving. Rosenblatt’s Perceptron is an example of this side. 

On the other hand, the side that worked in the symbolic systems framework, was focused on problem-solving tasks like games, puzzle-solving and theorems. This difference between the two became more significant as the AI community deemed pattern recognition tasks more trivial as compared to problem-solving tasks. But this led to a myth that while it was easier to automate man’s higher reasoning functions, it was harder to automate the functions that humans shared with other animals. Consequently, pattern recognition work became the basis of AI, and problem-solving became an additional bonus. 

There was another rift that was created between performance and learning due to this issue. While AI created performance systems that performed intelligent tasks, cybernetics and pattern recognition work were focused on building systems that learned. There was another subcategory that originated, called self-organising systems, that seemed similar to cybernetics. But self-organising systems were involved in problems of learning instead of problems of recognition. 

The AI war

Symbolic AI is easier to understand as compared to neural networks and has higher explainability. Symbolic AI may not be able to predict the price of oil in the next month, but neural networks can. However, a neural network cannot explain how it got the outcome. In symbolic AI, the computer first needs to understand the universe of a problem, specifying its objects and rules, until the computer can make its own assumptions on the basis of these rules. 

After a few years of research, it was found that neural networks were more helpful when it came to looking for uncertain answers. Several experts in recent years have supported the idea of a hybrid system or neuro-symbolic AI that can combine recognition (e.g., the number of objects shown in an image and their colour) and reasoning (e.g., are the surfaces of the objects metallic). 

Neuro-symbolic systems resolve several obstacles that arise with data in deep nets to a great extent. Not only is it expensive to acquire training data, but it is also difficult. Even a slightly more noisy image doesn’t work in a deep neural net. Also, because neural nets have low explainability, it is not an easy task to determine why the system produced a certain result. Deep nets are also not equipped to answer questions that are more abstract.  

In the words of Gary Marcus, “Let us invent a new breed of AI systems that mix an awareness of the past with values that represent the future that we aspire to. Our focus should be on figuring out how to build AI that can represent and reason about *values*,  rather than simply perpetuating past data.”

More Great AIM Stories

Poulomi Chatterjee
Poulomi is a Technology Journalist with Analytics India Magazine. Her fascination with tech and eagerness to dive into new areas led her to the dynamic world of AI and data analytics.

Our Upcoming Events

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan, 2023

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

Conference, in-person (Bangalore)
MachineCon 2023
23rd Jun, 2023

Conference, in-person (Bangalore)
Cypher 2023
20-22nd Sep, 2023

3 Ways to Join our Community

Whatsapp group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our newsletter

Get the latest updates from AIM