In their 2009 manifesto, Neural-Symbolic Cognitive Reasoning, Artur Garcez and Luis Lamb, discussed the 1990s popular idea of integrating neural networks and symbolic knowledge. They cited Towell and Shavlik’s KBANN, Knowledge-Based Artificial Neural Network that uses a system to insert rules, refine and extract data from neutral network; a model empirically proving to be effective. Industry leaders, including contingencies at IBM, Intel, Google, Facebook, and Microsoft, and researchers like Josh Tenenbaum, Anima Anandkumar, and Yejin Choi, are starting to apply this technique in 2022. The recent AI developments, challenges, and stagnancy is leading the industry to consider this hybrid approach to AI modelling.
Neuro-Symbolic AI is essentially a hybrid AI leveraging deep learning neural network architectures and combining them with symbolic reasoning techniques. For example, we have been using neural networks to identify the shape or colour a particular object has. Applying symbolic reasoning to it can take it a step further to tell more exciting properties about the object, such as the area of the object, volume and so on.
The stagnant AI: Why we need a neuro-symbolic approach
AI has been the talk of the town for more than a decade, and while it has stood true to several promises, the majority of the claims are still to be met, and the challenges connected to AI have only been increasing. In the past year, GPT-3 has asked individuals to commit suicide, Alexa has challenged a ten-year-old to touch a coin to the plug, and Facebook’s algorithm has identified a man of colour as a primate. This is not any different from Microsoft’s Tay asserting ‘Hitler was right’ or Uber’s self-driving cars crossing red lights a few years ago. With every GPT-3 development, we have seen a downfall. The present efforts to ensure explainable, fair, ethical and efficient AI need to be supported by changes in how we approach artificial intelligence.
Scientist and AI author and entrepreneur Gary Marcus recently wrote about deep learning hitting a wall and the responsibility of AGI. “It must be like stainless steel, stronger and more reliable and, for that matter, easier to work with than any of its constituent parts. No single AI approach will ever be enough on its own; we must master the art of putting diverse approaches together if we are to have any hope at all.”
The neuro-symbolic approach
Since AI use learning and reasoning in a quest to be like humans, the neuro-symbolic approach allows us to combine these strengths to make inferences based on the existing neural networks and learn through symbolic representations. Knowledgeable Magazine asserted this hybrid to show duckling-like abilities. Ducklings can imprint colours and shapes and differentiate between them. Moreover, they can differentiate between “same” and “different”, an aspect AI still struggles with. Here, symbolic AI would involve symbols for physical objects and colours in its knowledge base. The base also consists of general rules to differentiate between them. This, combined with the deep nets, allows the model to be more efficient.
The combinations demand humans supply a knowledge base/symbolic rules for the AI to leverage while the automated deep nets find the correct answers. “The hybrid uses deep nets, instead of humans, to generate only those portions of the knowledge base that it needs to answer a given question,” notes Knowledgeable Magazine.
Real-life applications
DeepMind has seen some of the best success with their board-game playing AI models Go, Chess, MuZero and more. These are hybrid models using symbolic AI. For instance, AlphaGo used symbolic-tree search with deep learning, AlphaFold2 combines symbolic ways of representing the 3-D physical structure of molecules with the data-trawling characteristics of deep learning. Deepmind has asserted the qualities of symbolic learning in AI in a recent blog post. “This approach will allow for AI to interpret something as symbolic on its own rather than simply manipulate things that are only symbols to human onlookers,” they said. This allows for AI with human-like fluency. IBM has asserted neuro- symbolic AI is ‘getting AI to reason’. The LNN technique, Logical Neural Network, was introduced and created on the foundations of deep learning and symbolic AI. Given the combination, the software can successfully answer complex questions with minimal domain-specific training.
Two major conferences have been held asserting this need—the 2019 Montreal AI Debate between Yoshua Bengio and Gary Marcus and the AAAI-2020 fireside conversation with Laureate Daniel Kahneman, Geoffrey Hinton, Yoshua Bengio and Yann LeCun. The key takeaway from these events was the need for AI to have a reasoning layer with deep learning to frame a rich future of AI.
