Why Causality Could Fuel the Next Evolution in AI

Fundamentally moving towards a paradigm where machines try to learn more like us and this is where causality comes into the picture
Listen to this story

Friedrich Nietzsche once said, “Time, space, and causality are only metaphors of knowledge with which we explain things to ourselves.”

Causality finds mention in multiple fields from Mathematics, Psychology, Genetics, and Epidemiology to Philosophy. We humans too primarily use causal reasoning as a tool to understand the world. Now, causality is also finding relevance in the field of AI.

Simply put, causality refers to the relationship between what we do (cause) and what happens as a result of it (effect). Researcher Bernhard Schölkopf, in a paper titledCausality for Machine Learning’, states: “Despite recent successes, if we compare what machine learning can do to what animals accomplish, we observe that the former is rather bad at some crucial feats where animals excel.”

AIM Daily XO

Join our editors every weekday evening as they steer you through the most significant news of the day, introduce you to fresh perspectives, and provide unexpected moments of joy
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

AI currently can do some pretty interesting things. Large language models (LLM) like GPT3 have shown exceptional capabilities in generating and summarising text, reasoning, writing poetry, fixing bugs in software, and more. 

However, such models also have their limitations. Even though they generate responses that appear to be intelligent and relevant, they do not have a deep understanding of the meaning or context of the text they generate.

Download our Mobile App

This is because these models learn from patterns in data and do not have a natural understanding of the world. They cannot learn about causes and effects from data alone. Hence, providing machines the ability to learn like we humans do could be the next step in AI.

Can causality overcome the current limitations of AI?

While LLMs thrive on data, it is also one of their biggest limitations. “Deep learning models are trained on millions and millions of data points. But in most settings, we don’t actually have that kind of data,” Rohit Bhattacharya, assistant professor of computer science at Williams College, Massachusetts, told AIM.

“So I think fundamentally moving towards a paradigm where machines try to learn more like us and this is where causality comes into the picture. We don’t require millions of data points in order to do the things that we do. Maybe reducing the amount of data needed to make these machines function the way they do is one big aspect where causal reasoning can play a role,” he said.

Another big limitation of current machine learning, including deep learning, according to Turing awardee Yoshua Bengio is the ability to properly generalise to new settings, like new distributions – what we call ‘out of distribution generalisation’. And humans are very good at that.

“If you have a good causal model, you can generalise to new settings. Current LLMs are like encyclopaedic thieves, they’ve read everything, but don’t understand it as deeply as we do. So naturally, they’re not able to reason with that knowledge as consistently as humans are,” Bengio told AIM.

He believes there are good enough reasons to think that humans excel at that because they have causal models of the world. “The angle that I’m currently exploring is related to causal machine learning, where we introduce notions of causality in our neural nets,” Bengio said.

Causality and LLMs

The most recent big breakthrough in AI has been LLM-based ChatGPT. While it’s intriguing to imagine a ChatGPT with reasoning powers, teaching LLMs to fully understand causality is a challenging task. “With the current way that large language models are trained, it puzzles me how you would incorporate causal reasoning into the actual current training and building of these models.

“I feel we need a fundamental shift in the way we build these models because right now, we’re just making predictions and that seems to have some amount of success. However, in order to move away from that, you sort of have to build from scratch,” Bhattacharya said.

Causality and AI isn’t new

According to Bengio, a bunch of these older ideas have the potential to make a comeback in modern-day AI, like old wine in a new bottle. He believes causality could be transformative. Although discussions about causality in AI have been ongoing, there has been a significant surge in interest from the community in recent years.

Much of the credit for causality in AI goes to celebrated computer scientist Judea Pearl. He is of the opinion that for machines to be truly intelligent, we need to teach them cause and effect. “In the 80s, Pearl was working on Bayesian networks, which was touted to be the way forward, and AI.  And so, causal Bayesian networks have always been a big part of AI. In some sense, these kinds of models have been deployed for a long time with some success, especially in the medical setting,” Bhattacharya said.

In his book ‘The Book of Why: The New Science of Cause and Effect’, Pearl argues that causal reasoning could provide machines with human-level intelligence.

Sign up for The Deep Learning Podcast

by Vijayalakshmi Anandan

The Deep Learning Curve is a technology-based podcast hosted by Vijayalakshmi Anandan - Video Presenter and Podcaster at Analytics India Magazine. This podcast is the narrator's journey of curiosity and discovery in the world of technology.

Pritam Bordoloi
I have a keen interest in creative writing and artificial intelligence. As a journalist, I deep dive into the world of technology and analyse how it’s restructuring business models and reshaping society.

Our Upcoming Events

24th Mar, 2023 | Webinar
Women-in-Tech: Are you ready for the Techade

27-28th Apr, 2023 I Bangalore
Data Engineering Summit (DES) 2023

23 Jun, 2023 | Bangalore
MachineCon India 2023 [AI100 Awards]

21 Jul, 2023 | New York
MachineCon USA 2023 [AI100 Awards]

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox

Council Post: The Rise of Generative AI and Living Content

In this era of content, the use of technology, such as AI and data analytics, is becoming increasingly important as it can help content creators personalise their content, improve its quality, and reach their target audience with greater efficacy. AI writing has arrived and is here to stay. Once we overcome the initial need to cling to our conventional methods, we can begin to be more receptive to the tremendous opportunities that these technologies present.