Article complete

A New ‘Common Sense’ Test for AI may Result in More Intelligent Machines

After undergoing several testing and validation, the model was able to evaluate the core psychological reasoning ability of an AI model

A series of floating geometric cubes against a light green background, suggesting a digital or abstract environment.

Key takeaways

  • AI systems are evolving rapidly but still struggle with common sense reasoning.
  • Researchers from IBM, MIT, and Harvard developed a model called AGENT to test AI's common sense.
  • AGENT evaluates psychological reasoning by using a dataset of 3D animations inspired by children's cognitive development.
  • The model assesses AI's social awareness and ability to interact in real-world settings.
  • The testing involves familiarisation and test videos to measure expected versus surprising behaviours.

Takeaways drafted with AI from the article text.

To surpass this complexity, researchers from IBM, MIT, and Harvard have created a model named AGENT (Action-Goal-Efficiency-coNstraint-uTility) that claims a new ‘common sense’ test could produce smarter machines.

After undergoing several testing and validation processes, the benchmark was able to evaluate the core psychological reasoning ability of an AI model. Meaning, it could actually give a sense of social awareness, interacting with humans in real-world settings. The model is based on a large-scale dataset of 3D animations inspired by experiments studying cognitive development in children. 

According to IBM, the animations depict someone interacting with different objects under different physical constraints. “The videos comprise distinct trials, each of which includes one or more ‘familiarization’ videos of agent’s typical behavior in a certain physical environment, paired with ‘test’ videos of the same agent’s behavior in a new environment, which are labeled as either ‘expected’ or ‘surprising,’ given the behavior of the agent in the corresponding familiarization videos.”

The trial seeks to demonstrate common sense. An AI model must have ‘built-in representations of how humans plan’. This would be implemented by combining the basics of physics and ‘cost-reward trade-offs’. While still not perfect, the findings show AGENT is a promising diagnostic tool for developing and evaluating common sense in AI. 

The company claims that in the future, AGENT could help significantly reduce the need for training, allowing businesses to save on time, computing energy, and money. As robots don’t understand human consciousness yet, the development of such benchmarking tools can measure how close we are to the levels of advancement in AI.

Published Updated

Spotted an error? Tell our editors. Read our editorial standards and corrections.