Advertisement

Active Hackathon

Facebook’s New AI System Can Pass Multiple-Choice Intelligence Tests

Recently, a team of researchers from Facebook AI and Tel Aviv University proposed an AI system that solves the multiple-choice intelligence test, Raven’s Progressive Matrices. The proposed AI system is a neural network model that combines multiple advances in generative models, including employing multiple pathways through the same network. 

Raven’s Progressive Matrices, also known as Raven’s Matrices, are multiple-choice intelligence tests. The test is used to measure abstract reasoning and is regarded as a non-verbal estimate of fluid intelligence.

THE BELAMY

Sign up for your weekly dose of what's up in emerging technology.

In this test, a person tries to finish the missing location in a 3X3 grid of abstract images. According to the researchers, there have been various similar researches, where the main focus entirely on choosing the right answer out of the various choices. However, in this research, the researchers focussed on generating a correct answer given the grid, without seeing the choices.

Behind the Model

As mentioned above, the neural network model is a combination of various advances in generative models, including employing multiple pathways within the same network. It uses the ‘reparameterisation’ trick along two pathways in order to make their encoding compatible, which are a dynamic application of variational losses and a complex perceptual loss which is linked with a selective backpropagation procedure.

In this research, the researchers considered the task of generating a correct answer to a Raven Progressive Matrix (RPM) type of intelligence test. In this test, each query includes eight images that are placed on a 3X3 grid size. The task of this test is to create the missing 9th image, such that it matches the patterns of the rows and columns of the grid.

The neural net model recognises the correct answer out of the eight possible choices by encoding each image and aggregating these encodings along rows and columns. The architecture of the AI model is mainly composed of three different pathways:

  • Reconstruction: The reconstruction pathway provides supervision that is more accessible to the network when starting to train.
  • Recognition: The recognition pathway shapes the representation in a way that makes the semantic information more explicit.
  • Generation: The generation pathway relies on the embedding of the visual representation from the first task, and on the semantic embedding obtained with the assistance of the second, and maps the semantic representation of a given query to an image.

How It Works

There are four key components in this method. They are:

  • An encoder (E)
  • A  generator (G) that is trained together as a variational autoencoder (VAE) on the images.
  • A Context Embedding Network (CEN), which encodes the context images and produces the embedding for the generated answer
  • Finally, a discriminator (D), which provides an adversarial training signal for the generator

The VAE pathway includes the encoder and the generator and it autoencoders the choice images as one image at a time. The CEN is composed of multiple sub-modules, which are trained together to provide input to the generator. 

Wrapping Up

The researchers stated, “In problems in which the solution space is complex enough, the ability to generate a correct answer is the ultimate test of understanding the question since one cannot extract hints from any of the potential answers. Our work has been the first to address this task in the context of RPMs.”

The success behind the model includes a number of crucial technologies, including applying the reparameterisation trick selectively and multiple times, reusing the same networks for encoding and to provide a loss signal, selective backpropagation, and an adaptive variational loss.

According to them, the machine learning algorithm is not only capable of generating a set of possible answers but also to be competitive to the state-of-the-art methods in multiple-choice tests. They also claimed that the neural net model could be used to develop an automatic tutoring system that adjusts to the proficiencies of each student.  

Read the paper here.

More Great AIM Stories

Ambika Choudhury
A Technical Journalist who loves writing about Machine Learning and Artificial Intelligence. A lover of music, writing and learning something out of the box.

Our Upcoming Events

Conference, Virtual
Genpact Analytics Career Day
3rd Sep

Conference, in-person (Bangalore)
Cypher 2022
21-23rd Sep

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
21st Apr, 2023

Conference, in-person (Bangalore)
MachineCon 2023
23rd Jun, 2023

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM
MOST POPULAR

A Case for IT Professionals Switching Jobs Frequently

For Indian companies, the ability to retain employees has become a tight ropewalk between transforming their working models and adopting a hybrid working model successfully. Over 60% respondents in the Qualtrics survey said that they would look for a new job, if forced to return to work from office full time.

What to Expect from Tesla AI Day 2022

By the looks of the promotional campaign, we can assume that the Optimus humanoid robot will be the highlight of this year’s Tesla AI day