Allen Institute for AI introduces new benchmark for computer vision models

GRIT is an evaluation only benchmark for evaluating the performance of vision systems across several image prediction tasks, concepts, and data sources.
Listen to this story

Allen Institute for AI, in collaboration with the University of Illinois, has released a new unifying benchmark for general-purpose computer vision models called General Robust Image Task (GRIT). This benchmark will help AI developers in building the next generation of computer vision programs to carry out a number of generalised tasks.

GRIT is an evaluation-only benchmark for evaluating the performance of vision systems across several image prediction tasks, concepts, and data sources. As per the authors, the GRIT model will help research in the following areas:

THE BELAMY

Sign up for your weekly dose of what's up in emerging technology.
  • General-purpose models: Evaluation of unified and general-purpose models that showcase skills across a diverse set of concepts
  • Robust specialised models: GRIT helps in simplifying and uniting the qualification of misinformation, calibration and generalisation under distribution shifts.
  • Efficient learning: GRIT includes restricted and unrestricted tracks for data selection.

The researchers hope to create a workshop on the GRIT benchmark and announce it at the 2022 Conference on Computer Vision and Pattern Recognition to be held on June 19-20. The authors say that this will encourage people to submit their methods and models and evaluate them on this benchmark. They also anticipate a significant amount of work in this direction and improvement in model performance.

More Great AIM Stories

Shraddha Goled
I am a technology journalist with AIM. I write stories focused on the AI landscape in India and around the world with a special interest in analysing its long term impact on individuals and societies. Reach out to me at shraddha.goled@analyticsindiamag.com.

Our Upcoming Events

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan, 2023

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

Conference, in-person (Bangalore)
MachineCon 2023
23rd Jun, 2023

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM