Allen Institute for AI, in collaboration with the University of Illinois, has released a new unifying benchmark for general-purpose computer vision models called General Robust Image Task (GRIT). This benchmark will help AI developers in building the next generation of computer vision programs to carry out a number of generalised tasks.

GRIT is an evaluation-only benchmark for evaluating the performance of vision systems across several image prediction tasks, concepts, and data sources. As per the authors, the GRIT model will help research in the following areas:

• General-purpose models: Evaluation of unified and general-purpose models that showcase skills across a diverse set of concepts
• Robust specialised models: GRIT helps in simplifying and uniting the qualification of misinformation, calibration and generalisation under distribution shifts.
• Efficient learning: GRIT includes restricted and unrestricted tracks for data selection.

The researchers hope to create a workshop on the GRIT benchmark and announce it at the 2022 Conference on Computer Vision and Pattern Recognition to be held on June 19-20. The authors say that this will encourage people to submit their methods and models and evaluate them on this benchmark. They also anticipate a significant amount of work in this direction and improvement in model performance.