AI4Bharat Builds 4,000-Test Benchmark, Finds AI Judges Fail Half the Time

Researchers from AI4Bharat at IIT Madras and BITS Pilani Hyderabad have developed a new benchmark called FOCUS to test how reliably vision-language models (VLMs) evaluate the outputs of other AI models.

The research paper, titled “Seeing Isn’t Believing: Uncovering Blind Spots in Evaluator Vision-Language Models,” has been accepted at the Conference on Language Modeling (COLM) 2026.

Vision-language models are increasingly being used as “AI judges” to evaluate tasks such as image captioning, visual question answering, and text-to-image generation. However, the researchers questioned whether these evaluator models are accurate enough to be trusted on their own.

To investigate this, the team created FOCUS, a benchmark designed to measure how well evaluator VLMs detect mistakes across both image-to-text (I2T) and text-to-image (T2I) tasks.

The benchmark contains more than 4,000 carefully designed test cases covering 40 different types of perturbations, or intentional changes that reduce output quality. These examples were created through a human-in-the-loop process and validated by expert annotators.

Using FOCUS, the researchers evaluated several frontier vision-language models across three commonly used evaluation methods while testing different prompting strategies.

The study found that current evaluator VLMs often fail to identify degraded outputs. In some cases, the models missed quality issues in more than 50% of the test samples. Their performance was particularly weaker when evaluating text-to-image generation compared to image-to-text tasks.

According to the researchers, these findings suggest that AI models should not yet be relied upon as standalone evaluators for benchmarking or training other AI systems.

Instead, they recommend using human oversight alongside AI evaluation and treating automated evaluation scores with caution until more reliable systems are developed.

Along with the paper, the researchers have also open-sourced the FOCUS dataset and codebase to help the broader AI community study and improve evaluator vision-language models.

The work was led by Mohammed Safi Ur Rahman Khan, Sanjay Suryanarayanan, Tushar Anand, and Mitesh Khapra, with affiliations to AI4Bharat at IIT Madras and BITS Pilani Hyderabad.