Article complete

CoRover.ai Claims Its BharatGPT-Instruct Model Outperforms Sarvam 30B

CoRover.ai said its 2B-parameter BharatGPT-Instruct model has surpassed the much larger Sarvam 30B on several reasoning, fairness, and multilingual benchmarks.

BharatGPT logo featuring a stylized "B" in orange and green next to the word "BharatGPT" in bold black font.

Key takeaways

  • CoRover.ai's BharatGPT-Instruct model has outperformed the larger Sarvam 30B in various reasoning and fairness benchmarks.
  • The BharatGPT-Instruct model achieved perfect scores in CoRover's enterprise evaluation, although it struggled with the GSM8K mathematics benchmark.
  • This development challenges the notion that larger AI models are inherently superior, highlighting the value of efficiency and local relevance.
  • CoRover.ai emphasises that intelligence in AI should focus on solving real-world problems rather than merely increasing model size.

Takeaways drafted with AI from the article text.

BharatGPT-Instruct led Sarvam 30B and Qwen 1.7B across several benchmarks and achieved 100% faithfulness in CoRover’s enterprise RAG evaluation. However, it underperformed on the GSM8K mathematics benchmark.

According to CoRover, the model outperformed both Sarvam 30B and Qwen 1.7B with scores of] 83.21% on the elementary science knowledge benchmark ARC-Easy, 53.84% on the advanced science reasoning benchmark ARC-Challenge, 68.67% on the common-sense reasoning benchmark WinoGrande, 78.51% on the physical solutions evaluation PIQA, and 68.28% on CrowS-Pairs, which evaluates social bias.

The new model challenges the assumption that bigger AI models are necessarily better, shifting the focus to smaller models that prioritise efficiency over scale while performing across local languages. 

“Scale is not the same as intelligence,” Ankush Sabharwal, Founder and CEO of CoRover.ai, said in a statement. According to Sabharwal, when AI is designed around a country’s problems, languages and requirements, model size does not necessarily determine performance.

“It is about building models that are efficient, relevant, trustworthy and capable of solving real-world problems at scale,” he added.

PortGPT, CoRover’s domain-specific version of BharatGPT, designed for the ports and shipping sectors, is already live, while other specialised models are under development.

Earlier this year, the Indian Army’s Department of Military Affairs selected CoRover to set up a sovereign AI-ML lab powered by BharatGPT to enable Army personnel to learn, build, and deploy secure AI agents.

The AI startup is also empanelled by the National e-Governance Division under the Ministry of Electronics and Information Technology as a trusted AI implementation partner for the public sector.


Published

Spotted an error? Tell our editors. Read our editorial standards and corrections.

More in this story

Generative AI · 5 stories
  1. Happiest Minds Launches Agentic AI Platform to Automate Software Engineering Tasks
  2. MiniMax CEO Gives Up Salary Until AGI as Company Announces $2 Bn Funding
  3. You are here: CoRover.ai Claims Its BharatGPT-Instruct Model Outperforms Sarvam 30B
  4. Neysa Deploys AMD GPU Cluster in Mumbai for AI Training and Inference
  5. Karnataka Hosts Nearly Half of India's AI CoEs: Zinnov-KDEM Report
About the author

Contributor

Kileuna is a journalist covering the real-world impact of artificial intelligence at the intersection of tech, policy, and society.

All stories
Belamy · Daily

The day’s AI news, before your first meeting.