IIT Guwahati Student Builds 5B AI Model With ₹1 Lakh Budget

Indian AI startup, Kritdhi, has unveiled a five-billion-parameter multimodal language model trained on a budget of ₹1.1 lakh (~$1,200), positioning it as one of the most compute-efficient efforts in the country’s fast-growing AI race.

Shaligram Dewangan, Founder of Kritdhi, and an AI researcher at IIT Guwahati, announced the model, called Dhi-5B, describing it as a compute-optimal system pre-trained from scratch in India. The base version of the model has four billion parameters and was trained on over 40 billion natural language tokens.

According to Dewangan, comparable models are nearly 10x more expensive to train, adding that the team is operating at what he called the “training efficiency frontier.”

The Dhi-5B model is being rolled out in three stages: Dhi-5B-Base, Dhi-5B-Instruct, and the final multimodal Dhi-5B with vision capabilities.

The base variant is trained on 40 billion tokens from the FineWeb-Edu dataset. It uses a 32-layer architecture with a width of 3072, SwiGLU MLPs, full multi-head attention with FlashAttention-3, 4096-context-length, 64k-vocabulary, and a batch size of two million tokens during training.

For optimisation, the team used the Muon optimiser for matrix layers and AdamW for the remaining parameters. Dewangan said the model incorporates recent architectural design choices and training methodologies, built on a custom codebase developed in-house.

Training was carried out in five stages. The first phase, pre-training, built the core model and produced the base variant. In the second phase, context length extension, the model’s context window was increased from 4k to 16k. The third stage, mid-training, involved annealing on high-quality datasets. 

The fourth stage, supervised fine-tuning, enabled conversational abilities and resulted in the Instruct model. The final phase, vision extension, added multimodal capabilities, allowing the system to process visual inputs.

The company is also presenting this model at the India AI Impact Summit.

Alongside the model announcement, Dewangan also announced the formation of Kritdhi, which he described as India’s next frontier AI lab in the making. The lab will focus on building advanced AI systems and exploring what he called the next big leap in intelligence.

Kritdhi plans to seek significant funding support to expand its research and development efforts.

The announcement comes at a time when Indian startups and research groups are increasingly attempting to build sovereign AI systems, amid rising global competition and escalating training costs. 

While Sarvam, BharatGen and others are building large models, claiming to train a five-billion-scale model on a constrained budget, Kritdhi is positioning itself as a cost-efficient alternative in the emerging ecosystem.