IT Services

    Why India's IT Giants are Swapping Bloated LLMs for Small Language Models

    By ·

    Why India's IT Giants are Swapping Bloated LLMs for Small Language Models
    Image by Nalini Nirad

    Given the GPU supply constraints, India is primed to lean on small language models (SLMs) due to their lower operational costs and hardware efficiency, as they can run on edge devices. Startups including Sarvam AI, Gnani.ai, and Shunya Labs have been turning to smaller AI models to reduce inference costs. Now, even Indian IT seems to have come to the conclusion that bigger is not always better. 

    During the Q1 FY27 earnings calls, senior executives at Infosys, HCLTech, and Tata Consultancy Services (TCS) agreed that SLMs—which are leaner, cheaper, domain-specific alternatives to the frontier large language models built by OpenAI and Anthropic—is the technology actually moving the needle with enterprise clients, particularly in regulated sectors such as banking and financial services.

    The shift comes even as conversation around generative AI has been dominated by frontier models trained on trillions of parameters. However, analysts say that enterprise money was never going to be spent on that race.

    "No banks will adopt an LLM model," Gaurav Vasu, Founder and CEO of research firm UnearthInsight, tells AIM, arguing that institutions handling sensitive customer or citizen data, from banks to hospitals to tax departments, cannot risk that data leaking through a public large language model. 

    Vasu notes that clients are actively seeking smaller, more controllable models instead, adding that large models "had never gone out of fashion" but had simply stopped making headlines while enterprises quietly held back.

    That caution shows up starkly in adoption numbers. While a Deloitte report shows that more than 80% of enteprises in India are interested or investing in AI initiatives, Vasu estimates enterprise-wide AI adoption globally remains under 10%. In his view, large foundational models remain overwhelmingly consumer-facing; however, enterprises are looking for something domain-specific.

    The bottomline is central to that shift. According to Marvel.ai, high-traffic enterprise deployments spend $50,000+ monthly on LLM APIs, whereas an equivalent SLM workload costs between $3,000 and $8,000.

    “​​For a country like India, the '95% of the profit' doesn't lie in building the world's largest general-purpose model, which is an expensive 'arms race' for tech giants. Instead, the real value lies in the application layer: building millions of affordable, energy-efficient SLMs that can run directly on a farmer's smartphone or a village clinic's tablet without needing a cloud,” Soumya Banerjee, Assistant Research Professor at the University of Cambridge, wrote in a LinkedIn post.

    Banking Leads the SLM Push

    At a recent press conference by Infosys, Chief Executive Officer Salil Parekh said small language models are gaining good traction in banking, built around the company's Finacle platform, where a smaller parameter set brings down running costs and improves token efficiency. He said the same efficiency logic is now being extended to IT operations work as well.

    “For a lot of the clients there, it's an easy way to leverage a small language model, where the parameter set is smaller. The cost of running it is smaller. The token usage is more efficient and so on. The same for IT ops. That's all part of that scaling,” Parekh said.

    HCLTech's C Vijayakumar was more expansive, tying the SLM push to the company's partnership with Indian AI startup Sarvam, which he said gives HCLTech stronger infrastructure and research capability to train small language models on enterprise data, though he clarified the underlying model need not always be Sarvam's own. 

    Vijayakumar also flagged client wariness toward proprietary, closed AI models, saying enterprises are increasingly turning to open-weight and open-source alternatives instead. 

    He framed HCLTech's data centre strategy around SLMs and enabling a full stack of high-margin AI services layered on top of computing capacity. 

    “50 MW is good capacity to build a large business if you want to do small language models and not frontier models,” he said, adding that the Sarvam tie-up is an early-stage opportunity to build client- and industry-specific small models.

    Sovereignty is a Separate Question

    SLMs can run on edge devices like smartphones and desktops, suitable for local frameworks. They ensure that sensitive data does not leave an organisation’s control.

    “In highly scrutinised or private equity-backed environments, data sovereignty is non-negotiable. SLMs allow you to lock down the data boundary, eliminate hallucination risks, and maintain absolute control over compliance and audit trails,” Brendan Hennessy, CEO of organisational intelligence platform Spark, commented in a LinkedIn post.

    With SLMs, organisations can also implement bespoke encryption, access logs, and audit trails tailored to their security needs, unlike generic cloud-based solutions.

    "By hosting models on-premises or in private clouds, sovereign SLMs reduce exposure to external threats like data breaches or unauthorised access. Role-Based Access Control can be tightly integrated, ensuring only authorised personnel interact with the model or its outputs," Sanjay G, CTO and Chief Architect of energy management solutions company Vigyanlabs, noted on LinkedIn.

    However, Vasu is careful to draw a line between SLMs and "sovereign" AI, which he describes as a matter of keeping a country's or client's entire data stack within its own geography—the model Sarvam is pursuing for India and one that Singaporean or other global banks would not accept if it meant their data sat in Indian data centres. 

    He suggests the HCLTech-Sarvam partnership could still spur more data-centre construction in hubs such as Lucknow, Manesar, and Noida, even if global financial institutions keep their own data elsewhere.

    Multiple Models, Not One

    TCS chief executive K Krithivasan offered a complementary view of where SLMs are heading. Rather than standardising on a single model, most large enterprises will likely run a portfolio—one large model alongside several small ones, or multiple versions within the same model family—choosing between an older or newer model, and between on-premise and cloud, token-based deployment, depending on the workload. 

    The demand for SLMs is already there. According to The Business Research Company, the global SLM market is valued at $11 billion and is expected to grow to $22.45 billion by 2030.

    The enterprise AI conversation is maturing, moving away from chasing foundational models and toward a more fragmented market where cost, data control, and regulatory comfort, not benchmark performance, determine what enterprises are willing to deploy.