Can Enterprise AI Agents Learn From Their Own Mistakes? A Bengaluru Startup Thinks So

AI agents are increasingly being deployed to handle customer support, sales, employee queries, and other enterprise workflows. But once these systems go live, most companies still treat them as relatively static systems—monitoring performance and manually updating them when something goes wrong.

NuPlay AI (formerly Nurix AI) is taking a different approach. The Mukesh Bansal-led company is building what it describes as self-improving AI agents that continuously track their performance, identify where they fail, and use those insights to improve how they handle future interactions.

However, NuPlay does not want an AI model to overstep and independently rewrite its own underlying parameters. Instead, the Bengaluru-based startup is building a feedback loop around the agent and the workflows it operates in. The system observes what happens in production, evaluates outcomes, identifies gaps, and tests potential changes before deploying them.

From Deployment to Continuous Improvement

The conventional enterprise AI workflow typically involves training or configuring an agent, deploying it, and then monitoring whether it performs as expected. If users encounter problems, teams analyse those failures and modify the system.

NuPlay wants to compress that feedback cycle. The company’s platform monitors agent interactions and evaluates them against the desired business outcome. If an agent fails to complete a task, produces an unsuitable response, or deviates from the expected workflow, the system attempts to identify what caused the failure.

“Agents find gaps in that—why it was isolated, the reasoning behind all of those things,” NuPlay AI CTO Anuj Modi tells AIM.

Those gaps could originate from several places, including the instructions given to the agent, the information available to it or the workflow itself. Modi describes these as potential “SOP” or profile gaps.

This creates a loop that looks less like traditional model training and more like continuous software optimisation: observe, evaluate, identify a problem, propose a change, test it, and monitor.

NuPlay describes this layer as NuLoop, which evaluates agent runs and feeds improvements back into the wider system. The company says its agents can learn from real-world performance and that it tests proposed changes before pushing them into production.

The Simulation Problem

One challenge with self-improving systems is that experimentation cannot happen directly on live customers. An agent allowed to change its behaviour in production without sufficient testing could introduce new errors while attempting to fix old ones. For an enterprise handling payments, refunds, customer accounts, or sensitive employee information, that could quickly become a business risk.

NuPlay says it addresses this through simulation. “We have a simulation agent operator which actually simulates those sequences. It applies those sequences again to the production system,” Modi explains. This allows the system to test changes before exposing customers to them.

The approach is increasingly becoming central to agent development. In its 2026 Hype Cycle for Agentic AI report, Gartner identified simulation as an important component of evaluating and continuously improving agentic systems, while AI developers are increasingly using evaluation frameworks to test agent behaviour before and after changes.

The Governance Problem

Self-improvement also introduces a difficult reality: what happens when an agent improves in a direction that the business does not want? The question is becoming critical amid the recent incidents of OpenAI and Anthropic agents going rogue, escaping containment, and breaching external organisations despite being instructed otherwise.

That concern grows as agents gain more autonomy. A system optimising for one business metric could potentially make decisions that technically improve that metric but violate company policy or customer expectations.

NuPlay’s approach to governance includes restrictions around what an agent is permitted to do. Modi describes the use of “white-listed things and black-listed things,” with certain actions explicitly allowed or prohibited.

The company also uses anomaly detection to identify behaviour that falls outside expected patterns.

Modi gives the example of an agent handling refunds. If an agent normally refunds around 4% of requests but suddenly begins approving a significantly higher proportion, the system should recognise that behaviour as an anomaly rather than allowing the agent to continue optimising independently.

In such cases, “it’s better to direct it to humans,” he remarks.

That human fallback is important because the value of a self-improving system ultimately depends on the boundaries placed around it. An agent that can identify and correct mistakes but cannot distinguish between an acceptable optimisation and a risky deviation could create more problems than it solves.

India Adds Another Layer of Complexity

For NuPlay, the challenge is not limited to enterprise governance. Building agents for Indian users also requires dealing with how people actually communicate.

India’s linguistic diversity means that conversations rarely fit neatly into a single language. Users frequently mix languages, switch between English and regional languages, and use speech patterns that differ considerably from formal written communication.

“The way in which we speak versus the way in which we write is very, very different,” Modi notes.

This creates another potential point of failure. An agent may technically understand each individual language but still struggle with context, code-switching, informal expressions, or the intent behind a conversation.

NuPlay’s approach is to allow agents to seek clarification. “It is possible that if you are not understanding, let's just ask clarifying questions to be able to understand better,” Modi says.

For enterprise deployments, this could be more useful than simply attempting to make an agent answer every query. Asking a customer to clarify a request may introduce some friction, but it can also prevent an incorrect action from being executed.

NuPlay says its platform is already being used across enterprise workflows, with the company claiming more than 150 enterprises in production and more than 500 workflows live.