AI Doesn’t Just Process Your Data. It Can Create New Data About You

For years, the privacy question around technology was relatively straightforward. What personal data does a company collect, where does they store it, and who can access it? AI complicates that picture.

An AI system can take information a company already holds, combine it with other signals and generate something new—a profile, recommendation, risk score, behavioural pattern or inference about an individual. A person may never have explicitly provided that information, yet the resulting inference can still influence what they see, which services they receive or how a platform treats them.

As AI moves from experimentation into production, this is becoming an increasingly important governance question for Indian companies.

“AI governance is about decisions, privacy governance is about data, and digital trust is about evidence,” Archana Moturi, Co-Founder and CPO, Assurtiv, says in a recent address at the Privacy by Design 2026 summit. 

The distinction matters because an AI system can transform data without necessarily creating an obvious new “data field”. A user’s prompt may be processed by an external model, stored in logs, retrieved through a vector database, combined with enterprise records and then used to generate an output or recommendation. The resulting inference can be more consequential than the original data point.

The Age-Verification Problem

The emerging debate around AI-powered age assurance offers a useful example of this challenge.

Google’s Play Age Signals API allows eligible apps to receive signals about a user’s age range or verification status, rather than requiring every app to independently determine a user’s age. Google says developers can receive signals such as SHARED, NOT_SHARED or VERIFICATION_REQUIRED, depending on the user’s circumstances and applicable jurisdiction.

But the broader question is what happens once an AI system starts inferring an attribute—in this case, whether someone is likely to be a child or teenager—from information that was collected for other purposes.

Meta has taken a different approach, using AI to analyse contextual clues across profiles, posts, comments, bios and captions to identify accounts that may belong to underage users. Its systems can also analyse visual cues in photos and videos, although Meta says this process does not use facial recognition or identify the person in an image.

The more information a platform analyses to infer an attribute, the more data it may need to process to arrive at that inference.

That is precisely where privacy governance becomes complicated.

Meta is facing a related challenge in the US. Under an August 2026 settlement with US states, the company agreed to strengthen age-assurance measures across Facebook and Instagram. Reuters reported that the settlement allows Meta to temporarily use some data generated by children it identifies to train an AI model for detecting under-13 users in the future.

From Data Mapping to Inference Mapping

This is also why enterprise data security is increasingly moving beyond simply identifying where databases are located.

A recent development from Bengaluru-based Matters.AI and Cockroach Labs illustrates the shift. Their new integration classifies personal and regulated information at the database-column level and maps it to the users, roles and privileges that can access it. It can also flag over-privileged access and cross-border residency conflicts.

The companies say the analysis runs inside the customer's infrastructure, with raw records and sampled values not transmitted to the Matters.AI control plane.

The timing is significant as Indian enterprises prepare for the substantive obligations under the Digital Personal Data Protection framework, which are scheduled to take effect from May 2027.

The financial consequences of failing to control that data are also rising. IBM’s Cost of a Data Breach Report 2026 puts India’s average breach cost at ₹25.5 crore, up 15.9% in a year. Financial services recorded an average cost of ₹40.9 crore per breach.

For AI companies, however, the challenge increasingly extends beyond knowing where the data is. It is also about understanding what the system can derive from it.

An enterprise may have controls around customer names, phone numbers or financial information. But what happens when an AI model turns those inputs into a risk score, behavioural profile, eligibility decision or prediction?

That is where AI governance starts overlapping with privacy engineering.

It also explains why AI agents are emerging as another governance challenge. An agent that can retrieve enterprise information, call an API and take an action is effectively another entity interacting with sensitive data. Its permissions, identity, audit trail and ability to act therefore need to be governed alongside those of human users.