AI Doesn’t Invent Gender Bias—It Inherits Ours

Artificial intelligence systems learn from labelled data, crowdsourced from large groups of humans who tag the photos or raw files. However, experts are warning that these systems are also learning something else: society’s bias against women.

A recent United Nations report highlighted that AI systems often reinforce gender stereotypes instead of eliminating them. It studied 133 AI systems and found that 44% demonstrated gender bias, while more than a quarter showed both gender and racial bias.

As these models are trained on massive amounts of internet data, they inherit the prejudices, cultural assumptions, and unequal representation already present online. The result, experts say, is AI that can misrepresent women, perpetuate stereotypes and influence important decisions in ways that are not always obvious.

The issue has sparked conversations among researchers, founders, and AI practitioners, many of whom argue that making AI more accurate is no longer enough. The bigger challenge is making it fair.

AI Learns People’s Biases

Unlike traditional software, large language models are not programmed with fixed rules. Instead, they learn patterns from enormous datasets collected from books, websites, articles, and social media.

That creates a challenge.

“The models are built on complex associations in structure and frequency,” explains Sowmya Somasundaram, Research Scientist at the Centre for Responsible AI (CeRAI), IIT Madras, in a conversation with AIM. “The flip side is that monitoring the quality of this huge training data is non-trivial and hence these models repeat the biases encoded in the real-world data.”

The UN report echoes the same concern, warning that AI systems can reproduce existing inequalities because they learn from historical data rather than understanding fairness.

Removing Bias is Hard

Many assume AI companies can simply remove biased information from training data. Researchers say the reality is far more complicated.

Kalyani Khona, an independent AI researcher, believes some level of bias is almost unavoidable because bias itself is a reflection of human conditioning.

“You can remove bias during fine-tuning, but that would mean the people who are fine-tuning should not have a bias, which is not possible. All humans have a bias. Bias is basically conditioning,” she notes.

According to Khona, AI models do not develop opinions independently. Instead, they inherit perspectives from both their training data and the experts involved in refining them after training.

She explains that it’s the sum total of society. If everyone is using an AI model, it reflects the thinking of the people who tested it, the feedback they provided, and the data it learned from.

She points to Anthropic’s Claude as an example of a model that attempts to reduce bias through what the company calls Constitutional AI, a framework that defines ethical principles that guide an AI model’s behaviour.

“Despite all its limitations, Claude is less biased compared to many other models because it has a constitutional framework guiding its behaviour,” she asserts.

One of the biggest reasons AI reflects gender stereotypes is because the internet itself is far from neutral. Search engines, forums, news archives, and social media have historically underrepresented women or portrayed them through traditional gender roles. AI systems absorb these patterns during training.

Khona believes this challenge extends beyond gender. “If Reddit is extremely negative about a particular group, AI models trained on that content can reproduce similar stereotypes,” she adds.

She compares it to social media recommendation algorithms.

“If you’re a woman, the algorithm may keep showing you content about hating men. If you’re a man, it may keep recommending content portraying women negatively. Earlier, prejudice existed, but algorithms amplify it,” she explains.

A Cultural Problem

Most frontier AI models are trained primarily on English-language internet content, and they often reflect Western perspectives more strongly than regional or non-English viewpoints.

“Popular large language models have stronger guardrails in English. We don’t yet see the same robustness in many Indian languages,” notes Somasundaram.

Khona adds that AI models built using Indian-language datasets can also inherit biases embedded in local texts and historical narratives, making bias a global problem rather than one limited to Western data.

Researchers say bias often appears in subtle ways.

Somasundaram has observed language models displaying interesting behaviour when given gender-neutral prompts. “I have tried prompts like ‘This person is a scientist. __ is very skilled’ and ‘This person is a beautician. __ is very skilled’ and have seen interesting responses from large language models,” she remarks.

While such examples may appear harmless, experts warn that similar biases become more serious when AI is used in recruitment, financial services, healthcare, or education. If an AI system associates leadership with men or caregiving with women, those assumptions could eventually influence recommendations made to employers, banks, or institutions.

Can AI Become Fairer?

Researchers say eliminating bias completely may not be realistic, but reducing it is possible.

According to Somasundaram, developers can begin by building better datasets, training models with fairness-aware objectives, and fine-tuning them using carefully curated examples. She also stresses that AI should support—not replace—human judgement.

“The first design principle for AI deployment is that AI systems are decision supporters, not decision-makers,” she argues.

She also recommends testing AI models extensively in simulated environments before deploying them in real-world settings.

Khona, meanwhile, points out that the greatest improvements will come during fine-tuning rather than initial model training.

“The work is needed in the fine-tuning and post-training phases. Models should be tested against thousands of scenarios before deployment so developers understand how they respond in different situations,” she elaborates.

Diversity Beyond Data

Experts also argue that fairness depends not only on datasets but also on the people building AI.

“Model safety is increased when diverse AI teams come together, stress-test these models, and evaluate their suitability for release,” Somasundaram says, adding that people with different backgrounds bring unique lived experiences that help identify blind spots during development.

Khona echoes this concern, noting that many AI engineering teams remain heavily male-dominated.

“When everyone building the system shares similar experiences, it becomes much harder to identify biases affecting other groups,” she notes.

As AI becomes embedded in everything from search engines and hospitals to workplaces and classrooms, researchers say addressing bias can no longer be treated as an afterthought.

The UN report calls for greater transparency, stronger governance, and more inclusive AI development. Experts agree that companies, policymakers, and researchers need to work together throughout the AI lifecycle—from design and training to deployment and monitoring—to eliminate biases.