As frontier AI models become increasingly capable of identifying vulnerabilities and automating reasoning, they are also assisting with offensive cyber operations. The global AI industry is entering what may become its most consequential phase yet: the AI cybersecurity arms race.
That reality came sharply into focus after Anthropic recently introduced Mythos, a model that can reason about cybersecurity vulnerabilities and complex system behaviours. The model is currently limited to Project Glasswing partners, such as Google, Amazon Web Services, and Cisco, and is not accessible to the public.
While the announcement immediately sparked concerns around offensive misuse, Sarah Bird, Chief Product Officer of Responsible AI at Microsoft, believes it is essential for defenders to have access to these models.
“We absolutely have to make it available to defenders because attackers are going to find technology and they’re going to build open-source technology,” she says in an exclusive interview with AIM. “If we don’t give defenders equally or more powerful technology, it’s going to be asymmetric.”
Microsoft recently unveiled a new multi-model, agent-based cybersecurity system called MDASH that uses more than 100 AI agents to autonomously identify software vulnerabilities and security flaws. The company said the platform achieved a leading score on the CyberGym benchmark and helped uncover real-world vulnerabilities before they were publicly disclosed.
Currently, the system is tested by customers as part of a limited private preview.
The announcement comes amid growing concerns that AI could also accelerate offensive cyber operations.
Notably, a criminal hacking group recently attempted a large-scale cyberattack that appeared to leverage AI to identify a previously unknown software vulnerability, according to a Google report published on May 11.
“We have high confidence that the actor likely leveraged an AI model to support the discovery and weaponisation of this vulnerability,” the report highlighted.
Google Threat Intelligence Group (GTIG) identified a threat actor using a zero-day exploit believed to be developed with AI. The report said that "prominent cybercrime threat actors” exploited the vulnerability via a Python script.
Google states the flaw could have let attackers bypass two-factor authentication on a popular open-source system administration tool, but they still needed valid credentials to succeed.
The Google Threat Intelligence Group has detected the first known instance of a threat actor using an AI-developed zero-day exploit in the wild. While the attackers planned a wide-scale strike, our proactive counter-discovery may have prevented that from happening. This finding…
— News from Google (@NewsFromGoogle) May 12, 2026
Why it Matters
As AI becomes more adept at identifying vulnerabilities and analysing attacks, companies are beginning to use it for defence as well. OpenAI has launched a cybersecurity initiative called Daybreak, focused on using AI to improve how software is built and secured.
The company said AI enables defenders to analyse large codebases, find vulnerabilities, validate fixes, understand unfamiliar systems, and speed up remediation.
The aim is to integrate secure code review, threat modelling, patch validation, dependency risk analysis, detection, and remediation into daily workflows. OpenAI stated it is already collaborating with industry and government partners.
Earlier, the company also launched GPT-5.4-Cyber, a powerful new model fine-tuned specifically for defensive cybersecurity, while scaling its Trusted Access for Cyber (TAC) programme to thousands of verified defenders.
Frontier AI models can dramatically improve cyber defence capabilities by helping security researchers identify risks faster than humans alone. At the same time, the very same systems could also be exploited for offensive cyber operations.
Bird acknowledges that Microsoft is spending significant time evaluating both dimensions of this challenge.
“We’re spending a lot of our time at Microsoft looking at what the technology can do in terms of attacks and people using it for offensive cyber capabilities, but also how we can empower defenders,” she says.
Responsible AI Starts Before the Product Exists
Bird emphasises that the core philosophy behind Microsoft’s Responsible AI program is preventing safety from becoming an afterthought.
She explains Microsoft integrates Responsible AI throughout the product lifecycle, embedding safety reviews, testing systems, and risk evaluations from the earliest stages of development.
“We’ve invested a lot in automated testing systems,” Bird says. “These tests can run overnight and give us much more information to make decisions.”
The company has also built systems that allow engineers to rapidly introduce new guardrails whenever new AI risks emerge.
Security in the Agentic AI Era
While earlier Responsible AI debates focused on hallucinations and toxic outputs, Bird believes the industry now faces a new phase focused on autonomous agents. As companies develop AI agents that act independently, new risks emerge. "What happens if an agent goes off task?” Bird asks. “What if an agent is vulnerable to prompt injection?"
According to Bird, one of the biggest misconceptions around Responsible AI is that safety evaluations are limited to the foundation model itself. In reality, she says, trustworthiness depends on how the entire AI application behaves once deployed.
Microsoft assesses the entire AI ecosystem, emphasising application interaction, workflows, and user experience, not just the model. Bird highlights the importance of observability and control, asserting that understanding system behaviour is crucial for detecting and addressing errors.
Bird says Microsoft has contributed to observability efforts, such as the OpenTelemetry standards, while also integrating monitoring systems directly into its AI platforms. The company is additionally developing AI-native guardrails capable of identifying when an autonomous system appears to be deviating from intended behaviour.
Testing Models
Microsoft Azure AI Foundry currently lists 11,000+ AI models from providers such as OpenAI, Meta, Anthropic, Mistral, Cohere, Hugging Face, NVIDIA, xAI, and Microsoft’s own Phi models.
Microsoft conducts evaluations on these high-profile AI models to better understand their capabilities and risks before customers deploy them.
Bird explains that different AI systems come with varying safety and capability profiles, making transparency critical for developers building applications on platforms such as AI Foundry.
“These models have different profiles, they have different safety profiles, they have different capability profiles, and so we want to help users build on top of Foundry to pick the right model,” she says.
However, Bird acknowledges the scale challenge. “There are thousands of models,” she says. “We don’t test every single one of them.”
Every Person Can Be a Builder
Despite the growing concerns around security and governance, Bird remains optimistic about AI’s democratising potential.
She believes AI could dramatically expand who gets to create software and technology products. “We’re preparing for a world where every person can be a builder,” she says.
But that democratisation also introduces new risks because many first-time builders lack traditional software engineering training around security, testing, and best practices.
To address that, Microsoft is attempting to embed safety directly into its developer platforms.
“Part of what we’re trying to do is build that safety and security in so that more people can be successful from the beginning,” Bird concludes.
Your reaction
Discussion
What's your take on this story?
No comments yet — be the first to share your view.