Key takeaways
- Nearly half of teams have deployed AI-generated code that failed in production.
- A significant portion of teams remains confident in AI code despite past failures.
- Most organisations lack thorough human review of AI agent outputs, with only a quarter reviewing extensively.
- Teams with higher review rates report fewer failures, yet issues persist across the board.
- A large percentage of teams struggle to explain AI's role in production bugs.
SmartBear released findings from its 2026 State of Software Quality and Testing report. The company said 46% of surveyed teams have shipped AI-generated code that later failed in production.
The survey covered 1,436 technology professionals whose organisations use AI in development, according to the company. Respondents are based in the United States, the United Kingdom, or other countries, as per the report.
Among teams that had shipped failed code, 69% still have a lot or complete confidence that AI-written code behaves as intended, the company said. Across all respondents, 73% of leaders hold that level of confidence, compared with 52% of practitioners, according to SmartBear.
“AI coding failures are already costing companies revenue, customers, and trust, yet leaders remain blindly confident that AI-generated code behaves as intended, even after watching it fail,” said Dan Faulkner, Chief Executive Officer of SmartBear. “Closing the gap between perception and reality demands quality and testing built for AI’s speed, complexity, and scale.”
According to the company, 95% of leaders and 86% of practitioners are confident their AI agents’ work is adequately reviewed. The company said only 25% of teams have humans reviewing more than 80% of what those agents produce.
SmartBear said teams that review more agent output reported fewer failures, though failures remained common in both groups. As per the report, 49% of teams reviewing more than 60% of agent output have shipped AI code that later failed, compared with 58% of teams reviewing 60% or less.
The company said 47% of organisations cannot explain how AI contributed to a bug or incident in production. According to the report, that figure rises to 73% among teams that have already shipped a failure.
SmartBear said 46% of teams check that the specification reflects intent before AI generates code from it. The report describes that point as the earliest and least costly place to catch a problem.
The company said 69% of US respondents now say AI writes or accelerates at least 41% of their code, compared with 43% in its January 2026 Closing the AI Software Quality Gap report. SmartBear also reported that 45% of US respondents are very or extremely concerned about application quality, up from 36% in January.
The company said 83% of respondents believe autonomous testing, in which AI agents generate, execute, adapt, and report on tests without manual scripting, would help them keep pace with AI code volume. According to SmartBear, 23% named trust in the results as the top barrier to adopting it, compared with 12% who named cost.
The company said 84% of respondents use at least one form of human review to validate AI-generated tests, while 3% rely on AI self-validation alone.
SmartBear said AI-powered testing and validation tools can help companies achieve application integrity, which it defines as continuous, measurable assurance that software works as intended.
