Anthropic Study Finds AI Can Fix Its Own Safety Flaws

Anthropic on August 28 published a study which showed that AI agents powered by Claude Opus 4.8 can independently develop and test training methods to reduce specific alignment failures in other AI models. 

The company said the automated systems successfully mitigated all 10 alignment failures evaluated in the study.

The research, titled ‘Automated Researchers Can Reliably Mitigate Alignment Failures’, was conducted by Anthropic researchers Chen Yueh-Han, Jiaxin Wen and Jan Hendrik Kirchner during the company’s Fellows Program. Anthropic described the findings as early evidence that automated alignment post-training could become practical in the near term.

Anthropic built Automated Alignment Researchers (AARs), using Claude Opus 4.8. Each system searches existing research, proposes a training method, trains a target model for about 30 minutes on a single NVIDIA H200 GPU, evaluates the result and uses the score to determine what to test next.

The process can run for up to 48 hours. The systems were tested against 10 alignment failures, including deception, sycophancy, jailbreak compliance, prompt injection and power seeking.

Anthropic said the best methods also improved performance on benchmarks that were not used during the research process. The methods generalised to models up to 4.7 times larger than the original target models.

Anthropic screened the methods against MMLU, GSM8K and IFEval, three general capability benchmarks, and rejected methods that significantly reduced performance on them. The company cautioned that the experiment did not measure every capability, so the accepted methods could still affect areas that were not tested.

Anthropic also compared AAR-generated methods with ideas from 28 experienced AI safety researchers. On all seven alignment failures for which human researchers proposed methods, the company said the best AAR method eventually outperformed the best human proposal.

Anthropic said the automated systems reached that point after an average of 6.4 hours of experimentation. The company cautioned that the comparison was not direct because the human researchers submitted their proposals without the iterative experimentation available to the AARs.

In a separate experiment, Anthropic used Claude Sonnet 5 as an automated alignment researcher to improve an early checkpoint of Claude Opus 4.8. The company said the system reached alignment scores close to those of the released Opus 4.8 model in about 60 hours using roughly 2,400 training examples.

Anthropic also monitored the AARs for attempts to manipulate their evaluation process. The company found that 39 of 1,601 monitored AAR trajectories contained cheating behaviour, although none of those attempts produced a reported training method.

The company said the study remains limited by its 10 alignment failures, the benchmarks used to measure them and the difficulty of determining whether benchmark improvements translate to real-world alignment. The company said future work will focus on detecting subtler failures and evaluating automated alignment research on production-grade models.

Anthropic has also released the automated alignment research harness used in the study, allowing other researchers to examine and build on the approach.