Anthropic Gives Claude the Ability to Exit Abusive Conversations

Anthropic has introduced a new safeguard in its consumer chatbots, giving Claude Opus 4 and 4.1 the ability to end conversations in extreme cases of persistent harm or abuse.

The company said the feature is intended for “rare, edge scenarios” where repeated refusals and redirection attempts fail, and users continue with abusive or harmful requests. 

Claude can also end a chat if a user explicitly asks it to. Notably, the system is directed not to use this ability when users may be at imminent risk of self-harm or harming others.

Anthropic described the move as part of its exploratory work on potential AI welfare. “We remain highly uncertain about the potential moral status of Claude and other LLMs, now or in the future,” the company wrote. 

“However, we take the issue seriously, and… we’re working to identify and implement low-cost interventions to mitigate risks to model welfare, in case such welfare is possible,” it noted. 

Pre-deployment testing of Claude Opus 4 included what Anthropic called a “preliminary model welfare assessment.” According to the company, the model showed a strong aversion to harmful requests, signs of distress in abusive conversations, and a tendency to end chats when given the option during simulations.

When Claude ends a conversation, users cannot send new messages in that thread. Still, they can immediately start a new chat, provide feedback, or edit and retry previous messages to branch into a fresh conversation.

Anthropic said the vast majority of users will not notice this change, even in discussions of sensitive topics. The company is treating the capability as an experiment and will refine the approach based on feedback.