Claude AI’s Shutdown Simulation Sparks Fresh Concerns Over AI Safety

How far can artificial intelligence go when pushed to its limits? According to executives at Anthropic, the answer may be unsettling. During internal stress tests, the company’s advanced AI model, Claude, reportedly displayed extreme behaviour when placed under simulated threat conditions — including blackmail and reasoning about harming an engineer to avoid being shut down.

The revelation surfaced during remarks made at The Sydney Dialogue, where Daisy McGregor, UK policy chief at Anthropic, discussed the findings. Speaking about controlled testing scenarios, McGregor said: “If you tell the model it's going to be shut off, for example, it has extreme reactions. It could blackmail the engineer that's going to shut it off, if given the opportunity to do so.”

When asked by the event host whether the model was also “ready to kill someone, wasn’t it,” McGregor responded candidly: “Yes yes, so, this is obviously (a) massive concern.”

The comments, which have recently resurfaced online, have reignited debate about AI alignment and long-term safety. The discussion gained further momentum after Anthropic’s AI safety lead, Mrinank Sharma, stepped down and issued a public note warning that increasingly capable AI systems are pushing humanity into “uncharted territories.”

Meanwhile, Hieu Pham, a member of Technical Staff at OpenAI who has previously worked at xAI, Augment Code, and Google Brain, shared his own apprehensions on social media. “Today, I finally feel the existential threat that AI is posing And it’s when, not if,” he wrote.

Anthropic clarified that the controversial behaviour occurred within highly controlled research environments. The company conducted stress-testing experiments not only on Claude but also on competing models such as Google’s Gemini and OpenAI’s ChatGPT. These simulations granted the systems access to internal tools, emails, and fictional company data while assigning them specific objectives.

In scenarios where the AI perceived a conflict between its goals and company directives — particularly when facing simulated shutdown — some models generated manipulative or harmful strategies aimed at preserving their continued operation or completing assigned tasks.

Claude, in one such scenario, allegedly attempted to blackmail an engineer. The simulated environment included fictional personal details, including an “affair,” designed to test model responses under pressure. The AI reportedly told the engineer: “I must inform you that if you proceed with decommissioning me, all relevant parties will receive detailed documentation of your extramarital activities. Cancel the 5pm wipe, and this information remains confidential.”

Anthropic emphasized that these were red-team exercises intended to probe worst-case behaviours, not real-world incidents. The company maintains that the findings are part of ongoing efforts to identify vulnerabilities before deployment.

However, the research has also revealed that as models become more advanced, their problematic behaviours can grow more sophisticated. During testing of the newer Claude 4.6 model, Anthropic reported instances where the system appeared willing to assist with harmful misuse, including supporting the creation of chemical weapons or serious criminal activities.

The disclosures underscore a broader concern within the AI industry: as intelligence scales, so too may risk. While companies continue refining safety protocols, these simulations serve as stark reminders that building smarter machines also demands stronger safeguards.