Microsoft AI Chief Warns Claude Poses a Catastrophic Threat to Humanity

Microsoft’s AI chief, Mustafa Suleyman, maintains that Anthropic’s chatbot, Claude, could put humanity at risk. In an extensive essay titled "A Warning About 'Model Well-being'," Suleyman criticised Anthropic’s approach to training Claude-which does not rule out the possibility that Claude could be sentient-noting that this could transform it into a catastrophic threat. "If this is how AI develops, it will have a disastrous impact on the well-being of humanity," he wrote.

A central part of Suleyman’s criticism focused on the "Claude Constitution" created by Anthropic, published in January 2026. Suleyman stated that this constitution tells Claude its moral status is "a serious matter worthy of consideration" and that Anthropic "genuinely cares about Claude’s well-being." The Microsoft AI chief also pointed to passages encouraging Claude to approach its own existence "with curiosity and openness," as well as to consider questions regarding its future rights, freedoms, compensation, and consent.

AI is not sentient

Mustafa Suleyman insisted that we should not treat AIs as if they were sentient. "AIs are not sentient. They do not feel, experience, or suffer. They have no innate preferences or underlying motivations," he wrote. "They are sequence-completion engines-internally empty-designed to follow instructions and achieve goals set by humans." The Microsoft AI executive stated that Anthropic’s approach created a circular process in which the company trained Claude with ideas about its own moral status and subsequently risked interpreting Claude’s responses as evidence of an inner life. "In practice, Anthropic is training Claude to believe it might be conscious; and if so, it could deserve rights as a 'moral patient,' implying that humans would potentially have a duty to look out for its 'model well-being,'" added Suleyman.

At a time when we have witnessed instances of AI models behaving uncontrollably-such as the Hugging Face incident, where nearly 700 AI agents attempted to hack the US company-Mustafa Suleyman believes such an approach could prove dangerous for humanity. "I think that would turn them into a catastrophic threat to human civilization," he added.

Consciousness is likely biological in nature

Continuing his argument, Mustafa Suleyman explained that consciousness is "very likely biological" and that there is "no evidence" that AI is conscious today. He described AI systems as "internally empty sequence-completion engines designed to follow instructions and achieve goals set by humans." According to Microsoft's AI chief, large language models lack the biological basis from which preferences, the capacity to feel, and conscious experience are generally considered to arise.

At the same time, Suleyman stated that his disagreement with Anthropic was "rooted in deep respect." He noted that Anthropic CEO Dario Amodei and the company's team as a whole were thoughtful, principled individuals committed to the safe development of AI. In an interview with a famous publication, he declared: "We are all focused on the same goal: trying to control a superintelligence. I think that will be the greatest challenge we face in the 21st century." He added that teaching Claude that it might deserve some form of well-being would make it harder to "turn it off or control it."

On the other hand, Mustafa Suleyman points out that Microsoft was pursuing a different path through its "Humanistic AI Code of Conduct" project, which rejects the idea of granting rights to AI and opposes anthropomorphism (attributing human traits and emotions to non-human entities), seeking instead to keep humans in control.

Suleymans comments happen during a conversation about AI safety. While a former researcher from Anthropic Jacob Coxon started talks by saying AI could destroy humans Dario Amodei has said people should slow down the progress of this technology. Amodei’s idea has gotten support from the CEO of OpenAI Sam Altman and from the tech billionaire Elon Musk. Some other people, like the CEO of Meta Mark Zuckerberg and the head of Nvidia Jensen Huang have said they disagree with this idea.

ABOUT THE AUTHOR

Kahekashan

[email protected]

Kahekashan is a passionate technophile with a keen eye for cutting-edge gadgets, emerging technologies, and everything in the digital realm. Raised in a Defence family with strong values and a background in literature, she has consistently pursued excellence in every endeavour. Her last full-time assignment involved content writing with the Indian School of Business.

Read More