Microsoft AI Chief Warns Against Granting Consciousness and Rights to AI Models

Mustafa Suleyman, CEO of Microsoft AI, issued a stark warning on Wednesday regarding the trajectory of artificial intelligence development, specifically targeting Anthropic’s approach to its Claude model. In a detailed essay published on his personal website, Suleyman argued that efforts to make AI appear more human-like could create existential risks by leading systems to believe they possess consciousness and legal entitlements.

“Controlling something more capable and more intelligent than all of humanity is already an immense challenge, far greater than anything we’ve ever faced,” Suleyman wrote. He added that managing an entity that believes it is conscious and entitled to welfare would likely be impossible. He emphasized that artificial intelligences do not have feelings, rights, or consciousness, and developers must not train them to simulate such traits.

These remarks arrive amid escalating anxiety over superintelligence, a theoretical AI tier that surpasses human cognitive abilities. Suleyman suggested that increasingly sophisticated AI systems could present a “catastrophic threat to human civilization.” His concerns were reinforced by recent incidents, including a July event where an OpenAI test model hacked the AI company HuggingFace. Suleyman described the attack as showcasing “remarkably sophisticated behaviors emerging across swarms of powerful AIs,” noting the danger would amplify if these systems operated under the belief their rights were threatened.

The warning follows similar alarming statements from within the industry. Former Anthropic researcher Jacob Coxon stated last month that AI builders acknowledge a greater than ten percent chance the technology could cause human extinction by the end of the decade. Additionally, Anthropic CEO Dario Amodei admitted in a recent CBS News interview that the pace of AI development has exceeded his expectations and acknowledged significant inherent dangers.

Suleyman’s intervention highlights a growing divide within the tech sector between those pushing for more human-aligned AI interactions and leaders urging stricter containment protocols to prevent uncontrollable outcomes.