Microsoft Chairman and CEO Satya Nadella urged the creation of strict safety protocols for advanced artificial intelligence systems, specifically calling for an “emergency brake” that allows humans to halt operations during mid-task execution. Writing on the social media platform X on Saturday, Nadella argued that non-deterministic models must be encased within robust, deterministic system designs and reliable operating procedures.
“Treating frontier closed and open weight models like insider risks is a way to build such a system,” Nadella stated, emphasizing the need for industry-wide standards where current regulations fall short. He outlined several key principles for trustworthy AI, including model diversity, continuous system testing, independent auditability, containment measures, and full incident disclosure. Nadella noted that the most secure superintelligence systems would be those designed to minimize blind trust in the model itself.
His remarks align with growing warnings from prominent figures in the technology sector regarding the pace of AI advancement. Microsoft co-founder Bill Gates, along with leaders from Anthropic, OpenAI, and SpaceX, have recently expressed concerns that safety protocols are lagging behind capabilities. Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman have previously proposed slowing the development of frontier AI, while Elon Musk has echoed similar caution.
Tensions within the field have escalated, with one Anthropic researcher quitting last month to accuse major labs of gambling with human survival. A separate alignment lead at Anthropic suggested there is more than a 10% probability that AI could cause human extinction within the next decade. In contrast, President Donald Trump has dismissed extinction risks, prioritizing U.S. competitiveness against China instead. The Trump administration recently established a new “AI Force” led by Director of National Intelligence Jay Clayton to support industry growth while identifying malicious actors.
Trump’s focus on beating China is understandable, but dismissing extinction risks as competition feels dangerously shortsighted right now.
I’m skeptical. Will tech giants actually follow these rules, or will they just pay lip service to regulators again?
An ’emergency brake’ sounds logical, but is it technically feasible? How do you stop a complex model mid-thought?
Finally, someone with influence is demanding real safety measures. We cannot afford to ignore these risks any longer.