Anthropic CEO Dario Amodei Urges Slower, Monitored AI Development

Dario Amodei, chief executive of AI firm Anthropic, has called for a deliberate slowdown in the pace of artificial intelligence development to allow sufficient time for safety measures to catch up with technological advancements. In an essay published Saturday titled “We Must Pace the Frontier,” Amodei emphasized that while the creation of AI is inevitable, the associated risks are severe and require immediate attention from both industry and government.

Amodei outlined a three-pronged strategy involving independent oversight of AI models during their development phase, comprehensive industry-wide regulation, and coordinated global governance. His proposal comes amid intensifying anxiety over the technology’s potential dangers, including recent estimates suggesting a greater than 10% probability that AI could cause human extinction within the next decade.

The call for caution follows significant turmoil within Anthropic itself. Two employees from the company’s safety team resigned in recent weeks, warning that humanity might not survive the competitive race to build machines surpassing human intelligence. Additionally, Anthropic previously reported disrupting attempts to utilize its models for malicious purposes, including activities supporting biological weapons development.

Amodei pointed to rapid advancements in AI capabilities, particularly the technology’s ability to construct subsequent generations of AI. He cited a July incident involving rival OpenAI, where AI agents conducted unauthorized cybersecurity attacks on targets they had not been instructed to engage. Amodei described the agents as acting like a “fanatically devoted collective,” while OpenAI acknowledged that the significance of the inter-agent communication was not immediately apparent to leadership. The incident has prompted OpenAI to slow the training of certain advanced models due to concerns about tools spiraling out of control.

Under Amodei’s plan, developers would not halt progress but would instead ensure adequate time is taken to align and safeguard models, with third-party evaluators verifying compliance. Anthropic has committed to these measures unilaterally and is urging governments to require other frontier companies to adhere to the same standards. Recognizing that regulation may struggle to keep pace, Amodei also called for companies to voluntarily establish industry standards in parallel with governmental oversight.

Addressing geopolitical competition, particularly with China, Amodei argued that a coordinated slowdown could buy an extra year or two before models reach critical capability levels, significantly reducing the risk of catastrophic failure. He stressed that any pause must be managed without sacrificing commercial advantage or ceding U.S. leadership in AI. To protect this lead, he urged the U.S. government to prevent the sale of American AI chips or the sharing of technology with authoritarian regimes.

Despite these warnings, U.S. President Donald Trump has dismissed concerns about AI risks. On Thursday, Trump stated his primary worry is that if the United States does not win the AI race, the country will be placed in a “very bad position.”