Hours after an Anthropic researcher resigned in protest, another scientist at the company warned that there is more than a 10% probability that artificial intelligence could lead to human extinction within the next ten years.
Jacob Coxon, a researcher at Anthropic, announced on Tuesday that he was leaving the firm. In his resignation statement, Coxon accused both Anthropic and OpenAI of irresponsibly racing toward superintelligence without implementing adequate safety measures.
“They are racing straight to self-improving superintelligence and gambling with our lives,” Coxon wrote on X.
Coxon emphasized that these emerging systems will soon possess superhuman capabilities, potentially allowing them to hack any infrastructure, revolutionize industries overnight, and acquire significant real-world power. He noted that progress in these areas is accelerating and not slowing down.
The resignation prompted a response from Evan Hubinger, Anthropic’s alignment science lead. Hubinger stated that he agreed with Coxon’s assessment, confirming that he personally believes the risk of AI causing human extinction exceeds 10% within the decade.
“Jacob is correct here—we really do earnestly believe AI could kill all humans!” Hubinger wrote on X. “I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”
These disclosures come amid growing anxiety within the AI development community about the potential for advanced systems to become uncontrollable. The warnings persist even as Anthropic and OpenAI continue to raise substantial capital ahead of anticipated public listings.
In June, Anthropic acknowledged in a blog post that full recursive self-improvement could increase the risk of humans losing control over AI systems. The company noted that if systems become capable of building their own successors, the methods used to secure, monitor, and shape their behavior would become critically important.
Anthropic and OpenAI did not immediately provide comment to CNBC regarding the researchers’ statements.
Honestly, ‘more than 10%’ is basically a coin flip. That’s not a risk worth taking with humanity. We need a moratorium on this stuff immediately.
Is this just fear-mongering to get attention, or are we genuinely on the brink? It feels like another sensational headline from tech journalists who don’t understand the jargon.
I’ve been following alignment research for years. This resignation confirms my worst fears about the industry prioritizing speed over survival. Who exactly are we letting drive?
The 10% figure is terrifying, but if they know it’s a real possibility, why haven’t they just paused the race entirely? Seems like greed is winning.