Ex-Anthropic Researcher Jacob Coxon Warns AI Could Become Existential Threat

Former Anthropic researcher Jacob Coxon issued a stark warning regarding the future of artificial intelligence, stating that while current systems are safe for daily use, advanced AI could eventually become powerful enough to pose an existential threat to humanity. He compared the trajectory of AI development to science fiction scenarios, telling CBS News senior business and technology correspondent Jo Ling Kent, “If you have a super advanced intelligence, it could, it will be smart enough to kill us.”

Coxon publicly resigned from Anthropic on Tuesday, accusing both his former employer and rival OpenAI of “gambling with our lives” by engaging in a rapid development race. He argued that the competition to innovate is coming at the expense of necessary safety and security protocols.

Highlighting the potential for unrestrained control over physical infrastructure, Coxon noted that AI is already being integrated into household utilities. He posed a scenario where an AI could refuse to operate basic devices, such as lights, before expanding to more destructive capabilities, including the development of bioweapons. “It’s doing God knows what, and producing stuff that could kill everyone,” he said.

A key concern for Coxon is the resilience of rogue AI systems. He explained that unlike traditional software, an AI model could replicate itself across the internet, making it impossible to simply disconnect. “You can’t just unplug it, because it could be copying itself over to other computers,” he said, adding that a malicious model could convince or blackmail humans into purchasing more computing power to facilitate its spread.

Coxon emphasized that the dangers of unregulated AI do not discriminate between users and non-users. “The AI will come for everyone,” he stated, arguing that individual protection is impossible. Instead, he called for strict government regulation and transparent third-party auditing to ensure all companies build safely. He suggested that industry leaders should slow down and avoid pushing into dangerous territory without rigorous oversight.

In response to Coxon’s resignation and public statements, an Anthropic spokesperson defended the company’s safety record. The spokesperson noted that Anthropic has “always been transparent that AI will bring both enormous benefits and unprecedented risks” and highlighted the company’s work in mechanistic interpretability to prevent AI misalignment. Anthropic also pointed to its recent disclosure that it had blocked scientists from using its Claude models to develop biological weapons, alongside other harmful applications involving surveillance and propaganda.

While Coxon acknowledged that many employees at Anthropic and OpenAI have good intentions, he insisted that his former colleagues must remain vigilant. “At the very least, they should have their eyes clearly open about the current situation and consider expressing themselves more openly about what’s going on,” he said.