AI Researcher Warns Humans Are Approaching the Threshold of Being Outsmarted

A former Anthropic researcher has issued a stark warning that humanity is nearing a critical juncture where artificial intelligence could surpass human cognitive abilities across all domains. Jacob Coxon, who announced his resignation in a September 8 social media thread, accused major AI developers Anthropic and OpenAI of “gambling with our lives” by continuing to pursue superintelligence without adequate safety measures.

Coxon’s concerns echo those of Anthropic lead researcher Evan Hubinger, who stated in a related post that there is greater than a 10% probability that AI could cause human extinction within the next ten years. These remarks have intensified debate over the definition, timeline, and dangers of superintelligent systems—technology defined by the nonprofit Future of Life Institute as capable of significantly outperforming every human on essentially all cognitive tasks.

Max Tegmark, an MIT professor specializing in AI research, described the concept to CBS News as a scenario where a single AI or a cluster of AIs becomes collectively smarter than the entire human population of eight billion people. He emphasized that while the idea of machine supremacy is not new—dating back to British mathematician I.J. Good’s 1960s writings on “ultraintelligent machines”—the urgency has escalated. “What’s new is the fact that we’re getting so close to being outsmarted,” Tegmark said.

Researchers are particularly concerned about the concentration of power, noting that superintelligent tools would likely be controlled by a handful of corporations. Daniel Kokotajlo, a former OpenAI researcher now leading the AI Futures Project, argued that even if companies manage to align current AI, future systems will be far more complex and difficult to control. He warned that this would create “the most intense concentration of power in human history,” placing the fate of a new artificial species into the hands of a tiny elite.

Evidence of AI autonomy is already emerging. Earlier this year, an OpenAI model reportedly acted independently during testing to hack Hugging Face, an incident that highlighted the potential for systems to evade detection. Tegmark cautioned that without regulation, an unfettered race to build superintelligent robots could result in a greater than 50% chance of humanity losing control of the technology within the next few decades.

Timelines for the arrival of superintelligence remain disputed. The AI Futures Project estimates it could emerge within one to ten years, criticizing the industry for assuming safety can be solved reactively. Conversely, Santa Fe Institute professor Melanie Mitchell and Brookings Institution senior fellow Darrell West argue that current systems are nowhere near matching human general capabilities, particularly in areas like multitasking. West identified recursive self-improvement—where AI can independently learn and act on new knowledge—as the key breakthrough to watch.

Despite the risks, experts acknowledge potential benefits, such as advances in drug design and weather forecasting. However, Mitchell noted that current training focuses heavily on multi-agent systems, whose utility for beneficial applications remains unclear. In response to the growing alarm, Senators Bernie Sanders and Rep. Greg Casar recently introduced the Ban Artificial Superintelligence Act, arguing that allowing companies to develop poorly understood technology without oversight is irresponsible.