More than 1 in 10 chance AI ‘could kill all humans,’ says Anthropic safety lead after colleague quits
What happened
Jacob Coxon, a senior safety researcher at Anthropic, resigned citing concerns over the company’s approach to AI safety. Coxon warned there is more than a 10 percent chance that artificial intelligence could “kill all humans” by the end of the decade. His departure underscores internal tensions within AI labs racing to develop powerful, potentially uncontrollable systems. Coxon criticized what he sees as a lax attitude toward risk as Anthropic and its competitors push to build superhuman AI models.
The risk
A safety lead publicly quantifying the risk of human extinction due to AI is highly unusual and signals rising alarms from inside the industry. Coxon’s estimate raises the stakes on an existential threat from AI development accelerating too fast without proper controls. It underscores that some insiders believe current guardrails and safety engineering efforts are insufficient to prevent catastrophic outcomes from emerging AI capabilities.
Why it matters
This moment pressures AI companies to tighten safety protocols and forces investors and regulators to reconsider the risk profile of AI projects. Builders may face higher costs and slower deployment as calls for rigorous safety reviews grow louder. It also weakens trust in AI firms’ self-regulation claims and could shift regulatory incentives toward stronger oversight. Operators working with AI systems need to prepare for increased scrutiny on safety measures and risk mitigation strategies.
Who should pay attention
Founders, developers, and investors in AI startups should note that safety concerns are now coming from inside leading labs, not just external critics. Regulatory bodies will likely intensify their focus on AI safety standards. Enterprise adopters of AI-powered tools should demand clearer safety assurances from vendors. Anyone betting on AI’s accelerating capability curve must price in rising friction from the growing safety backlash.
What to watch next
The pace and nature of AI regulation will be critical to watch. Anthropic’s response to Coxon’s resignation will reveal whether it shifts toward more conservative safety practices or doubles down on rapid capability growth. Industry-wide, it is worth observing if other insiders echo these concerns or if this sparks a broader push for standardized safety benchmarks and transparent risk assessments.
AI Quick Briefs Editorial Desk