AI researcher Jacob Coxon resigned from Anthropic, citing ethical concerns that the industry's race toward self-improving superintelligence poses an existential threat to humanity that companies are failing to address.

Key Points

• Jacob Coxon, a former researcher at both Anthropic and OpenAI, resigned over fears that AI development could lead to human extinction.
• Anthropic team leader Evan Hubinger publicly supported Coxon’s claims, stating he believes there is a greater than 10% chance AI could kill all humans within a decade.
• The viral post has garnered over 156 million views, prompting Illinois Governor JB Pritzker to call for immediate industry and government intervention.
• Industry experts note that intense competition for market dominance and upcoming IPOs are pressuring companies to prioritize speed over safety protocols.
• Previous departures, including former Anthropic safety lead Mrinank Sharma, highlight a growing trend of researchers leaving major firms due to ethical disagreements.

Why it Matters

The public resignation highlights a deepening divide between the rapid commercialization of AI and the internal warnings from experts regarding the technology's long-term safety. This tension creates significant regulatory pressure and reputational risk for industry leaders as they balance the pursuit of artificial general intelligence with the potential for catastrophic outcomes.