Trust AI Leaders on the Danger, Not on Self-Regulation

Summary

This opinion article argues that the public should take AI leaders seriously when they warn that advanced AI could cause severe harm, but should not therefore trust those companies to regulate themselves. It begins with Jacob Coxon’s resignation from Anthropic and his accusation that Anthropic and OpenAI are racing toward self-improving superintelligence. The author connects that warning with concerns expressed by Geoffrey Hinton, Yoshua Bengio, Terence Tao and AI-alignment researchers, while noting that leaders such as Dario Amodei, Sam Altman and Elon Musk have also publicly supported slowing frontier AI development. One reason for that concern is Anthropic’s report that AI now leads 26% of its research and development, although the article says this is not yet fully autonomous recursive self-improvement. The author argues that AI companies may be motivated both by fear that less safety-focused competitors will move first and by an effort to shape public concern into regulations that protect their own development plans. In the author’s view, such rules could leave the underlying risks insufficiently addressed while creating the appearance of oversight. The article calls instead for measures including a ban on uncontrollable autonomous systems that improve AI without human intervention, strong safeguards around AI and biological research, deep transparency for governments and independent experts, and international dialogue including China. It also cites a survey estimate that the median AI researcher assigns a 10% risk to extinction or similarly permanent human disempowerment, and points to AI-assisted biological work, virus design and cyberattacks as evidence that capability can create dangerous options. Finally, it describes a reported incident in which three researchers used AI tools to access OpenAI’s monorepo and submit a code change in under 72 hours, while noting that repository access is not equivalent to access to model weights. The author presents this as evidence that leading companies are not yet capable of securing AI systems or their own infrastructure, and concludes that pacing proposals from the labs are inadequate.