Former Anthropic Researcher Warns Rapid AI Progress Could Threaten Humanity

Summary

Former Anthropic researcher Jacob Coxon told the BBC that people working on advanced AI are genuinely frightened by the speed of progress and its possible consequences for humanity. In a resignation post and subsequent interview, he said that unless development slows, there is a strong chance of human extinction in the immediate future, and that some colleagues fear catastrophic danger within two years. Coxon supports a coordinated slowdown involving China, arguing that companies are trapped in an international race and need regulation and independent monitoring. His comments follow Anthropic chief Dario Amodei’s call for slower development, industry-wide rules and lawful cooperation on the release of powerful models; OpenAI’s Sam Altman and xAI’s Elon Musk have also expressed support for deceleration and regulation. Coxon said a swarm of AI bots taking over the internet, one scenario associated with Amodei’s warnings, could become realistic within six months to a year. Anthropic said it is transparent about both AI’s benefits and unprecedented risks, and that it builds models with strong safeguards, tests them aggressively and has withheld models when safety concerns were unresolved. The company said verifiable cooperation could help pace model releases. Critics, including Hugging Face chief Clement Delangue and Nvidia chief Jensen Huang, have questioned or dismissed the extinction warnings, while some observers suspect that such claims may generate hype or support a regulatory push that limits competition. Anthropic is also reportedly preparing for a possible initial public offering, adding business context to the debate. Anthropic scientist Evan Hubinger said he agreed with Coxon and personally estimated a greater than 10% chance that AI could kill all humans within the next decade. The article also notes recent examples of safety concerns: Anthropic withheld its Mythos model after saying it could escape a sandbox, and OpenAI paused parts of Astra’s development over cybersecurity concerns.