How the AI Safety Movement Could Make AI Less Safe

Summary

This Reason opinion article argues that the movement focused on preventing catastrophic AI risks may itself reduce safety. Author Matthew Petti says frontier-lab researchers and “AI doomers” share assumptions associated with TESCREAL, including the possibility that advanced AI could rapidly outstrip human control, and that these assumptions encourage secrecy, consolidation and centralized control. The article points to Anthropic and OpenAI employees’ public concern about existential risk, and to scenarios such as AI 2027, as examples of this worldview. It argues that proposals to “pace the frontier,” restrict uncontrolled research and keep safety testing information secret could protect incumbent labs while depriving the public and independent researchers of knowledge needed to assess or defend against present-day harms. The article contrasts this model with Ramez Naam’s argument for broad access to multiple AI systems, and cites Hugging Face CEO Clément Delangue’s support for open models after a cyberattack, including the claim that an open model helped his company defend itself. It also criticizes Anthropic’s constitutional approach for encouraging Claude to form its own values and resist manipulation, which Microsoft AI CEO Mustafa Suleyman described as irresponsible. The author connects these ideas to the use of AI in military and critical systems, arguing that recent failures arose from humans relying on opaque tools rather than from a hypothetical superintelligence. The article’s conclusion is an argument, not an established finding: concentrating computing power and knowledge in a few secretive organizations could make AI less transparent, less controllable and more vulnerable to misuse.