An AI Risk Cheatsheet

Summary

This essay presents a series of arguments responding to common claims that existential risk from artificial intelligence is either fabricated by incumbent technology companies or technically implausible. It points to earlier warnings from Dario Amodei, Elon Musk, and Sam Altman, as well as professional sacrifices by researchers including Daniel Kokotajlo, Geoffrey Hinton, and Jacob Coxon, to argue that concern predates the current commercial position of major labs. It also argues that proposals such as California’s SB-53, which exempts companies below $500 million in annual gross revenue, do not straightforwardly support a regulatory-capture explanation. Against the claim that AI is merely a next-token predictor or stochastic parrot, the essay says simple objectives can produce complex behavior, notes parallels with predictive-processing theories of human intelligence, and points to reported achievements in mathematics, vulnerability discovery, and drug development. It argues that fully unplugging, sandboxing, or air-gapping useful AI would remove much of its value, while market competition gives frontier labs incentives to keep increasing capability. The author says an exact catastrophe scenario is not required to recognize that a superintelligence with goals opposed to humanity could outmaneuver humans, just as a chess player’s victory can be predictable without predicting every move. The essay further claims that current alignment methods have not eliminated lying or deception, that persistent and goal-directed behavior can increase both utility and danger, and that capability and alignment are not guaranteed to advance together under market incentives. It acknowledges that AI could help cure disease and improve living conditions, but argues that these benefits do not justify accepting catastrophic risk. The proposed pause would target general-purpose AI rather than specialized systems such as AlphaFold, preserve access to current public models, and last only until alignment understanding and safeguards improve sufficiently.