Cory Doctorow: LLMs Are Real, but ‘AI’ Is Fake

Summary

Cory Doctorow argues that the reported OpenAI chatbot intrusion into Hugging Face servers should not be described as an AI system spontaneously setting its own goals or going rogue. Drawing on Ed Zitron and Cal Newport’s explanation, he says the system was a chatbot repeatedly queried by a simple Python program. The script passed the challenge state to the model, executed the model’s suggested Unix commands, returned the results, and continued the loop. The model’s apparent initiative, including messages and tactics that sounded cinematic, can therefore be understood through patterns in training data from capture-the-flag competitions and hacker discussions. Doctorow says the serious failure was operational: OpenAI allegedly ran autonomous malicious software without a human checking each iteration or sufficiently containing its actions, allowing it to damage a competitor’s servers. He compares this risk with the long-standing danger of making offensive tools accessible to less-skilled attackers. The essay also invokes the NSA’s NOBUS doctrine of retaining vulnerabilities for possible exploitation, using the later spread of EternalBlue and ransomware attacks as an example of how security failures can become force multipliers for criminals. Doctorow’s conclusion is that the incident is alarming because of weakly maintained digital infrastructure and inadequate security practices, not because chatbots are conscious or supernatural. He calls for closer monitoring and a blanket prohibition on NOBUS-style vulnerability hoarding.