What If LLMs Make Local Failures Cheap but Systemic Risk Harder to See?

Summary

This opinion essay asks whether the growing ability of large language models and agents to generate, review, debug, refactor, and replace code could alter the distribution of systemic risk. Its central analogy is a building whose small cracks are instantly patched: local failures become cheaper, but the human attention that once investigated recurring cracks may disappear even as underlying structural weaknesses remain. The author does not claim that AI-written code is inherently bad; the concern is that cheap repair could make small warnings less visible and potentially leave rarer, more correlated failures harder to detect. The essay extends the question from software to society. Humans have different goals, incentives, histories, and judgments, and their friction may provide information and counterweights within organizations and civilization. If shared models, post-training methods, interfaces, agent frameworks, and abstractions mediate increasingly large portions of everyone’s work, the author asks whether many distinct human objectives could be projected into a smaller behavioral space, reducing decision diversity and introducing shared blind spots. The article compares expert intuition to a compressed model formed through decades of feedback, while noting that such intuition can be difficult to explain and may also be wrong. It uses inexpensive interventions, such as suggesting a walk, to illustrate that when actions become cheap, the harder problem may be knowing where to aim and how to interpret the result. On this view, the AI era may be a sensing problem disguised as a generation problem: anomaly detection, indexing, goal preservation, disagreement, and knowing when not to act could matter more as outputs multiply. The author proposes measurement rather than a verdict, calling for logs of objectives, environments, constraints, human and machine decisions, available information, timestamps, and failures. The open questions are whether human diversity persists when LLMs perform much of intermediate cognitive work, whether cheap local fixes alter tail risk, and whether disagreement that once exposed imbalance is being removed from the system.