The Decoherence Trap: How AI Could Make Software Complexity Run Away

Summary

The essay introduces “decoherence” as the accumulation of locally valid but incompatible or redundant models that make future changes increasingly difficult. It argues that software is especially vulnerable because incremental patches can create a maze of dependencies, paradigms, and undocumented assumptions whose interactions grow combinatorially. A graph-based simulation is presented to show how local decisions can either preserve coherence or make later work slower; the author stresses that the simulation uses arbitrary weights and is illustrative rather than proof. The essay connects this risk to AI coding agents, arguing that faster code production is not equivalent to more finished work or better software. It cites a Harvard working paper reporting 30% more code but no measurable increase in completed work after coding-agent adoption, 49% longer pull-request review times, and nearly twice as many requests for changes. It also cites a Carnegie Mellon study of Cursor adoption that found an initial velocity increase followed by persistent complexity, and an analysis of more than 300,000 AI-authored commits across over 6,000 repositories that reported quality issues in roughly one in eleven commits, with some assistants reaching one in seven, and nearly a quarter of those issues remaining. The author argues that agents may therefore accelerate decoherence unless humans maintain architectural coherence and finish repair work. The essay extends the argument to rapid AI self-improvement: games and formal mathematics can be evaluated inside a data-center “bubble,” but useful software depends on changing real-world contexts, making coherence difficult to measure without external feedback. It concludes that this creates a constraint on isolated software-intelligence explosions and presents “Decoherence Collapse” as a more immediate risk: systems may become so complex that organizations depend on agents to maintain them while agents are unable to restore coherence. The proposed response is to slow or stop destabilizing change, invest in coherent abstractions and open-source processes, and keep humans directing AI-assisted work.