Can AI Self-Improvement Overcome Diminishing Returns?

Summary

Ramez Naam argues that AI is already helping researchers and engineers improve AI, but current evidence does not show that this feedback loop is strong enough to produce a rapid takeoff to artificial superintelligence. He distinguishes productivity gains and increasingly autonomous improvement from a runaway Type 5 loop, and says narrow superintelligence is already present in highly verifiable domains such as games, formal mathematics, and parts of coding. The article emphasizes a large gap between benchmark forecasts and real-world research performance: OpenAI data put the 80%-success research horizon at roughly 15 minutes, compared with about four hours for a cited METR benchmark horizon and 11 hours in an AI 2027 forecast. Inside OpenAI, researchers used 124 times more tokens per person, produced about seven times as many lines of code, and ran 1.6 times as many experiments per researcher, but these intermediate gains did not translate proportionally into research progress. Anthropic reported roughly fourfold productivity uplift in a staff survey, while estimating that about 40 times greater productivity would be needed to double overall AI progress through that channel. The article also discusses logarithmic returns from test-time compute, weaker-than-linear gains from agent swarms, and the difficulty of generating novel research ideas rather than incremental experiments. A model by Cunningham and colleagues places the self-sustaining RSI threshold around 15% to 19% more research productivity per additional ECI point; Naam’s rough calibration from OpenAI’s experiment data suggests only about 2% to 3% today, implying a five- to tenfold gap. He stresses that these estimates are uncertain and that hardware, investment, better data, memory, research judgment, or a major architectural breakthrough could change the outlook. His conclusion is skeptical of a near-term runaway takeoff, while calling for better shared measurements of how AI-assisted research produces genuinely better AI.