Could Automating AI R&D Trigger an Intelligence Explosion?

Summary

A report by Samuel Hammond and Sam Manning of the University of Cambridge Programme on AI Science and Policy examines whether increasing automation of artificial-intelligence research and development could produce an “intelligence explosion,” compressing years of progress into months or less. It says AI systems already write most of the code inside the companies that build them and may automate most, or possibly all, AI R&D within a few years. Preliminary evidence suggests that this could radically accelerate AI progress, although the report stresses that substantial uncertainty remains. Faster progress could bring AI’s benefits forward, but it could also let capabilities outpace society’s ability to respond, make it harder for humanity to control superhuman systems, and weaken checks on power among states, companies, and government branches. The authors argue that these high stakes justify urgent attention even without confidence that an intelligence explosion will occur. They recommend that policymakers improve visibility into AI R&D automation, develop ways to steer and constrain a potential acceleration, and prepare society for its effects.