How Accurate Have AI Progress Forecasts Been So Far?
Summary
The Forecasting Research Institute reviews the accuracy of AI progress forecasts collected from mid-2022 through August 2026 across capabilities, adoption, impacts, and geopolitics and governance. Its clearest finding is that experts and superforecasters repeatedly underestimated performance on AI benchmarks. In the 2022 XPT study, they assigned low probabilities to later observed results across MATH, MMLU, QuALITY, and IMO Gold Medal benchmarks; the median expert expected IMO gold-level performance in 2030 and the median superforecaster in 2035, while the result arrived in July 2025. More recent forecasts show the same pattern: FrontierMath resolved at 40.7% against median forecasts of 31% from experts and 30% from superforecasters, while LiveCodeBench Pro (Hard) had reached 53.8% in May 2026 versus end-2026 forecasts of 14% and 12%. Forecasters also appear to have underestimated AI-company revenue growth: a question with median forecasts of $20 billion, $16 billion, and $25 billion for the largest AI company’s 2026 ARR was already near $100 billion by September 22, 2026, according to the article’s assessment. The record on adoption and diffusion is more mixed: work-hour assistance and some infrastructure measures were closer to observed or projected values, while experts overestimated autonomous-vehicle ride-hailing adoption and underestimated spending growth on leading models. Forecasters overestimated the share of STEM students completing all three biorisk laboratory tasks with LLM help, predicting 16.2% to 40% against an actual randomized-trial result of 5.2%. There is not yet enough resolved evidence to judge forecasts of macroeconomic, societal, or major-risk impacts. FRI cautions that early-resolving questions and interim comparisons with later LLM projections bias the analysis toward detecting underestimates, that median forecasts can hide better-performing subgroups, and that most questions remain unresolved. The institute plans to update the Longitudinal Expert AI Panel and highlight faster-progress, more accurate, and continuously updated forecasts.