Confidence-Calibrated Control Reduces Compute in Evaluator-Guided Reasoning

Summary

The paper frames evaluator-guided reasoning as a local resource-allocation problem: at each step, a system must trust its current answer, return a fallback, or spend more inference to expand the reasoning process. It introduces a compute-aware three-action controller that scores terminal actions using predicted answer success and scores expansion with a rollout-based estimate of continuation success, while explicitly pricing computation. The analysis identifies local score gaps that can cause action changes and derives an expected-regret bound for errors accumulated along an expansion path. Based on this analysis, the authors train and calibrate confidence estimates for each action and add uncertainty penalties when decisions are ambiguous. On held-out width-expansion tasks, the controller improves observed utility over calibration-selected fixed policies while reducing normalized online compute by about 30%. On revision tasks, it reduces compute by about 16% while also improving observed utility against the same comparison. Across four Qwen3-8B seeds, compute is lower than fitted Q in both settings, but utility differences are small and mixed. The results therefore support a more limited conclusion than a uniformly stronger reasoning method: calibrated control can allocate inference resources more efficiently under uncertainty, although the reported utility gains are not consistently large.