OBC-Prune Uses Outcome-Based Calibration to Prune Reasoning Models

Summary

Large reasoning models often generate lengthy chain-of-thought traces, increasing inference cost. Existing pruning calibration methods may treat all reasoning tokens equally, preserving weights associated with erroneous computations as readily as those associated with correct reasoning. OBC-Prune addresses this by constructing difficulty-matched pairs of correct and incorrect rollouts from problems the model answers inconsistently. It uses intervention-based analysis to estimate how much each reasoning sentence affects subsequent predictions, then converts those estimates into per-token weights for calibration. The weights can be used with one-shot pruning methods including SparseGPT, Wanda, and ALPS, without changing the pruning algorithms themselves. Experiments on DeepSeek-R1-Distill-Qwen models with 1.5B, 7B, and 14B parameters at 40% and 50% sparsity showed consistent gains over state-of-the-art calibration baselines across most model sizes and sparsity settings. Tests on MATH500, LiveCodeBench, and AIME 2025 support the paper’s claim that preserving causally important reasoning circuits is more effective than preserving activations based only on statistical salience.