Task Difficulty Determines Whether Chain-of-Thought Carries the Answer

Summary

The study introduces continuation-based causal testing to determine whether a model’s written chain-of-thought actually constrains its final answer. The method perturbs one reasoning step, truncates the chain, and requires the model to continue from the corrupted prefix, measuring behavioral load-bearingness rather than mechanistic faithfulness. Tests on Gemma-2-9B-IT, Llama-3.1-8B-Instruct, and DeepSeek-R1-Distill-Qwen-7B across GSM8K, MMLU, and BIG-Bench Hard show that load-bearingness follows model-relative task difficulty. On easier tasks, models often silently bypass their own traces; on harder tasks, they follow corrupted steps and propagate errors. In a matched 2x2 analysis, error propagation increased 16-fold between GSM8K and BIG-Bench Hard multistep arithmetic, while a variance analysis of 28,584 continuations attributed 98.8% of explained deviance to task difficulty and 0.8% to perturbation type. Reasoning-focused reinforcement learning reduced error propagation and compressed this gradient. A four-variant judge analysis and a blind two-annotator study of 500 cases found stable labels, with Cohen’s kappa of 1.00. Hidden-state probes separated bypass, self-correction, and error-propagation modes, but activation steering changed only about 25% of error-propagation cases at best. The authors conclude that chain-of-thought is easiest to read when it carries little signal and most consequential when errors can spread before oversight intervenes; its behavioral mode is readable but not reliably controllable.