Separate Answer Calls Help Language Models Follow New Evidence

Summary

Reasoning traces can improve an AI model’s accuracy, but they can also conflict with newly supplied evidence. This study separates those effects by pairing verified evidence versions with the reasoning traces generated from each version, then answering from identical trace-evidence pairs in two configurations: continuing the model’s own reasoning or making a separate answer call. Across GPT-OSS, Gemma4, and Qwen3 on HotpotQA and MuSiQue, the separate call improves evidence following over continuation by 29.9 to 60.6 percentage points. Removing the trace reduces accuracy by similar amounts in both configurations, showing that a trace can be useful even when the model does not defer to it. In conflicts, continuation usually repeats the stale conclusion, while a separate call follows the current evidence much more often. Additional analyses find that the final line of a trace carries most of its benefit, that gold evidence is most helpful before the trace is written, and that distractors reduce correction under a separate call. Renewed reasoning follows current evidence in most conflicts, whereas a verification instruction does not. The authors conclude that trace deletion alone cannot show which information source an answer followed, so evaluations should report both trace benefit and evidence deference together with the answer configuration.