AI News

The paper studies how to place forks in tree-structured rollouts for reinforcement learning with verifiable rewards (RLVR). Forks estimate step-level credit from differences between sibling outcomes, but a fork placed after an outcome is already settled produces little useful signal, and sampling budgets usually permit only a few forks per chain. The authors formalize useful fork locations as pivots in the chain's value curve and propose belief-shift branching, which reads the model's answer belief at candidate boundaries and forks immediately before the largest divergence between consecutive beliefs. They implement the idea through three access levels: a black-box probe, a logit-lens depth profile, and an offline-trained activation direction used during validation before RL. The signal only determines fork placement and adds about 1% of step compute on mathematics and less than 5% on code when run inside the rollout engine. In validation against Monte Carlo value curves, belief shift ranks first in all eight model-by-benchmark panels, ahead of entropy, structural, and LLM-judge baselines. Across three model families and two domains, it leads every reported mathematics aggregate and every OLMo code column. On OLMo-3-7B, the reported gains over the strongest baseline are 2.6 aggregate points, 2.9 points on AIME 2026, and 6.5 points on LiveCodeBench-medium.