Behavioral Self-Rewarding Improves Training for Software Engineering Agents

Summary

Reinforcement learning with verifiable rewards can reliably supervise software engineering agents through executable tests, but binary pass/fail rewards do not distinguish between trajectories with different debugging quality. This paper introduces Behavioral Self-Rewarding (BSR), which reuses the current policy to compare observable debugging behavior among trajectories from the same task that receive the same verifier outcome. BSR aggregates these pairwise preferences into task-local scores and converts them into bounded auxiliary rewards while preserving the verifier's pass/fail ordering. The method uses already-collected trajectories, so it does not require a separately trained reward model or additional environment interactions. Experiments on SWE-bench Verified and SWE-bench-Live report consistent improvements for evaluated Qwen3-14B and Qwen3-32B configurations, supporting policy-generated comparative judgments as a complementary training signal for software engineering agents.