What Multi-Harness Reinforcement Learning Learns About Coding-Agent Portability

Summary

This study examines whether comparing rewards across execution harnesses helps reinforcement-learned coding agents become more portable. Starting from one Qwen3-8B supervised warm start, the authors replay the same frozen task-harness records from Aider, OpenHands, Qwen Code, and SWE-agent with equal update counts. They compare Within, which forms one relative-advantage group for each task-harness pair, with Cross, which pools harnesses within each task. Every checkpoint is evaluated with a sealed SWE-bench Verified oracle on four source harnesses and on a minimal harness excluded from training. Across 24,000 sealed evaluations, the evaluation harness changes mean solve rate from 2.14% to 9.27%, a 4.3-fold range, while the training recipe changes it by a factor of 1.16. On the unseen harness, Cross exceeds Within by only 0.25 percentage points at eight attempts per task, with a 95% confidence interval of -0.48 to +1.02; across three seeds, the difference is 0.16 points with an interval of -0.41 to +0.72. Each method’s seed range, 0.42 to 0.45 points, is larger than the difference between methods, and their largest gains occur on the same source harness. An out-of-fold classifier can identify the generating harness from Cross’s advantage, but not from Within’s, indicating configuration adaptation rather than portable capability. Re-collecting half the data on-policy does not change the result. The authors recommend reporting the grouping boundary and testing multi-harness RL on an unseen harness.