Beyond Prompts: Optimizing Harnesses for LLM Tool Agents

Summary

This paper studies how to improve multi-turn LLM tool agents without retraining their underlying model by changing the surrounding runtime harness. Its search space is limited to prompts and guarded middleware at the tool boundary, rather than arbitrary rewrites of agent execution. The authors propose an optimizer-agnostic evaluation protocol that reports mean held-out lift, worst-condition lift, repeatability, logged cost diagnostics, and RelLift95(B), a conservative estimate of the gain selected under a fixed budget. They instantiate the approach with prompt-only and prompt-plus-middleware optimizers, including PRISM, which clusters failures and routes repairs to prompt, middleware, or joint edit surfaces within a Pareto search. On BFCL multi-round, tau2-Retail, and tau2-Telecom, PRISM achieves mean held-out improvements of 14.2, 14.9, and 10.1 percentage points, respectively, and records positive empirical RelLift95 on all three benchmarks. Component ablations attribute much of the margin to routing edits according to the failure surface and enforcing edit-pattern constraints. Across optimizers, some procedures occasionally discover large improvements but select brittle updates, leading the authors to argue that harness reliability should be reported alongside average held-out performance.