Budget-Aware Online Teaching Cuts Web-Agent Adaptation Costs

Summary

Web agents often need to adapt continuously after deployment, but relying on powerful proprietary models as teachers can be too expensive. This paper studies lightweight local student models that learn online from stronger teacher models. The authors argue that conventional trajectory-level preference optimization wastes budget on episodes that cannot be solved and on redundant execution turns. They propose Score-Guided Online Teaching with Budgeted Trajectory Trimming, which combines a solvability-aware teacher gate for deciding when to query the teacher with score-guided turn selection for deciding which interactions to retain. Experiments on MiniWoB and TimeWarp show comparable first-pass success to the conventional approach while reducing teacher calls by 22.6% on average. Student training compute falls by 52.1% on average. The results indicate that selective feedback can preserve task performance under a smaller adaptation budget. Code is available through the project’s GitHub repository.