Debate-to-Skill: Capability-Bound Process Supervision for Industrial Query-to-Agent Annotation

Summary

The paper examines industrial query-to-agent matching, where topical relevance can be mistaken for an agent’s executable capability, particularly for long-tail and boundary-sensitive requests. It frames annotation as capability-bound process supervision and introduces Debate-to-Skill. The method combines reusable decision principles, structured deliberation, verifier-based verdict extraction, and refinement driven by disagreements. On an industrial Query2Agent benchmark, the authors compare it with direct-label supervision, reasoning-based supervised fine-tuning, and structural ablations. The evaluation is designed to test whether supervising the capability-critical decision process improves judgments on grey-zone cases where semantic relatedness and executable capability diverge. The abstract does not report the benchmark’s numerical results.