Know What You Don’t Know: EvoSyn Evolves Data Synthesis for Agent Learning

Summary

EvoSyn is a learning-feedback-driven multi-agent framework for synthesizing training experience for interactive agents. The paper addresses a gap in existing approaches: environment feedback, validation results, and learner failures can reveal weaknesses, but they do not directly indicate how future synthesis should change or whether the current generation space can express the missing interactions. EvoSyn maintains a revisable state that specifies which interaction factors can be deliberately generated and where synthesis effort should be directed. A role-specialized multi-agent system converts learner feedback into revisions; a Designer selects synthesis directions, and a Planner turns them into executable experiences. The framework supports adaptation within the existing generation space toward weak or under-covered regions, while also expanding that space when newly discovered factors need to become explicit generation controls. Experiments on τ²-Bench and CAR-Bench report a 13.3-point average task-success improvement over the strongest synthesis baseline. The resulting synthetic experience is also reported to be higher quality and structurally broader, with controllable factors discovered that would be missed without learner feedback.