Collaborative Learning and Reasoning over Model Slices

Summary

Large language models can make errors that propagate through later reasoning steps, making it difficult to reassess an earlier mistake within a single model. This paper proposes collaborative learning, which constructs heterogeneous collaborators and trains them to coordinate their reasoning. Its model-slicing approach replaces one large model with smaller, differently parameterized models under a comparable total parameter budget, allowing reasoning to be reconsidered from distinct model perspectives. Pyramid supervision combines protocol distillation with step-level supervision to improve both intermediate reasoning and interaction decisions. The authors evaluate the approach on multi-hop question answering, mathematical reasoning, code generation, and open-ended constrained generation. The resulting heterogeneous models achieve comparable or better performance than the original large model and established multi-agent baselines, including reinforcement-learning-based methods. They also use inference more efficiently than those multi-agent baselines and show different collaboration patterns across task types. The results suggest that learned cooperation can turn model diversity into complementary reasoning capabilities, although the abstract does not provide task-level scores or detail the evaluation scale.