OrchSLM Studies How Small Language Models Can Be Orchestrated

Summary

Large language models can be difficult to deploy in agentic pipelines because they require substantial cloud infrastructure and can introduce latency, privacy, connectivity, and computational-cost challenges. The paper argues that specialized small language models may be better suited to many repetitive, narrowly defined subtasks. However, their limited capacity and context windows can make long-horizon reasoning and interaction-heavy methods such as iterative verification or debate difficult. It introduces OrchSLM, a routing framework that unifies non-interactive orchestration methods: heterogeneous SLMs independently produce candidate solutions, and a router coordinates cached samples without additional model interaction. The framework exposes design choices as controllable parameters, allowing the authors to study how orchestration behavior changes with task structure, model-pool composition, and multi-agent consensus.