Routers Are Emerging as a New Primitive for AI Model Selection

Summary

Vals argues that model routers are becoming a core component of the AI stack because the performance frontier is uneven across tasks and no single model leads all of its eight professional benchmarks. Its Index covers finance, software engineering, legal, and tax work, and the article says five models occupy the top positions across the eight benchmarks. A task-level oracle that could choose the best model in hindsight would outperform the best single model by 14 points while costing about 40% less, although real routers cannot access perfect hindsight. Routers instead select from a model pool using prompt characteristics, benchmark or usage signals, and increasingly decisions made at every step of an agent workflow. The article says long-running agents make routing harder: a cheaper model can create recovery work, changing models can sacrifice prompt-cache savings, and task decomposition is difficult to predict. Existing router benchmarks often evaluate one choice on one public question, which can miss these effects and can reward always choosing the cheapest model on saturated datasets. Vals reviews current efforts from OpenRouter, Fireworks, Not Diamond, and Ramp, then proposes independent comparisons using a common harness, private long-horizon tasks, fixed prompts and tools, documented configurations, and live end-to-end routing. Routers would be compared with the best and cheapest single models in their own pools on task outcome, cost, cache usage, and completion time, with the oracle as a reference. The Vals Index is suited to this design because six of its eight benchmarks are private, tasks are expert-written, and evaluations use zero-data-retention policies. Vals says it will publish a public router leaderboard and invites router developers to participate.