Seldon Replaces Frontier-Model Calls with Compiled Programs in Two Production Experiments

Summary

Seldon describes Seldon Functions, a system that compiles a fixed LLM call into a program assembled from smaller models, specialized models, and deterministic code. The system uses a trusted reference model to define the quality bar, searches complete programs, publishes comparison metrics, and leaves activation to the user; the original model remains available as a runtime fallback. In an agent-trace experiment, an activated program using GPT-5.6 Luna passed 76 of 80 held-out checks, compared with 74 for the GPT-5.6 Sol baseline, while costing 80% less per request and running 21% faster. On 40 external traces, the compiled pipeline achieved a task-ID ARI of 0.56 versus 0.33 for the structured Sol baseline, although both extraction arms completed only 20 traces because strict quote validation rejected long traces. In a separate PII experiment, a GLiNER span model followed by deterministic processing handled all 66 fictional documents for $0.00014, compared with about $1.14 for the Sol baseline, and was several times faster through the gateway. However, it missed some standalone first names, leaked entities in 14 documents, and achieved lower coverage than a local Presidio plus GLiNER setup. The PII task used five labels on synthetic documents and is not an anonymization certification. Compilation cost about $15 to $17 per function, with reported break-even points of roughly 950 to 2,300 calls or documents, depending on the workload and unsettled charges. Seldon says the experiments were independently designed, used held-out or fictional data, and do not benchmark PostHog or Mercor systems.