AI Applications Need Primitives Beyond Text Generation

Summary

Seldon argues that many production LLM calls are latent programs rather than indivisible text-generation tasks. A request to extract customer-supplier relationships, for example, can be decomposed into entity recognition, normalization, candidate generation, entity linking, relation extraction, and schema validation; support-ticket triage similarly combines classification, sentiment, ranking, account resolution, and validation. The proposed approach gives each operation a typed interface, such as a fixed-label classification, an ordered score, a probability, a span, a canonical identity, or exact arithmetic, while preserving the source text alongside structured outputs. Seldon describes a v1 taxonomy of seven families and 41 subgroups: classification, sequence or span labeling, structured extraction, entity matching or retrieval, similarity or pairing, normalization or transformation, and numeric or analytical operations. Each operation can be lowered to rules, indexes, formula engines, small classifiers, sequence models, embedding systems, or larger language models, while calls that do not fit the vocabulary remain on a frontier model. The system first searches production traces, including inputs, outputs, corrections, and failures, to recover a candidate program; compilation and implementation selection follow that synthesis step. Its proposed runtime can combine deterministic preprocessing, cheaper models, retrieval, validation, and a frontier fallback behind a novelty or confidence gate. Seldon says this can reduce cost when cheap operators agree with the original call, but emphasizes that the diagrams do not prove savings, confidence must be calibrated before thresholds automate decisions, and held-out traces measure performance on a distribution rather than proving semantic equivalence. The taxonomy is explicitly provisional and may be adapted by domains such as law, finance, healthcare, and logistics. Seldon presents the approach through an open beta for routing traffic into such pipelines.