Jev Puts AI Decisions Inside Software State Machines

Summary

The article proposes using non-autoregressive decision models to choose the next legal transition in a software state machine, such as which onboarding screen a user should see. TypeSafe AI’s Jev accepts application state and returns typed choices, scores, or truth estimates with probabilities and confidence, rather than generating text. Developers define the available actions and enforce constraints such as permissions and billing rules in ordinary code; the model only selects among actions already allowed. The article compares Jev, a hosted service with roughly 115 ms round-trip latency, to Von, an Apache-2.0 model that can run locally in about 18 ms. On the 49-task jabr v2 suite, Jev scores 96.6% macro accuracy versus Von’s 72.0%; on Choice routing, the scores are 96.8% and 83.0%. The author argues that these tradeoffs make Von useful when local speed matters and Jev preferable when the measured accuracy advantage justifies a network dependency. The article also warns that personalized routing can undermine experiment attribution: prediction that a journey will work better does not establish causation, so teams should test decision policies and separately log exposure and outcomes. Both models have limits: they can misread underspecified or irrelevant context, Jev is not intended for arithmetic or date calculations, and validly typed answers may still be wrong. The proposed architecture therefore uses confidence thresholds to decide when to automate, fall back to deterministic behavior, or escalate. A deterministic baseline, called System 0, must keep the application functioning when a model or network is unavailable. The author has not shipped this design and presents it as an architecture to try, not a validated production pattern.