Neurosymbolic Routing Improves Language-Model Reasoning on Edge Devices

Summary

Small language models that run on edge devices offer private, low-latency reasoning, but they can be unreliable on arithmetic, algebra, and formal-logic tasks. This paper presents a neurosymbolic router that classifies each query and sends structurally deterministic tasks to exact symbolic solvers, reserving the language model for open-ended word problems. The routing policy is learned as a deterministic finite automaton with the L* grammatical-inference algorithm, using the small language model as a membership oracle and labeled data as an equivalence oracle. On a GPU-free Raspberry Pi 4B with 8 GB of RAM, the system achieved 100% routing accuracy and 98.3% overall accuracy on 100 previously untested prompts from DeepMind Mathematics, GSM8K, and RuleTaker, using a 512-token reasoning budget. Word-problem accuracy was 93.3%, compared with 72.0% for Program-of-Thought and 58.7% for a tool-calling agent using the same solvers. Formatted queries bypassed the model and were answered in 1-11 milliseconds. In a 30-token configuration, the router ran 8.8 times faster and used 2.8 times less energy than Program-of-Thought.