Closed-World Resolution Against Tool Hallucination in LLM Agents

Summary

Tool-augmented LLM agents can issue calls to tools that do not exist or provide arguments that no declared schema accepts, a failure mode that ordinary tool-selection and security gates do not address. The paper frames this as a structural blind spot: a gate can constrain a real tool call, but it cannot reject a call that was never a valid registry decision. It introduces a five-class taxonomy of tool hallucination and uses a training-free closed-world resolver, called the Resolution Rung, as a reference point. The resolver checks registry membership and the tool signature, and the authors argue that this defense must run before any causal gate. They identify one irreducible residue: borrowed arguments that are indistinguishable from a valid call under the available schema. Across ten hosted models and two invocation surfaces, the study records 322 genuine hallucinations. Fabricated-tool calls are much more common on an unconstrained raw-JSON surface than on the other surface, with counts of 34 and 3 respectively, while a 675B model performs similarly to a 7-8B model. The paper then examines the Model Context Protocol, where merging multiple servers into one namespace creates collision and shadowing cases that a single registry cannot represent. Its second M1-M5 taxonomy accompanies 154 hallucinations measured on a live MCP surface, including failures from frontier models that were clean on the single-registry surface. The authors release the versioned Hallucinated-Tools Benchmark so future resolvers can be compared on a common evaluation.