Estimating Ambiguity-Induced Uncertainty in LLMs Without Generating Answers

Summary

Reliable LLM deployment requires distinguishing uncertainty caused by ambiguous inputs from uncertainty caused by gaps in model knowledge. Existing methods generate several clarifications, ask the model to answer each one, and compare the answers; the authors argue that this response-based process is redundant, costly, and vulnerable to epistemic leakage. They introduce a clarification-only method that estimates ambiguity-induced aleatoric uncertainty directly from the space of plausible interpretations. In an ambiguity-detection evaluation across three benchmarks, the approach raises AUROC from 60.85 to 63.34. It also reduces output-token use by 4-26 times and API calls by 2.2-3.5 times. The resulting estimates have substantially lower correlation with epistemic uncertainty, supporting the intended separation between the two uncertainty sources. The paper’s theoretical analysis and experiments therefore suggest that ambiguity is better measured from interpretations than from the answers produced after clarification.