Confidence-Gated LLM Routing Cuts Emotion Recognition Costs

Summary

This paper studies how contact-center conversational AI systems can use large language models for emotion recognition without sending every utterance through an expensive, higher-latency pipeline. It compares a low-cost stacked ensemble built from sentence embeddings, windowed dialogue context, and RandomForest, XGBoost, and logistic-regression stacking; prompted GPT-4o-mini configurations; and a confidence-gated hybrid that escalates only the ensemble's least-confident predictions to an LLM. On IEMOCAP, the ensemble outperformed every tested LLM setup, reaching 0.595 weighted F1 versus 0.460-0.536, with statistical significance at p < 0.0001, sub-10-millisecond latency, and much lower cost. The ranking reversed on MELD and CMU-MOSI, showing that neither a pure ensemble nor a pure LLM pipeline is reliable as a universal default. The hybrid Pareto-dominated both pure systems across all three datasets, reporting weighted F1 scores of 0.620, 0.643, and 0.824. Because most utterances remain on the near-zero-cost ensemble path, the estimated cost is about $10-85 per million utterances, compared with $99-170 for an LLM-only pipeline. The escalated turns were disproportionately associated with emotion or sentiment shifts, making the routing decision an interpretable signal for operators rather than an opaque cost-accuracy control. The ensemble was also reported to be well calibrated and safely under-confident. The pattern held across three datasets and two LLM providers. The authors present confidence-gated cascading as a deployment recipe for conversational-AI and CCaaS platforms deciding when an LLM call is justified, while the paper's contribution is the transfer of an established general machine-learning pattern to dialogue-contextual emotion recognition.