How Guided AI Tutors Respond to Persistent Student Impasses

Summary

This study examines how a guided large language model tutor responds when chemistry students remain stuck, focusing on the assistance dilemma between revealing an answer too early and withholding help for too long. The researchers analyzed 20,462 student turns from 1,260 authentic tutoring sessions and identified 6,630 impasse turns, grouped into conceptual errors, expressed uncertainty, and explicit help-seeking. They used these cases to simulate baseline, no-direct-answer, and guided-tutor conditions. In a sample of 150 impasses, the baseline tutor gave the answer directly in 50.7% of responses, the no-direct-answer tutor asked a follow-up question every time, and the guided tutor varied its response according to context. Analysis of authentic interactions found that every additional impasse turn reduced the odds of recovery on the next turn by 12.7% (AOR 0.873, p < .001). Early dropouts often became trapped in recursive concept elicitation before reaching execution. The benefit of questioning declined as impasses deepened: the scripted-question-by-depth and follow-up-by-depth effects had AORs of 0.78 and 0.83. By contrast, addressing the student's error became more beneficial, with an AOR of 1.14. After a scripted question failed, repeating it led to recovery in 28.1% of cases, compared with 39.8% when the tutor addressed the underlying error. The authors argue that impasse depth and type can serve as turn-level signals for triggering graduated, state-sensitive assistance in real time learning analytics.