Collaborative Theory of Mind Trains LLMs to Self-Revise by Predicting Users

Summary

Large language models often perform well on individual tasks but struggle to collaborate over multiple turns, such as by acting before clarifying a user's needs or failing to revise mistaken assumptions after feedback. The paper introduces Collaborative Theory of Mind (CToM), which has an assistant draft a response, predict how the user will react, and use that prediction to revise the response before presenting it. During training, the user's reply to each draft is used as the target for next-turn prediction, and this objective is combined with group-relative policy optimization (GRPO) to optimize both task performance and interaction efficiency. Across mathematical reasoning, coding, and writing tasks, CToM outperforms all reported baselines. On a collaborative mathematical reasoning task, it raises final accuracy from 74.1% with naive GRPO to 79.6%. Ablation results show that prediction supervision also reduces the number of user replies needed under the same reward, by 8% for coding and 17% for writing, while improving task performance. Behavioral analyses find that user-response predictions become more accurate after training, and that greater prediction accuracy is associated with larger gains from self-revision in collaborative performance.