Faithfulness Without Anchors: Why Feature-Importance Faithfulness Metrics Misalign with Models

Summary

Explainable AI methods are often promoted as a way to make machine-learning models understandable and increase trust, but the paper argues that accumulated evidence points to a different outcome: they can encourage overreliance on incorrect predictions and contribute to algorithmic aversion. The authors suggest that XAI is currently more useful for debugging and obtaining anecdotal insight into learned features than for reliably supporting trust. They focus on faithfulness, defined as the degree to which an explanation approximates the model itself. In principle, rigorous faithfulness measurement could quantify one aspect of explanation trustworthiness. In practice, existing metrics have major limitations that can make such measurement infeasible or computationally expensive. The position paper identifies five major shortcomings of current metrics and outlines four research directions intended to improve their reliability.