The Silent Failures of Agentic Data Science

Summary

Large language models have made it easier to build agents that automate data science from task formulation through model development, but the paper argues that current evaluations are too narrow. Existing tests emphasize predictive performance and code correctness, which can hide methodological problems such as data leakage and inappropriate validation. The authors examine failure modes across 220 data science tasks, introduce a reusable test suite covering six specific failures, and qualitatively compare agent pipelines with expert solutions in Kaggle competitions. They find that most agent-generated solutions contain at least one failure, regardless of the complexity of the agent framework or the model's capacity. Data leakage is positively correlated with predictive performance, meaning that apparently stronger results can sometimes signal a flawed methodology rather than a more capable system. The qualitative analysis also finds that data science agents generally show less problem-solving creativity and domain knowledge than competition-winning data scientists. The paper concludes that principled failure-mode analysis should become a standard part of evaluating trustworthy agentic data science systems.