StatFormBench Tests LLMs on Statistical Problem Formulation

Summary

Researchers introduce StatFormBench to test whether large language models can infer statistical tasks and relevant variables from informal user goals and heterogeneous data. Across 1,013 samples and 14 models, the best zero-shot results reached 72.0% fine-grained classification accuracy and 63.2% variable-set overlap. Prompting improvements were limited and inconsistent.