LLM Agents Help Refine Cognitive Algorithms from Human Behavior

Summary

The paper presents a hybrid approach for discovering cognitive algorithms from behavioral data, combining human expertise with the flexibility of large language models. Human-created cognitive models are represented as probabilistic programs and supplied to a group of LLM agents. The agents identify mismatches between a model and observed behavior, propose code-level revisions under researcher-defined constraints, and check whether the revised structure remains faithful to the intended model. A probabilistic inference module then estimates latent variables and computes data likelihoods for each revision. The authors evaluate the pipeline on human behavior from a problem-solving task designed to reveal different cognitive algorithms. Across the evaluation, revised models consistently fit the behavioral data better than their original versions. The revisions also expose a small group of recurring innovations that account for meaningful variation in how people behave on the task. The work frames cognitive-model discovery as program refinement, retaining human guidance while using LLM agents to explore scalable algorithmic changes.