InvestigationWorlds Tests AI Agents on Legal Evidence Reasoning

Summary

InvestigationWorlds is an agentic environment for evaluating legal investigation. It builds each environment from a real U.S. federal court case obtained through PACER and uses an attorney-validated generation pipeline to create role-tagged documents around the original record. The setting is based on summary judgment motions, whose evidence record supports a court-adopted hypothesis used to decide the motion. Each case contains multiple coherent factual interpretations, but only one matches that hypothesis. In an evaluation of 100 cases, agents often selected an incorrect hypothesis even when they retrieved relevant evidence, indicating difficulty in distinguishing the court-adopted account from plausible alternatives.