World Editing: Testing AI Agents on Executable Game Worlds

Summary

Interactive world models can generate environments and act within them, but deliberately modifying an existing executable world has received less study. The paper defines world editing as intervening in an existing world while preserving properties that should remain unchanged, and introduces intervention depth to describe how strongly an edit couples entities, dynamics, and systems. The authors implement the setting through industry-grade game modding and create IGMWorld with IGMBench, a benchmark containing 110 tasks and more than 1,100 executable state and behavioral criteria across Minecraft and Terraria. Tasks cover property, entity, dynamics, and system interventions, with deterministic executability, behavioral, preservation, and visual checks. The strongest evaluated configuration solves 78.2% of tasks under a strict task-level measure, while criterion-level performance reaches 94.8%. Reliability generally falls as intervention depth increases, and the trend remains when tasks have similar numbers of evaluation criteria. Most failed edits still build and load, indicating that the harder problem is making the modified world behave as requested. Visual consistency is a separate weakness: every evaluated configuration remains below a 50% joint visual pass rate. The results position world editing as distinct from world generation and interaction, while presenting executable games as a practical testbed for studying the capability.