Continual Discovery of Enterprise World Models Under Changing Rules

Summary

In enterprise systems, changing one field can trigger updates, record creation, or approvals through organization-specific business rules. The paper studies how an agent can discover these hidden rules by interacting with records and observing outcomes, then maintain a world model that is revised as the rules change. To evaluate the problem, the authors introduce EnterpriseWorldShift, a live ServiceNow environment containing nine tables, 25 hidden rules, and 600 evaluation actions. The benchmark uses four versions of the same enterprise world: the tables and records remain fixed while a rule is modified, added, or removed, testing discovery, revision, extension, and retirement. Their Continual Discovery Agent (CDA) carries its model across these successive worlds. It predicts rule effects more accurately than answering each question by looking up the running system, the approach used by prior work, with an improvement of up to 8.98 IoU points. CDA can also answer from its maintained model without querying the live system.