DerivAudit Audits Whether Long-Term Agent Memories Are Supported by Evidence

Summary

Long-running LLM agents compress earlier interactions into persistent memories that later tasks may use as premises. The paper identifies a derivation problem: evidence can be distributed across prior interactions, while compression can also combine supported facts into a stronger claim that the history never established. It formalizes three requirements for valid memory: evidence scope, compositional validity, and admission reliability. The authors introduce DerivAudit, which asks whether support exists beyond writer-provided citations, whether the composed memory adds unsupported meaning, and how the write-time admission decision affects later use. Across two natural memory corpora, expanding the audit to the broader pre-write history recovered support for nearly 60% of memories that appeared unsupported from their citations alone. However, 17-21% remained unsupported after expansion. The experiments also show that broader evidence does not automatically improve admission reliability: unsupported memories were still frequently admitted across verification models, and evidence expansion made reliability worse on two backbones. The results frame long-term memory as a derivation and admission problem, rather than simply a citation-retrieval problem.