You Didn’t Deploy the Agent You Evaluated

Summary

Anuclei argues that evaluating an AI agent does not establish that the same behavior will run in production. Changes to prompts, tool schemas, policies, available knowledge, memory, retrieval results, credentials, model fallbacks, or orchestration can alter the effective agent while its human-facing name remains unchanged. Because agents can retrieve sensitive information, call tools, modify systems, spend money, or trigger further decisions, production governance must answer what was evaluated, what was certified and authorized, what context and capabilities were present, and what actually executed. The proposed foundational primitive is a behavioral revision that follows the agent through declaration, evaluation, certification, authorization, execution, observation, and audit. Certification should bind to an identified behavior, requirements, and evidence, while runtime authorization should verify that the certified revision is permitted to perform a specific action in its current context. The article also treats runtime context as part of the program: provenance should show what classes of information influenced a decision, where they came from, and whether they were allowed. Traces and telemetry remain useful, but execution evidence must connect identity, certification, policy, granted capabilities, context, side effects, and authorization timing without relying on the agent’s own explanation. With sufficient provenance, production interactions can become reproducible evaluation cases for counterfactual testing, such as asking how a new revision or policy would have behaved without repeating dangerous side effects. Anuclei calls the resulting category an “Agent System of Record,” intended to connect identity, evidence, certification, authority, execution, and accountability around existing models and tools. Its central principle is that trust should come from independently recorded evidence and enforced controls, allowing enterprises to move from autonomous agents toward accountable agents.