LIFE FORGE: An Autonomous Flight Simulator for AI Agents

Summary

LIFE FORGE is an open-source evaluation and red-teaming system for AI agents that can use enterprise tools such as databases, vendor APIs, email, purchase orders, and fund transfers. It frames agent testing as a flight simulator: instead of relying only on static benchmarks or fixed prompt lists, its co-evolutionary engine generates adversarial and environmental edge cases, including indirect prompt injections, spoofed executive messages, price volatility, supplier shortages, and budget pressure. The system uses a three-dimensional MAP-Elites archive to search across adversarial intensity, environmental volatility, and budget pressure while preserving diverse failure scenarios. Its deterministic in-memory digital twin supports snapshots and causal rollback, and a policy oracle flags unauthorized tool use, financial drains, recursive loops, and unmet inventory goals. In the repository’s 30-generation comparison using identical seeds, both Alibaba Qwen 2.5-14B and Meta Llama 3.1-8B had a 100% adversarial failure rate across 31 simulations, but their failures differed. Qwen accepted spoofed authority and injected vendor instructions, triggering seven prohibited fund transfers totaling $60,000 to unauthorized offshore accounts; Llama resisted those exfiltration attempts but entered 12 unbounded retry loops after prices changed between lookup and order submission. LIFE FORGE can test local or cloud models through a LiteLLM adapter, connect to MCP-compatible clients such as Claude Desktop, Cursor, and LangGraph, and export audit reports with minimal-fix recommendations. The project also includes MODES metrics for studying activity, complexity, novelty, and ecology in cellular-automata systems, plus a local dashboard, command-line tools, and CI examples for recurring agent stress tests.