Asclepius: An Adaptive Harness for Long-Horizon Clinical Agents

Summary

Most language-model agents are evaluated on short, single-task trajectories, but real clinical deployments may operate for hours while managing competing demands. This study uses the Clinical Environment Simulator to test an agent through an entire emergency-department shift under continuous time and resource pressure. Agents usually reach the correct diagnosis, yet often fail to complete critical actions on time, exposing an execution gap. The authors measure three coupled failure modes: drift from instructions, incomplete treatment, and a severity-equity gap in timeliness. They introduce Asclepius, which rewrites its operating manual between shifts using trace-level feedback, provides an external clinical-skills library for high-stakes regimen knowledge, and divides per-turn queue decisions among three isolated subagents. On held-out batches not used while evolving the harness, Asclepius raises critical-action correctness by 22% over a strong baseline framework (p = 0.024) while preserving diagnostic accuracy; five LLM judges from three model families show consistent gains. Across all ten batches, improvements reach 25% for critical actions and 13% for timeliness. The results indicate that reducing the execution gap requires all three components to work together, rather than addressing any single failure mode alone.