Risk-Constrained Stopping Improves Safety Audits for Clinical Diagnosis Agents

Summary

Clinical diagnosis agents must choose both which test to request next and whether to diagnose or defer. The paper introduces Cros, a risk-constrained stopping layer that ranks state-wise error risk, designs policies on disjoint development splits, and applies LTT-style exact tests to selective diagnostic error and minimum autonomous coverage. Its finite-sample guarantee requires the candidate family, testing rule, and randomization to be fixed before calibration labels are accessed. On a 1,834-episode MIMIC-derived abdominal-pain benchmark, the full ranker reached an exploratory state-error AUROC of 0.853, compared with 0.715 for maximum class probability and 0.552 for the backbone's native stop score. On a previously viewed 367-episode evaluation split, analytically averaging over frozen Cros weights produced 16.9% selective error at 78.8% coverage, with cost 5.57 and 0.68 tests, versus 30.8% error at 100% coverage, cost 8.14, and 1.53 tests under native stopping. Continuing interaction was not uniformly safer: error was 28.3% with HPI alone and 34.3% after a full workup. A uniform-weight mixture was cheaper on the viewed split but did not meet the locked development margins, and Cros nominally met the joint criterion in only 6 of 20 development resplits. Because evaluation labels had been inspected during earlier development, the authors characterize the findings as exploratory feasibility and audit evidence, not a confirmatory safety certificate.