GAVEL Uses LLM Judging to Compare Clinical Timelines with Case Reports

Summary

Clinical timeline extraction from case reports is difficult to evaluate because expert reference timelines may contain imperfect annotations and event alignment can be imprecise. The researchers introduce GAVEL, an LLM judge protocol that compares two timelines against the original case report and produces a discrepancy type, a verdict, and the relevant report passage for every difference. They evaluated its event matcher, reviewed 2,738 findings from GPT5.6sol and DeepSeek V3.2, ranked six LLM-based extractors alongside two human annotators, and tested GAVEL-guided timeline merging. The matcher’s true match rates were 60% immediately below and 48% immediately above a 0.10 cutoff. Manual review confirmed 89.4% and 88.6% of the findings in the reported groups. Across 126 reports, readers preferred the merged timelines in 77.0% of comparisons, with a 95% confidence interval of 69.8% to 84.1%. Merging also reduced discrepancies attributed to the evaluated timeline from 7.63 to 0.85 per report. GAVEL is designed to compare and revise timelines against the source report without treating either timeline as ground truth.