Building a reliable timeline of what happened to a patient during a hospital stay is one of the harder unsolved problems in clinical NLP. A discharge summary describes symptoms, diagnoses, treatments, and outcomes, but it is written retrospectively and rarely in chronological order. Structured electronic health records provide explicit timestamps for labs, medications, and procedures, but they cover only part of the clinical course and their timestamps can mean different things: order time, collection time, administration time, or documentation time. The result is that downstream tasks like treatment-window analysis, temporal phenotyping, and forecasting models all depend on timelines that may be incomplete, internally inconsistent, or subtly wrong.
This paper from researchers at the National Library of Medicine (NIH) and Princeton University attacks both the reconstruction problem and the evaluation problem at once. Rather than treating each event occurrence as a free-text description passed between pipeline stages, they assign every clinical event a persistent unique identifier tied to its exact span in the discharge summary. That identifier survives text-only time estimation, structured evidence retrieval, timestamped source-row grounding, and joint revision. They also introduce GAVEL, an LLM-based judge that compares two complete UID-aligned timelines against both the narrative and structured record, issuing typed, evidence-linked verdicts rather than a single preference score.
The back-reference failure problem
Clinical notes contain repeated mentions of the same type of event. A patient may receive an initial CT scan and a follow-up CT scan. A medication may be ordered, administered, held, and restarted. Weakness may recur across shifts. If a reconstruction pipeline carries only free-text event descriptions between stages, identical or near-identical mentions get merged, omitted, or cross-linked to the wrong evidence. The final timestamp may look clinically plausible but be assigned to the wrong occurrence. Existing evaluation metrics compound the problem: they align predicted events to a single clinician-authored reference and report aggregate agreement, which cannot distinguish a model error from an annotation error or two defensible interpretations.
The closest prior work, TKW2 (Text Knows What, Tables Know When), built a scaffold of central narrative events, calibrated it with retrieved structured rows, then attached non-central events in a second pass. That approach demonstrated the value of structured evidence as partial temporal signal, but it propagated events as mutable text strings without a persistent identifier. The present framework retains the same text-primary multimodal philosophy but changes the unit carried through the pipeline from a description to a source-grounded occurrence.
How the UID-preserving pipeline works
The reconstruction pipeline has five stages. First, an instruction-following language model tags every clinical event occurrence in the discharge summary, inserting UID markers around each mention while preserving the surrounding note. Each occurrence gets a case-local UID linked to its character span. Distinct occurrences of the same type, such as two separate CT scans, receive distinct UIDs even when their surface forms are identical.
Second, the UID-tagged note and mention inventory go to a temporal-reasoning model that estimates a point time for each event relative to admission (t=0), plausible temporal bounds, a binary flag indicating whether the timing is explicitly stated in the text, and up to five contextual UIDs used for temporal reasoning. This text-only estimate is both the unimodal baseline and the starting point for multimodal revision.
Third, for each UID the model generates up to three contextualized queries describing structured observations that could help locate that occurrence in time. These queries use the event mention and its narrative context rather than the mention string alone, so a generic phrase like "reversal agent was administered" can retrieve a named medication administration from the structured record. The queries feed into a dense retrieval step using Qwen3-Embedding-8B, followed by reranking with Qwen3-Reranker-8B. Retained summaries are expanded to their original event names, values, and timestamps, with the UID and query that retrieved each candidate preserved.
Fourth, a joint revision pass receives the UID-tagged note, the complete text-only timeline, the UID-linked timestamped evidence, and admission and discharge times. It revises the full inventory in one pass, allowing the placement of one occurrence to be considered alongside presentation, testing, treatment, transfers, and outcomes. A structured timestamp is used only when it refers to the same clinical occurrence and the appropriate timestamp type; otherwise the text-derived estimate is retained. Every valid output must contain each original UID and mention exactly once, and the model may not add, remove, merge, rename, or duplicate occurrences.
The critical design choice is that structured timestamps are treated as evidence rather than as event onset by default. An order time is not assumed to represent administration. A result time is not assumed to represent specimen collection. The model must interpret the timestamp type in clinical context before applying it.
GAVEL: adjudicating timelines against the source record
Reference-based metrics quantify agreement with one clinician-authored timeline but cannot determine which account is supported when two timelines differ. GAVEL (Grounded Adjudication of Variations across Extracted timeLines) addresses this by comparing two UID-aligned candidate timelines against the discharge summary and up to 900 structured event-series summaries from the same encounter.
A single frozen prompt performs four dependent operations. It anchors each candidate event to narrative and structured support, or marks it as unsupported. It matches events that describe the same clinical occurrence across the two timelines, allowing differences in wording and granularity. It compares values and times for matched events, treating point times differing by less than the maximum of three hours or 10% of the distance from admission as equivalent. It classifies each discrepancy into one of six types: VALUE, TIMING, A_ONLY, B_ONLY, SHARED_UNSUPPORTED, or DUPLICATE, and assigns one of five verdicts: A, B, BOTH, NEITHER, or UNCLEAR.
Each finding contains candidate rows, narrative and structured evidence, the verdict, and a concise reason. The judge treats both timelines symmetrically and does not assume either candidate is correct. Absence from the summarized structured record is not treated as evidence that a note-supported event did not occur, because many clinical events documented in the narrative have no structured counterpart.
Experiments: six models, 40 summaries, one clinician
The evaluation uses 40 mixed-critical-care discharge summaries: 15 from the i2b2 corpus and 25 from MIMIC-IV, each linked to structured EHR data from the same encounter. A single clinician constructed one reference timeline per case. For 20 cases the clinician worked without model suggestions; for the remaining 20 MIMIC-IV cases the clinician could inspect LLM-suggested structured temporal anchors but retained responsibility for event selection, evidence interpretation, and final timestamp assignment.
Six open-weight models served as reconstruction backbones: GLM 5.2 FP8, DeepSeek V3.2, Qwen3.5-397B, GPT-OSS-120B, Mistral 3.5 Medium, and MiniMax M2.7. Within a run, the same backbone handled event tagging, text-only temporal inference, anchor-query generation, and joint revision. Two prior methods served as baselines: single-step LLM-Timeline (instantiated with GLM 5.2) and TKW2 (instantiated with DeepSeek V3.2). Events were aligned using recursive best matching with PubMedBERT cosine distance at a threshold of 0.1, and temporal accuracy was measured with three metrics: event match rate, temporal concordance (c-index), and AULTC (Area Under the Log-Time CDF).
Results: GLM 5.2 multimodal is the clear winner, but the effect is model-dependent
GLM 5.2 showed the cleanest improvement from multimodality. Its event match rate stayed at 0.790, identical to the text-only variant, while concordance rose from 0.781 to 0.802 and AULTC rose from 0.758 to 0.773. The confidence intervals for the temporal metrics did not overlap between the unimodal and multimodal variants.
The other backbones showed smaller gains or trade-offs. DeepSeek V3.2 multimodal actually lost event match rate (0.617 vs. 0.643 unimodal) while gaining modestly on concordance. Qwen3.5-397B gained AULTC but lost both event match rate and concordance. Mistral 3.5 Medium gained AULTC but lost event match rate. GPT-OSS-120B and MiniMax M2.7 showed no consistent multimodal advantage. The threshold-sweep analysis confirmed that the modality effects are model-dependent: no configuration uniformly dominated both event recovery and temporal quality across all thresholds from 0.01 to 0.50.
Against prior methods, the UID-preserving framework substantially improved event recovery. LLM-Timeline (GLM 5.2) achieved 0.552 event match rate; TKW2 (DeepSeek V3.2) achieved 0.482. The proposed GLM 5.2 multimodal at 0.790 represents a 43% increase in event recovery over LLM-Timeline and a 64% increase over TKW2 on the same cohort and threshold.
What the ablations reveal
Four ablation variants for GLM 5.2 isolate the contribution of each pipeline component.
- Removing UIDs from joint revision while retaining the event inventory, initial estimates, retrieved evidence, and backbone reduced event match rate from 0.790 to 0.693. The higher temporal scores in that condition (concordance 0.794, AULTC 0.792) were computed on a smaller matched subset and do not indicate better overall reconstruction. This confirms that UIDs primarily preserve event retention, preventing repeated or similarly worded occurrences from being merged or dropped during revision.
- Removing source-row linkage left event match rate unchanged at 0.790 but returned concordance to 0.781 and AULTC to 0.755, essentially reverting to text-only temporal performance. The timestamps attached to the retrieved source rows, rather than the summary text alone, provided the temporal benefit.
- Removing query generation (using the source mention as the retrieval query) or removing the reranker caused no consistent degradation for GLM 5.2 at the selected operating point.
Extended ablations across DeepSeek V3.2 and Qwen3.5-397B showed that the effects are backbone-dependent. For DeepSeek V3.2, removing the reranker or query generation produced higher temporal point estimates but substantially lower match rates (0.465 and 0.393 vs. 0.617 full multimodal). No component removal was uniformly harmful across all three backbones, and no variant dominated all three metrics.
GAVEL validation and five-source comparison
Manual validation sampled 150 stratified findings from 10 cases across three pairings among the clinician-authored, GLM 5.2 multimodal, and DeepSeek V3.2 multimodal timelines. A primary reviewer upheld 122 of 150 findings (81.3%; 95% CI, 74.3 to 86.8). After standardizing to the corpus distribution, the confirmation rate was 78.2% (95% CI, 70.3 to 85.8). A second reviewer independently evaluated 50 shared findings and agreed on 45 (90.0%; Cohen's kappa 0.494, PABAK 0.800). All five inter-reviewer disagreements concerned whether GAVEL should have emitted or resolved a finding, not which candidate was better supported.
In the full five-source comparison (clinician-authored, GLM 5.2 multimodal, GLM 5.2 unimodal, DeepSeek V3.2 multimodal, DeepSeek V3.2 unimodal), 374 pairwise games produced 13,321 findings, of which 10,924 (82.0%) were decisive. Under prespecified severity weights, the clinician-authored and GLM 5.2 multimodal timelines had the two highest Bradley-Terry point estimates. Their marginal intervals overlap.
The controlled within-backbone comparisons are more informative. GLM 5.2 multimodal earned 79.6% of weighted points against its text-only counterpart (95% CI, 66.5 to 90.0), separating clearly from parity. DeepSeek V3.2 multimodal earned 56.5% against its text-only version (95% CI, 39.1 to 74.4), which did not separate from 50%. This mirrors the reconstruction results: multimodal evidence helps GLM 5.2 but not DeepSeek V3.2.
The error profile shows that wrong timing accounted for 60.2% of decisive findings and missed positive events for 27.1%. Together these comprised 87.2% of all decisive errors. Unsupported additions (events present in one timeline but absent from the record) accounted for only 1.2%. Both multimodal model sources had fewer errors per matchup than their unimodal counterparts. Interestingly, the clinician-authored source had the lowest wrong-time rate but the highest missed-positive and wrong-value rates, illustrating why the clinician reference is useful without being treated as infallible.
Limitations and trade-offs
The evaluation covers 40 critical-care discharge summaries from a single clinician, limiting generalization across institutions and measurement of reference-annotation variability. The reconstruction is text-primary, so events recorded only in structured data may be absent from the candidate timeline. Structured timestamps do not always represent event onset, and the pipeline must interpret timestamp types in context. GAVEL validation estimates the correctness of findings it reports, not discrepancy recall, because manual review did not independently search for disagreements the judge failed to flag.
The model-dependent nature of the multimodal benefit is a practical concern. GLM 5.2 clearly gains from structured evidence integration, but DeepSeek V3.2 does not. Whether a given backbone will benefit cannot be predicted from its general capability level. The ablation results also show that removing any single component is not uniformly harmful, suggesting the pipeline has some redundancy but also that the marginal value of each component depends on the backbone's ability to use the information it provides.
From a deployment perspective, the framework is explicitly intended for research data curation and clinician-reviewed workflows, not autonomous care. The occurrence-level audit trail, which links each final timeline row to its narrative span, generated retrieval queries, and timestamped structured evidence, allows a reviewer to inspect why a time changed. GAVEL complements this by directing review to specific timing, value, omission, or duplication disagreements rather than requiring the reviewer to evaluate the entire timeline from scratch.
What this means for developers building clinical NLP systems
The core lesson is that event identity matters as much as event extraction. When building pipelines that pass information between stages, whether for timeline reconstruction, relation extraction, or any other structured output task, persistent identifiers prevent the back-reference failures that free-text propagation introduces. The UID is cheap to implement and has a measurable effect on event retention.
The second lesson is that structured EHR data is selective temporal evidence, not a universal timestamp source. Retrieving the right rows and interpreting their timestamp types in clinical context is more valuable than simply feeding all available structured data to the model. The query-conditioned retrieval approach, where the model generates context-aware queries for each event, outperforms direct mention-to-row retrieval.
For evaluation, the GAVEL framework points toward a more useful paradigm than single-reference agreement scoring. When two timelines disagree, the question should be which account the source record supports, not which is closer to one selected reference. The typed, evidence-linked finding format gives reviewers and developers actionable information about where and why timelines diverge.
The 40-summary evaluation is small, and the single-clinician reference limits what we can say about upper-bound performance. But the ablation methodology is sound and the component contributions are clearly separated. For teams working on clinical timeline reconstruction, multimodal EHR alignment, or LLM-based evaluation of clinical NLP outputs, this paper provides both a working framework and a careful decomposition of what each component actually contributes.