{"id":"ea381eef-856c-4435-a37f-50f1d3eae7a1","arxiv_id":"2506.04831","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A single LLM fine-tuned with a learned summary bottleneck forecasts next-hour EHR states and iteratively simulates multi-hour patient trajectories across ED, ward, and ICU on MIMIC-IV.","lead":"EHR2Path turns a patient's hospital record into a text story and trains a small language model to predict the next hour of that story, then loops the predictions to simulate days ahead. The paper asks whether one flexible model can handle the full messy mix of notes, vitals, labs, and medications better than narrower task-specific models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Claim of complete in-hospital trajectory simulation is not actually evaluated; all simulation tasks cap horizons at 24 hours or 3 days, so the central feasibility claim is unsupported.","rationale":"The Reader's verdict is CONDITIONAL, and my analysis supports keeping that condition, though for a different primary reason. The Reader's weakest assumption concerned hourly aggregation losing sub-hourly acute changes. That is a legitimate representation concern but it is tied to the model's chosen one-hour time step; the paper defines its task as hourly prediction, so the lossy aggregation is a design choice rather than a direct contradiction of the stated claim. The more load-bearing issue is the mismatch between the headline claim of simulating 'complete in-hospital trajectories' and the actual evaluation, which only spans horizons of 24 hours or 3 days. This mismatch is visible in Table 7 and Section 4.2: no task rolls out until hospital discharge without a time cap. The LOS indicator, intended to make rollouts converge, is only tested on short-horizon tasks. If the central claim were reworded to 'short-horizon pathway forecasting and simulation,' the evidence would be substantially more secure. The baseline comparisons and metric choices noted by the Reader are also worth tightening, but they do not undermine the core feasibility claim as directly as the missing full-trajectory evaluation does. I therefore do not recommend changing the CONDITIONAL verdict: the paper is a solid short-horizon forecasting contribution, but the 'complete trajectory' claim needs either a dedicated full-stay simulation experiment or a scoped-down statement in the abstract and introduction. This is not a rejection of the method, only a demand that the central claim match the evidence.","tokens_in":608,"tokens_out":8624,"duration_ms":320622,"concrete_test":"Run the released simulation code on a held-out cohort without any time cap: start at hospital admission, iteratively predict the next-hour state, and continue until the model emits a discharge/death token (with a safety limit of 30 days). Compare the resulting simulated stay durations, terminal dispositions, and per-hour event F1/value MAE against ground truth. If a substantial fraction of rollouts fail to terminate or diverge after the first 24 hours, the complete-trajectory claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim, stated in the abstract and introduction, is that EHR2Path enables 'iterative simulation of complete in-hospital trajectories.' However, the evaluation defined in Section 4.2 and detailed in Appendix Table 7 never actually simulates a full hospital stay. Every rolling simulation task has an output window capped at '24h or until [unit] discharge' (e.g., ED Vital Signs, Hospital Medications, Hospital Lab Values, ICU Vital Signs, ICU Inputs), or an outcome horizon of 'within 3 days' (ICU Imminent Discharge) or 'within 24h' (ICU Imminent Mortality). The one task that mentions hospital discharge (Hospital Discharge Diagnosis) is a direct, non-rolling prediction with the LOS indicator set to zero, not a multi-step simulation. The LOS indicator is designed to let rollouts terminate at a final state, but its ablation (Table 4) is only measured on ICU Imminent Discharge (a 3-day horizon) and a direct prediction task. Thus, no experiment demonstrates that the model can iteratively simulate a trajectory from admission to discharge/death without a pre-imposed time cap. The paper's differentiating contribution is precisely pathway-level simulation of complete trajectories, so this gap directly undermines the strongest claim. The reported results support short-horizon forecasting (up to 24 hours) and bounded outcome prediction, but not 'complete in-hospital trajectory simulation' as worded.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces EHR2Path, a multimodal LLM-based framework that serializes heterogeneous EHR data (ED, ward, ICU) into structured text, uses a Masked Summarization Bottleneck to compress long-term history into a small set of learned summary tokens, and is trained to predict the next hour's sparse EHR state. The authors propose three variants (text-only, summary-only, summary+text) and two fine-tuning strategies, and evaluate on nine tasks over MIMIC-IV, including next-time-step prediction, rolling simulation up to 24 hours or 3 days, and outcome prediction. The main reported results are next-hour event macro/micro F1 values around 0.47/0.78 for the text model and 0.48/0.76 for the summary model, vastly above a statistical baseline (0.02), with competitive or better results than the ETHOS baseline on shared tasks, and improvements after fine-tuning for outcome prediction.","tokens_in":16777,"tokens_out":3846,"duration_ms":44839,"significance":"If the claims are substantiated, EHR2Path is a valuable contribution: it demonstrates that a single LLM can ingest a broad set of routine EHR modalities (including free-text notes and dense ICU chart events) and produce reasonable next-hour forecasts, and the Masked Summarization Bottleneck is a practical technique for extending effective context length without proportional token growth (reportedly up to 20x more historical context at similar input size). The evaluation is patient-level split, uses held-out test sets, and reports confidence intervals; the code is released. However, the paper's headline claim of \"iterative simulation of complete in-hospital trajectories\" is not actually evaluated, and the lossy hourly aggregation may obscure clinically important sub-hourly dynamics. These issues are load-bearing for the clinical-simulation narrative, though the underlying forecasting and summarization machinery appear sound and the claims could be re-scoped or the evaluation extended.","major_comments":[{"comment":"The paper repeatedly claims that EHR2Path enables \"iterative simulation of complete in-hospital trajectories,\" but no experiment in Section 4.2 or Appendix Table 7 rolls out a simulation from admission to discharge or death. Every rolling simulation task is capped at \"24h or until unit discharge\" (ED Vital Signs, Hospital Medications, Hospital Lab Values, ICU Vital Signs, ICU Inputs), ICU Imminent Mortality is capped at 24h, and ICU Imminent Discharge at 3 days. Hospital Discharge Diagnosis is a direct, non-rolling prediction with the LOS indicator set to zero. The LOS indicator is designed to let rollouts terminate, but Table 4 only measures its effect on ICU Imminent Discharge (a 3-day horizon) and on a direct diagnosis task, not on a full-stay rollout. The central differentiating claim is therefore unsupported by the reported evidence.","section":"Abstract; Section 4.2; Appendix Table 7"},{"comment":"The simulation evaluation does not go beyond a 24-hour horizon in Figure 5, yet the paper claims \"extended time horizons\" and \"full in-hospital pathways,\" while hospital stays in MIMIC-IV often span hundreds of hours. Because iterative autoregressive rollout accumulates errors, the observed gradual decay over 24 hours does not establish stability over multi-day stays. The paper should either include full-stay rollouts (or at least multi-day rollouts with termination) or explicitly re-scope the claims to short-horizon simulation.","section":"Section 5.2; Figure 5"},{"comment":"The preprocessing aggregates all raw EHR events hourly (average for numerical values, most frequent for categoricals) before serialization into text. This step discards sub-hourly dynamics, including rapid vital-sign swings and acute deterioration that are clinically important in ICU settings. As a consequence, the predicted trajectories may be smooth but clinically misleading, and the paper's claim of preserving \"real-world noise and incompleteness\" is undercut. The authors should provide evidence that hourly aggregation preserves acute changes (e.g., by comparing against raw event-level evaluation for high-frequency chart events) or explicitly state this limitation in Section 5.5.","section":"Section 4.1"}],"minor_comments":[{"comment":"The phrase \"noise (+-20%)\" should be written as \"noise (±20%)\" or \"noise in the range ±20%\"; as written it is ambiguous whether noise is added multiplicatively or additively.","section":"Section 3.2.3"},{"comment":"The sentence \"E2P-S+T consistently ranks first or second\" is not fully supported by Table 2: for ICU Imminent Mortality, E2P-S+T (0.57) is below ETHOS (0.61) and E2P-T (0.53) is below, while E2P-S is 0.50; the consistency claim should be qualified with the confidence intervals or the specific tasks where it holds.","section":"Section 5.2 / Table 2"},{"comment":"Table 5 selects bottleneck size 8 based only on validation loss after a limited number of training steps; no downstream task metric (e.g., next-step F1 or simulation accuracy) is reported across sizes, so it is unclear whether the chosen size is optimal for the actual prediction tasks.","section":"Section 5.4 / Table 5"},{"comment":"The phrase \"strong baselines\" in the abstract overstates the comparison: the statistical baseline is intentionally trivial, and ETHOS is not adapted to the full feature space or the exact output format of EHR2Path; the restricted-data comparison gives a more honest picture, though ETHOS's lower F1 there (0.04 vs 0.12) still favors the proposed model.","section":"Section 5.1"}],"recommendation":"major_revision","confidential_remarks":"The core forecasting and summarization contributions are sound, and the experimental setup (patient-level split, held-out test, confidence intervals) is careful. My main concern is that the paper's stated contribution—full in-hospital trajectory simulation—is not matched by the evaluation, which is capped at 24 hours or 3 days. This is fixable either by adding a full-stay rollout experiment (which may be expensive but is feasible with the LOS indicator) or by toning down the abstract and introduction to describe short-horizon forecasting and bounded simulation. I lean major_revision rather than reject because the gap is in the framing and the evaluation scope, not in the internal correctness of the proposed method. I would also encourage the authors to address the hourly-aggregation limitation explicitly, as it affects the clinical interpretation of the simulated trajectories."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth your time for one reason: the Masked Summarization Bottleneck. The idea of compressing long patient histories into a small set of learned summary tokens with an attention mask that forces future-state prediction to attend only to those tokens is a clean, sensible way to handle long-context EHR data, and the ablations show it improves token efficiency while retaining forecasting performance. The breadth of data integration—notes, labs, vitals, meds, chart events, demographics—is genuinely wider than what I've seen in prior trajectory models like ETHOS or Foresight.\n\nThat said, the central claim overreaches. The abstract promises \"iterative simulation of complete in-hospital trajectories,\" but every rolling simulation task in Table 7 is capped at 24 hours or three days. No experiment rolls the model out to actual discharge or death. The LOS indicator ablation is measured on a three-day ICU horizon and a direct prediction task, so we don't know if the model really converges to a final state over a full stay. That's a load-bearing gap because \"full pathway\" is the paper's main selling point. The authors should either run a full-stay simulation (even on a sample of shorter stays) or revise the claim to \"short-horizon forecasting and bounded simulation.\"\n\nThe baseline story is softer than it looks. The statistical baseline is trivial, and ETHOS, the only trajectory baseline, underperforms because it can't predict many event types. No comparison to more competitive timeline models like Foresight or MOTOR is made—those may not have public code, but the paper should at least discuss the gap. The metrics are reasonable, though the \"modified MAE\" with maximum error imputation deserves a clearer explanation.\n\nThe hourly aggregation into text is a real limitation—sub-hourly swings in vitals or labs are lost, which could make trajectories look plausible but miss acute deterioration. The authors acknowledge the trade-off, but it deserves a sensitivity analysis.\n\nOverall: this is a serious, competent empirical paper with a genuine architectural contribution. The short-horizon forecasting results are credible, and the summarization mechanism is worth borrowing. The major issue is that the paper's strongest claim isn't tested. I'd send it to reviewers, but the authors need to either add a full-stay simulation or reframe the contribution.\n\nFor you: bring it to reading group for the bottleneck idea, and cite it if you work on long-context EHR modeling.","headline":"A useful architectural idea—the Masked Summarization Bottleneck—wrapped in an abstract that overclaims full-stay simulation when every task is capped at 24 hours or 3 days.","tokens_in":17284,"tokens_out":2225,"would_cite":true,"duration_ms":26271,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","92C50"],"pacs":[],"model":"deepseek-v4-flash","headline":"A single LLM predicts next-hour hospital states across ED, ward, and ICU, and rolls them into full simulated patient pathways.","keywords":["electronic health records","patient pathway simulation","trajectory forecasting","large language models","masked summarization bottleneck","MIMIC-IV","next-hour state prediction","iterative simulation"],"falsifier":"On a cohort of patients who experience rapid deterioration (e.g., an unplanned ICU transfer within six hours of ED vitals), compare the model's next-hour vital-sign forecasts against raw chart data at five-minute resolution: if the model's error on the hour preceding deterioration is no worse than on stable hours, the hourly bottleneck is safe; if it systematically misses the deterioration by predicting near-baseline values, the representation is lossy.","tokens_in":16319,"feed_emoji":"🏥","tokens_out":6350,"duration_ms":56793,"temperature":0.7,"pith_summary":"EHR2Path tries to establish that a single language-model-based system can forecast and simulate a patient's entire in-hospital course — emergency department, ward, and ICU — from the full breadth of routine electronic health record data, including free-text notes, vital signs, lab values, medications, and dense bedside charting. The point is to move beyond isolated outcome classifiers toward a pathway-level model that predicts the next hour's state and can be rolled forward to simulate multi-hour or multi-day trajectories. The paper reports that this works on MIMIC-IV, with the summary-based variant matching or beating text-only models while using far fewer tokens, and with fine-tuned variants outperforming strong outcome-prediction baselines on several tasks. A sympathetic reader would take the central claim to be that routine, noisy, heterogeneous EHR data are sufficient for pathway-level forecasting if the history is compressed well.","feed_headline":"One LLM predicts next-hour hospital states across ED, ward, and ICU","feed_subtitle":"A summarization bottleneck gives the model 20x more history at a fraction of the token cost.","key_machinery":"The central mechanism is the Masked Summarization Bottleneck, a custom attention mask appended to an LLM. For each section of the patient record, a sequence of input tokens is followed by m summary tokens and then output tokens; the mask lets output tokens attend only to the summary tokens, forcing the model to write the information needed for next-hour prediction into m compact embeddings during training. Because the summaries are optimized to predict the next state rather than reconstruct the input, they act as a task-optimized lossy compression of the full history. This is paired with a hierarchical text representation (clinical units such as ED, hospital, and ICU; categories such as vitals or medications; individual features), where identical consecutive values are merged into intervals and missing data simply absent, and with a Length-of-Stay countdown token that is dropped or noised during training and re-estimated at inference so simulations converge to discharge.","core_discovery":"On its own terms, the paper's discovery is that next-hour EHR state prediction over a very broad feature space — with outputs covering vitals, labs, medications, procedures, chart events, and care transitions such as admission, discharge, and death — is learnable by an LLM, and that iterating this prediction produces plausible full-pathway simulations. The Masked Summarization Bottleneck is what makes this feasible: it compresses each section of a patient's history into a handful of learned summary tokens, so the model can attend to an average of ~9,800 tokens of context using only ~220 input tokens in the summary-only variant, roughly 20 times more history than a 24-hour text window. In the reported experiments the combined summary-plus-text model is the most stable, ranking first or second across simulation tasks, and it outperforms the ETHOS baseline on the shared task of hospital lab-value development. Fine-tuned for outcomes, the model exceeds the specialized baselines MEME and REMed on three of four tasks, including ICU imminent mortality (0.83 accuracy vs. 0.71).","pith_inferences":["Because the bottleneck is trained to optimize next-hour prediction, the summary tokens are not guaranteed to preserve information relevant to other uses, such as long-horizon risk scoring; a model wanting both may need multiple summary sets or a multi-task training objective.","The hourly aggregation step averages numerical values and takes the most frequent categorical value. If clinically significant sub-hourly deterioration is washed out, the simulated trajectories could look smooth while missing acute events; a test on high-resolution bedside chart data would quantify this.","The LOS countdown is learned as a number, not derived from clinical triggers. In deployment, where the true remaining stay is unknown, the model must rely on its own predicted LOS, and errors could compound across rolls.","The same text-serialization and summarization recipe could be applied to outpatient or lifetime records, but the paper's evaluation is limited to a single hospital system, so cross-hospital transferability is untested."],"forward_implications":["A single model can replace a collection of narrow task-specific predictors, since next-hour forecasting over all features subsumes tasks like vital-sign, lab, medication, and input development.","Iterative rollout enables multi-day simulation of in-hospital trajectories, which could be used to generate what-if scenarios for treatment planning and discharge timing.","Up to 20x more historical context can be supplied without proportional token growth, reducing cost for long-stay patients.","Outcome-oriented fine-tuning turns the same pathway model into a competitive foundation model for tasks like imminent mortality and imminent ICU discharge.","Prediction quality degrades gradually rather than catastrophically over a 24-hour horizon, suggesting the simulated trajectories remain informative for clinical use."],"supporting_citations":[{"why":"MIMIC-IV dataset, source of all patient records used for training and evaluation.","marker":"(Johnson et al., 2023b)"},{"why":"ETHOS baseline, the prior trajectory-simulation model whose feature scope and token window EHR2Path extends.","marker":"(Renc et al., 2024)"},{"why":"MEME baseline, an LLM-based ED outcome classifier that EHR2Path fine-tuning is compared against.","marker":"(Lee et al., 2024)"},{"why":"REMed baseline, a long-context ICU outcome model used as comparison for outcome fine-tuning.","marker":"(Kim et al., 2024)"},{"why":"MIMIC-Extract pipeline, source of the hourly aggregation scheme.","marker":"(Wang et al., 2020)"},{"why":"Prior work whose preprocessing approach and evaluation tasks partly inspire the simulation tasks.","marker":"(McDermott et al., 2021)"},{"why":"Information bottleneck principle, theoretical basis for the summary compression objective.","marker":"(Tishby et al., 1999)"},{"why":"LoRA, the parameter-efficient fine-tuning method used for training.","marker":"(Hu et al., 2022)"},{"why":"Qwen2, the base LLM backbone.","marker":"(Yang et al., 2024)"}],"fun_headline_variants":["LLM predicts full hospital pathways from routine EHRs","Summarization bottleneck gives LLM 20x more patient history","EHR2Path simulates next-hour states across ED, ward, ICU","One model forecasts deterioration, recovery, and care transitions","Pathway-level EHR modeling beats specialized baselines"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that aggregating raw EHR events into hourly averages and most-frequent categorical values, then serializing them as text, preserves the acute clinical changes a pathway model needs to predict; if sub-hourly deterioration is averaged away, the forecasts will be smooth but misleading.","fun_headline_variants_meta":{"raw":{"variants":["LLM predicts full hospital pathways from routine EHRs","Summarization bottleneck gives LLM 20x more patient history","EHR2Path simulates next-hour states across ED, ward, ICU","One model forecasts deterioration, recovery, and care transitions","Pathway-level EHR modeling beats specialized baselines"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000403,"raw_usage":{"total_tokens":2138,"prompt_tokens":1023,"completion_tokens":1115,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":639,"completion_tokens_details":{"reasoning_tokens":1033}},"tokens_in":639,"tokens_out":1115,"duration_ms":8081,"temperature":1.0,"reasoning_tokens":1033,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:33:34.360666+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a cohort of patients who experience rapid deterioration (e.g., an unplanned ICU transfer within six hours of ED vitals), compare the model's next-hour vital-sign forecasts against raw chart data at five-minute resolution: if the model's error on the hour preceding deterioration is no worse than on stable hours, the hourly bottleneck is safe; if it systematically misses the deterioration by predicting near-baseline values, the representation is lossy.","supporting_citations":[{"cited_title":"Zero shot health trajectory prediction using transformer","cited_arxiv_id":null,"evidence_quote":"ETHOS baseline, the prior trajectory-simulation model whose feature scope and token window EHR2Path extends."},{"cited_title":"General-purpose retrieval-enhanced medical prediction model using near-infinite history","cited_arxiv_id":null,"evidence_quote":"REMed baseline, a long-context ICU outcome model used as comparison for outcome fine-tuning."},{"cited_title":"Mimic-extract: A data extraction, preprocessing, and representation pipeline for mimic-iii","cited_arxiv_id":null,"evidence_quote":"MIMIC-Extract pipeline, source of the hourly aggregation scheme."},{"cited_title":"A comprehensive ehr timeseries pre-training benchmark","cited_arxiv_id":null,"evidence_quote":"Prior work whose preprocessing approach and evaluation tasks partly inspire the simulation tasks."},{"cited_title":"Pereira, and William Bialek","cited_arxiv_id":null,"evidence_quote":"Information bottleneck principle, theoretical basis for the summary compression objective."}],"review_version":1}