{"id":"1ed98612-9b7a-45e5-8337-722976e42930","arxiv_id":"2411.09361","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Pretraining on time-to-event labels from longitudinal EHRs improves 3D CT model predictions of future outcomes by up to 31.6% relative AUROC, while preserving diagnostic accuracy.","lead":"This paper shows that pretraining 3D CT scan models to predict the timing of future medical events, using patients' electronic health records, improves their accuracy at forecasting outcomes like death and readmission. A test on about 19,000 chest CT scans found large gains in risk prediction without hurting standard diagnostic classification.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 8 prognostic benchmarks are drawn from the same INSPECT code space as the 8,192 pretraining tasks, and the paper never states that these outcomes were excluded; the headline AUROC/C-index gains may therefore reflect in-task improvement rather than general prognostic transfer.","rationale":"The reader's weakest assumption was proxy-outcome validity; that is a real threat, but the more immediately decisive gap is that the benchmark tasks may be contained in the pretraining task set, which can be checked directly from the code lists. The label-density confound is also real but secondary: it affects interpretation of which component of TTE helps, whereas task overlap affects whether the headline numbers establish the claimed general capability. The paper has genuine strengths—public code and checkpoints, three architectures, bootstrap CIs, a sensible base/visit ablation, and external RSPECT diagnostic transfer—but RSPECT does not provide prognostic outcomes, so it cannot rescue the outcome-prediction claim. The verdict remains conditional: the central claim is plausible and well engineered, but it needs a holdout-task or external-cohort prognostic evaluation before the abstract's general claim can be accepted without qualification.","tokens_in":35557,"tokens_out":8517,"duration_ms":86742,"concrete_test":"Retrain the TTE model with all codes in the ontology subtrees of the 8 benchmark outcomes (mortality, readmission, PH, ATX, CMG, CONS, EDM, PEFF) explicitly removed from the 8,192-task selection, then rerun Tables 2 and 3 against base/visit. If the AUROC/C-index advantage over base/visit disappears or becomes non-significant, the headline results are explained by task overlap rather than by general temporal pretraining. A complementary external check is to freeze the TTE encoder and evaluate it on an outcome absent from pretraining, e.g., lung cancer incidence or 5-year mortality in an external CT cohort, requiring a significant improvement over base/visit.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4 builds the pretraining signal by 'treating each code as a separate task' and selecting 8,192 codes from the EHR ontology by conditional entropy; Section 5 then defines 5 of the 8 prognostic evaluation tasks (ATX, CMG, CONS, EDM, PEFF) from the same INSPECT EHR codes (Appendix B), with PH, mortality, and readmission also natural entries in that code/event space. The manuscript does not state that these evaluation outcomes were held out of the 8,192-task set. If they were included, base/TTE was directly optimized to predict the exact future codes used for evaluation, whereas base/visit—the most informative baseline—was pretrained on the same code vocabulary but only with same-visit labels and therefore did not directly optimize the future-event evaluation tasks. The reported average +23.7% AUROC and +29.4% C-index gains then compare an in-distribution evaluation against out-of-distribution baselines, rather than demonstrating that time-to-event pretraining learns generalizable prognostic representations. No external prognostic task is used; RSPECT supplies only diagnostic labels, so the outcome-prediction claim rests entirely on tasks from the pretraining distribution. This gap is load-bearing because the abstract's promise of leveraging EHR temporal supervision for outcome prediction, and the framing around prognostic pixel biomarkers, require transfer beyond the pretraining vocabulary.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes time-to-event (TTE) pretraining for 3D medical imaging encoders. Using 18,945 chest CT scans from INSPECT linked to longitudinal EHR data, the authors convert medical codes into 8,192 TTE prediction tasks and continue-pretrain SwinUNETR, DenseNet-121, and ResNet-152 backbones. The frozen encoders are then probed on prognostic TTE tasks, prognostic binary classification, and diagnostic classification (RSPECT). The central claim is that TTE pretraining improves outcome prediction—average AUROC +23.7% and Harrell's C-index +29.4% across 8 benchmark tasks—without hurting diagnostic classification. The paper includes baselines (base, base/MTL, base/visit), bootstrap confidence intervals, significance tests, and additional metrics (time-dependent C-statistics, integrated Brier score, subgroup analyses, GradCAM).","tokens_in":35838,"tokens_out":3047,"duration_ms":31669,"significance":"If the effect is real and generalizes beyond the pretraining label vocabulary, TTE pretraining would be a valuable and scalable way to inject temporal prognostic signal into 3D imaging encoders, with practical implications for medical foundation models. The study is carefully executed in several respects: it uses public datasets, releases code and checkpoints, evaluates three architectures, includes an ablation that shares the same 8,192-task vocabulary (base/visit), reports bootstrap CIs with significance tests, and provides calibration and subgroup analyses. These are genuine strengths. However, the central generalization claim is weakened by the apparent overlap between pretraining tasks and evaluation tasks, and by the confounding of temporal scope with label density in the key comparison. As a result, the headline numbers are better interpreted as in-distribution task improvement than as evidence of transfer to novel prognostic outcomes.","major_comments":[{"comment":"The manuscript does not state that the 8 prognostic evaluation tasks (Mortality, Readmission, PH, ATX, CMG, CONS, EDM, PEFF) were held out of the 8,192 pretraining tasks. Section 4 says pretraining tasks are created by 'treating each code as a separate task' and selecting 8,192 codes from the EHR ontology (INSPECT), while Section 5 defines the five additional TTE tasks (ATX, CMG, CONS, EDM, PEFF) from the same INSPECT EHR event space (Appendix B); Mortality, Readmission, and PH are also EHR events. Because the manuscript is silent on exclusion, a reader cannot rule out that base/TTE was directly optimized to predict the exact future codes used for evaluation, whereas base/visit—the most informative baseline—was pretrained on the same code vocabulary but with same-visit labels and thus did not directly optimize those future-event evaluation tasks. In that case, the reported +23.7% AUROC and +29.4% C-index gains compare an in-distribution evaluation against an out-of-distribution baseline, rather than demonstrating generalizable prognostic transfer. This is load-bearing for the abstract's claim that temporal supervision from EHRs improves outcome prediction. The authors should either (a) verify and state explicitly that all evaluation codes were excluded from the 8,192-task pretraining set, and provide the task-selection details needed to confirm this, or (b) re-run the evaluation on held-out tasks that were provably not in the pretraining task family, ideally including an external dataset with survival outcomes. Without such evidence, the headline claims should be reframed as in-vocabulary task improvement.","section":"§4 (Creating TTE Pretraining Tasks), §5 (Datasets & Evaluation Tasks), Appendix B"},{"comment":"The comparison between base/TTE and base/visit, which the paper identifies as the most informative ablation, varies temporal scope and label density simultaneously. Base/visit uses the same 8,192 tasks but labels assigned to the same visit as the CT scan, yielding on average 3x fewer labels per image (Figure 3); base/TTE uses future events with full survival times. The observed gains could therefore be driven by the higher label density, the temporal distance of the labels, or the survival objective itself. The text attributes the improvements specifically to temporal supervision (e.g., 'TTE Supervision Improves Training Data Efficiency' and 'TTE Pretraining Improves Prognostic Performance' in Section 7), but the design cannot separate these factors. A control that matches the number of labels per image while ablating the future-time component (e.g., binary future-event occurrence with the same density, or TTE labels subsampled to the base/visit density) is needed to support the claim that time-to-event modeling, rather than simply more labels, drives the effect. This is a load-bearing point for Hypothesis 2 and for the paper's central contribution.","section":"§5 (Model Baselines), §6 (Results), Figure 3, Tables 2–3"}],"minor_comments":[{"comment":"In Equation (5) of Appendix M, the indicator 'I(Se ≤ t ≤ Sp)' appears to contain a typo; the main text's survival function uses intervals with start Sp and end Ep, so the indicator should likely read 'I(Sp ≤ t ≤ Ep)' or equivalent. Please correct this and the corresponding line in Equation (6).","section":"Appendix M, Eq. (5) and Eq. (6)"},{"comment":"Some entries in the confidence-interval tables are malformed: Table 14 contains '(-0.579 0.001)*' (missing comma) and Table 16 contains a stray '0.020' before a parenthesized interval in the SwinUNETR base EDM row. These formatting errors make the tables hard to read and should be fixed.","section":"Appendix K, Tables 14 and 16"},{"comment":"The text says 'INSPECT defines 3 prognostic binary tasks' and then later introduces '5 additional prognostic TTE tasks'; the relationship between the binary prognostic tasks and the TTE task set is described in a way that could confuse readers about which tasks are evaluated in which table. Please clarify whether the same underlying outcomes are evaluated in both formulations and how the eight benchmark tasks in the abstract map to Tables 2 and 3.","section":"§5 (Datasets & Evaluation Tasks)"}],"recommendation":"major_revision","confidential_remarks":"The core methodological concern—overlap between pretraining and evaluation task families without an explicit hold-out statement—is serious enough that the current claims in the abstract and conclusion outrun the evidence. However, the issue is fixable within the scope of the manuscript: the authors can either confirm exclusion with task-selection details or add held-out and/or external evaluations. The label-density confound in the base/TTE vs base/visit comparison also needs an additional ablation. I recommend major revision rather than rejection because the experimental infrastructure (architectures, baselines, metrics, public data, released code) is strong and the central idea is plausible. I would also encourage the authors to be more precise in the abstract about 'in-vocabulary' versus 'held-out' outcome improvements. I do not see evidence of intentional misreporting, but the omission of the hold-out statement is a notable oversight for a paper whose headline is about prognostic transfer."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe headline is simple: TTE pretraining does something real for 3D CT encoders, but the size of the reported effect is not credible from the evaluation as designed. The 8,192 pretraining tasks are built from EHR codes, and the 8 prognostic benchmarks are defined from the same INSPECT code space. The paper never states that those evaluation codes were held out of the pretraining task set. If they were not, base/TTE was directly optimized to predict the exact future events used for evaluation, while base/visit was not. That turns the +23.7% AUROC / +29.4% C-index headline into an in-task comparison, not evidence of generalizable prognostic transfer.\n\nWhat is genuinely new and good here: this is the first large-scale TTE pretraining study for 3D imaging, with 8,192 survival tasks, three architectures, and careful baselines. The base/visit control is a smart ablation — same code vocabulary, same number of tasks, visit-level labels instead of future events. The bootstrapped CIs and significance tests are there. Diagnostic performance on RSPECT is roughly preserved, which is a useful negative result. The code and checkpoints are public, so the work is reproducible. Those are real merits.\n\nThe soft spots are about evaluation, not execution. The main confound is the label overlap I mentioned. A second, smaller issue: the abstract reports relative gains, which flatters the numbers (e.g., an absolute AUROC jump of ~0.1 reads as 23.7% relative). Third, the survival gain over base/visit is modest in absolute terms — C-index around 0.73 vs 0.70 on the five added tasks — and the confidence intervals in Appendix K show several comparisons where the advantage is not overwhelming. None of that kills the idea, but it brings the claim into proportion.\n\nBottom line: the paper is a solid contribution in need of a matched hold-out evaluation. A serious referee should ask for a clear statement of whether evaluation codes were excluded from the 8,192-task set, a matched label-density baseline, and ideally one external outcome task. I'd send it out, but I'd expect revision before I'd trust the headline numbers.","headline":"TTE pretraining moves prognostic signal into 3D CT encoders, but the headline gains are inflated by likely overlap between the 8,192 pretraining codes and the 8 benchmark outcomes.","tokens_in":36397,"tokens_out":3158,"would_cite":false,"duration_ms":29734,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Time-to-event pretraining—predicting when each EHR medical code first appears after a CT scan—improves 3D imaging outcome prediction by an average 23.7% AUROC and 29.4% C-index across eight benchmarks, without sacrificing diagnostic…","keywords":["time-to-event pretraining","3D medical imaging","longitudinal electronic health records","survival analysis","CT scans","prognostic biomarkers","self-supervised learning","foundation models"],"falsifier":"Train an identical model with the same 8,192 tasks but shuffle event times across patients, preserving each code's marginal time distribution; if the AUROC and C-index gains over the same-visit baseline persist, the model is not actually using temporal order, and the claimed mechanism is wrong. Alternatively, re-run the pipeline on an external health system with different coding practices; if the gains vanish, the signal is documentation-driven rather than image-driven biology.","tokens_in":35348,"feed_emoji":"🩻","tokens_out":8170,"duration_ms":75114,"temperature":0.7,"pith_summary":"Time-to-event pretraining is a proposed way to teach 3D medical image encoders to see future risk, not just current anatomy. The paper claims that using thousands of electronic-health-record codes as survival-analysis tasks—predicting the time until each code first appears after a CT scan—gives the encoder a temporal supervision signal that self-supervised methods lack. On a public dataset of 18,945 chest CTs with a median five years of follow-up, the resulting encoders improve average AUROC by 23.7% and Harrell's C-index by 29.4% across eight benchmark outcome tasks, with no loss on diagnostic classification. If correct, this establishes longitudinal EHR timing as a scalable pretraining signal for medical imaging foundation models.","feed_headline":"Time-to-event pretraining lifts CT prognosis by 23.7%","feed_subtitle":"Using 8,192 EHR tasks, 3D encoders gain 29.4% in C-index without hurting diagnosis.","key_machinery":"The load-bearing mechanism is the time-to-event task set plus a piecewise exponential neural network (PEANN) survival loss. Each medical code in an ontology knowledge graph is treated as a separate pretraining task whose label is the time from the CT scan to that code's first occurrence; right-censored patients (including those who die before the code appears) still contribute through the survival likelihood. Task selection keeps 8,192 informative codes out of 4.3 million candidates by solving an entropy-based vertex cover over the ontology's directed acyclic graph, following the selection rule used for structured-EHR pretraining. After continued pretraining, the encoder is frozen and evaluated with a logistic-regression head for classification or a Cox proportional-hazards (DeepSurv) head for survival tasks.","core_discovery":"The paper's central discovery claim is that temporal supervision—specifically, predicting the time until each of 8,192 EHR medical codes first occurs after a CT scan—is a usable and effective pretraining signal for 3D medical image encoders. On the INSPECT dataset of 18,945 chest CTs linked to a median of five years of follow-up, encoders continued-pretrained with this time-to-event objective outperform models trained with the same tasks restricted to the current visit and models trained with multitask classification, by an average 23.7% AUROC and 29.4% Harrell's C-index across eight prognostic benchmarks. The same encoders do not lose diagnostic classification performance on eight RSPECT image-labeling tasks, and calibration (integrated Brier score) improves by an average of 54%. The paper interprets this as evidence that future-guided temporal context is what existing self-supervised methods are missing.","pith_inferences":["If the temporal signal is the active ingredient, the same recipe should transfer to other modalities with longitudinal EHR linkage, such as MRI or PET, and to 2D radiography; the paper only tests chest CT, so this is an extrapolation.","A risk not settled by the paper is that code timing may partly encode follow-up intensity or documentation practice; testing on an external health system with different coding behavior would separate biology from billing artifacts.","Because task selection maximizes conditional entropy, the chosen 8,192 codes may over-represent frequently documented events; ablating by ontology type (diagnoses vs. medications vs. procedures) would show which code families actually drive the prognostic gain.","A practical extension the paper does not develop: TTE pretrained encoders could be used for opportunistic screening of underdiagnosed conditions from routine CTs, since the pretraining objective foregrounds future-disease signal in pixels."],"forward_implications":["Encoders pretrained this way can be adapted to new tasks with lightweight heads: the frozen encoder plus linear or Cox head is the evaluation protocol, so downstream deployment is cheap.","The same CT scan yields roughly three times more training labels when future EHR time is used, so the method raises data efficiency for a given pretraining corpus.","Prognostic gains come with preserved or slightly improved diagnostic classification, so adding temporal supervision does not force a trade-off against current-image labeling tasks.","Right censoring is handled inside the pretraining loss, so patients who are lost to follow-up contribute to pretraining instead of being dropped.","The paper's reported gains extend across three architectures (SwinUNETR, DenseNet-121, ResNet-152), suggesting the effect is not tied to one backbone."],"supporting_citations":[{"why":"Supplies the INSPECT dataset of paired chest CTs and longitudinal EHRs, the source of all pretraining images and time-to-event labels.","marker":"Huang et al. (2024)"},{"why":"Supplies the conditional-Shannon-entropy vertex-cover task selection and the 8-piece, 8,192-task TTE hyperparameters used for pretraining.","marker":"Steinberg et al. (2024)"},{"why":"Defines the piecewise exponential neural network whose survival likelihood is the pretraining loss.","marker":"Fornili et al. (2014)"},{"why":"Provides the 3D self-supervised SwinUNETR weights that serve as a starting checkpoint for the strongest architecture.","marker":"Valanarasu et al. (2023)"},{"why":"Merlin is the EHR-supervised contrastive-plus-classification pretraining approach that TTE must beat as a baseline.","marker":"Blankemeier et al. (2024)"},{"why":"DeepSurv supplies the Cox proportional-hazards head used to evaluate downstream time-to-event performance.","marker":"Katzman et al. (2018)"},{"why":"RSPECT provides the radiologist-labeled CT dataset used to check that pretraining does not hurt diagnostic classification.","marker":"Colak et al. (2021)"},{"why":"Defines the C-index used as the primary prognostic ranking metric across the benchmark tasks.","marker":"Harrell et al. (1982)"}],"fun_headline_variants":["EHR time-to-event supervision lifts 3D CT prognosis 29.4% C-index","Pretraining 3D CTs on disease timing improves AUROC 23.7%","Time-to-event pretraining on 18,945 CTs lifts C-index 29.4%","Future health prediction as pretraining signal: CT models gain 23.7% AUROC"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the time until a medical code appears in a patient's EHR is a valid, learnable proxy for the clinical outcome tied to that code, so that the image-to-code timing relationship is a real biomarker signal rather than an artifact of who gets tested or how often they visit.","fun_headline_variants_meta":{"raw":{"variants":["EHR time-to-event supervision lifts 3D CT prognosis 29.4% C-index","Pretraining 3D CTs on disease timing improves AUROC 23.7%","Time-to-event pretraining on 18,945 CTs lifts C-index 29.4%","Future health prediction as pretraining signal: CT models gain 23.7% AUROC"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00094,"raw_usage":{"total_tokens":4020,"prompt_tokens":952,"completion_tokens":3068,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":568,"completion_tokens_details":{"reasoning_tokens":2970}},"tokens_in":568,"tokens_out":3068,"duration_ms":22262,"temperature":1.0,"reasoning_tokens":2970,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:43:31.669560+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train an identical model with the same 8,192 tasks but shuffle event times across patients, preserving each code's marginal time distribution; if the AUROC and C-index gains over the same-visit baseline persist, the model is not actually using temporal order, and the claimed mechanism is wrong. Alternatively, re-run the pipeline on an external health system with different coding practices; if the gains vanish, the signal is documentation-driven rather than image-driven biology.","supporting_citations":[{"cited_title":"MOTOR: A time-to-event founda- tion model for structured medical records","cited_arxiv_id":null,"evidence_quote":"Supplies the conditional-Shannon-entropy vertex-cover task selection and the 8-piece, 8,192-task TTE hyperparameters used for pretraining."},{"cited_title":"Security, Data Storage, and Compliance: All authors involved in data handling have completed institutional training on HIPAA and data privacy before engaging with the data","cited_arxiv_id":null,"evidence_quote":"RSPECT provides the radiologist-labeled CT dataset used to check that pretraining does not hurt diagnostic classification."}],"review_version":1}