{"id":"45a78485-6162-4201-89e0-1e5e21754393","arxiv_id":"2412.10955","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A multimodal ResNet-LSTM using chest X-rays, EHRs, and ECGs achieves AUROC 0.86 for T2DM screening, but the evaluation leaks patients across train/test splits.","lead":"This paper tests whether adding chest X-ray images to electronic health records improves automated screening for type 2 diabetes, reporting an AUROC of 0.86. The improvement over an X-ray-only model is hard to trust because patients were split by hospital visit, not by person, so the same patient can be in both the training and test sets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 2.3% AUROC improvement is not a controlled comparison: the only CXR-only baseline is an external model trained on a different cohort, and no same-dataset unimodal baseline is reported, so the central claim is unsupported even before considering patient leakage.","rationale":"The reader's verdict is REJECT, and I agree with that outcome, but my most load-bearing concern differs from the reader's weakest_assumption. The reader focuses on patient leakage from the random episode-level split; that is a genuine and serious flaw, acknowledged in Section V. However, the central claim is specifically a 2.3% improvement over a CXR-only baseline, and that baseline is an external model evaluated on a different cohort. Even a perfectly clean patient-level split would not make the comparison valid, because the baseline's AUROC of 0.84 was computed on Pyrros et al.'s own test data, not on the MIMIC test set used here. The paper's 'Missing CXR' ablation is not a unimodal baseline; zeroing CXR is not the same as removing the CXR stream from training and inference. Thus the headline result is not supported by a controlled experiment. I would add that the shared CXR/ECG across episodes creates both patient-identity leakage and temporal leakage, since the same image may be acquired after the episode being predicted. A same-dataset CXR-only baseline evaluated on a patient-level split would settle whether the fusion actually adds value. The paper deserves credit for public data and released preprocessing code, but the empirical claim requires re-analysis before it can be accepted.","tokens_in":21456,"tokens_out":7467,"duration_ms":64457,"concrete_test":"Train a CXR-only model with the same ResNet50 encoder and a classification head on the DE+C train split (N=9863), without EHR or ECG inputs, and evaluate on the same DE+C test split (N=2819) using a patient-level split that keeps all episodes of a patient in one partition. If the CXR-only AUROC falls within the 95% CI of ResNet-LSTMJoint (0.8425–0.8751), the claimed 2.3% improvement is not reproduced under controlled conditions.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (abstract; Section IV-B-2, Table IV) is that the ResNet-LSTMJoint model 'surpasses' the CXR-only baseline by 2.3% AUROC. That baseline is not a model trained and evaluated under the same protocol: it is the published ResNet34 of Pyrros et al. [20], trained on 271,065 CXRs from a different cohort, with AUROC 0.84 on its own test set. The paper reports no CXR-only or EHR-only model trained on the DE+C or DE+C+G train split and evaluated on the corresponding test split. The 'Missing CXR modality' ablation (Section IV-C-3, Table V) zeroes out CXR inputs rather than providing a unimodal CXR-only baseline, so it cannot validate the additive value of CXR over EHR. Consequently, the 2.3% gap may reflect differences in cohort, label prevalence, or image-acquisition protocol rather than the multimodal fusion itself. This is the most direct threat to the stated claim; the episode-level random split (Section III-C) and reuse of the same CXR across a patient's episodes (Section V) further inflate the absolute AUROC, but they are secondary to the absence of a controlled baseline.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes multimodal deep learning models for noninvasive screening of type 2 diabetes (T2DM) from chest X-ray (CXR) images, time-series electronic health records (EHR), and 12-lead ECG signals, using MIMIC-IV, MIMIC-CXR-JPG, and MIMIC-IV-ECG. Two fusion paradigms are evaluated: an early-fusion ViLT transformer and a joint-fusion ResNet-LSTM with two training strategies. The best model, ResNet-LSTMJoint, is reported to achieve AUROC 0.8616 on the EHR+CXR+ECG dataset and 0.8592 on the EHR+CXR dataset, and the paper claims a 2.3% AUROC improvement over a CXR-only baseline from the literature. Ablation studies examine lack of pre-training, noisy inputs, and missing CXR modality.","tokens_in":21680,"tokens_out":4240,"duration_ms":38282,"significance":"If the central claim were validated, the work would offer a useful contribution to opportunistic T2DM screening, given the ubiquity of CXR imaging. The paper has concrete strengths: it uses public benchmark databases, releases its preprocessing pipeline, reports bootstrapped confidence intervals, and includes three ablation studies that give insight into model behavior. However, the headline comparison is not a controlled experiment, and the evaluation protocol contains a patient-level data leakage mechanism. Because these issues directly affect the main claim, the current version does not provide reliable evidence for the stated improvement over CXR-only screening.","major_comments":[{"comment":"The random split of episodes rather than patients violates the independence assumption for evaluation. Section III-C states that episodes are 'partitioned randomly' into train, validation, and test sets, and Section V acknowledges that 'subsequent ICU stays of a patient' are treated as separate samples sharing the same CXR and ECG. With 14,091 episodes from 7,861 patients, many patients will contribute episodes to more than one split, and the same CXR/ECG is reused for all episodes of a patient. This allows patient-specific features to leak across splits and inflates the reported AUROC. The authors should perform a patient-level split (grouping episodes by subject_id) and report test metrics under that protocol, together with the number of patients per split and the fraction of patients that appear in more than one split.","section":"§III-C and §V"},{"comment":"The 2.3% improvement over the 'CXR-only baseline' is not a controlled comparison. The baseline is the externally published ResNet34 model of Pyrros et al. [20], trained on 271,065 CXRs from a different cohort and evaluated on that cohort's own test set. No CXR-only, EHR-only, or ECG-only model is trained on the DE+C or DE+C+G train splits and evaluated on the corresponding test splits. Consequently, the 2.3% gap could reflect differences in cohort composition, label prevalence, image acquisition, or evaluation protocol rather than the benefit of multimodal fusion. The authors should train unimodal baselines on the same data and splits (e.g., a ResNet50 CXR-only model, an LSTM EHR-only model, and an ECG-only model) and compare against the multimodal models under identical conditions.","section":"§IV-B-2, Table IV"},{"comment":"The 'Missing CXR modality' ablation does not establish the diagnostic utility of CXR in the way the text claims. The experiment takes a model trained with CXR inputs and zeroes CXR at test time for 30, 50, or 70% of samples. This measures robustness to missing inputs, not the additive predictive contribution of CXR relative to a unimodal model. The statement that 'This ablation study also clearly demonstrates the diagnostic utility of CXRs' is not supported by the design, because the drop from AUROC 0.86 to 0.74 could also reflect a distribution shift introduced by zeroing a modality the model was trained to use. The paper needs a model trained without CXR at all, or a proper unimodal baseline, to support conclusions about CXR's contribution.","section":"§IV-C-3, Table V"},{"comment":"The label definition is not temporally aligned with the screening claim. T2DM status is assigned from T2DM ICD codes appearing anywhere in the patient's EHR, while the CXR, ECG, and EHR inputs are drawn from the same hospitalization period (e.g., the CXR is selected within a 30-day window around the first admission and final discharge). As a result, the model may be detecting an already-documented diagnosis rather than screening for undiagnosed T2DM. The paper should either restrict the label window to diagnoses occurring after the index date, or reframe the task as detection/classification of known T2DM rather than screening. This distinction is important for the clinical interpretation of the reported performance.","section":"§III-B and §V"}],"minor_comments":[{"comment":"The phrase 'resized to a width of 384 pixels on the shorter side' is ambiguous; it should probably read that the shorter side is resized to 384 pixels while maintaining the aspect ratio.","section":"§III-B-2"},{"comment":"Table III reports only episode counts; patient counts per split would help readers assess the extent of patient overlap across splits.","section":"Table III"},{"comment":"The ResNet-LSTM training details mention 'early stopping is set to a patience level of 5 training epochs' and a learning-rate reduction factor of 0.5 on validation-loss plateau; specifying which of these triggered first in the reported runs would improve reproducibility.","section":"§IV-A-2"},{"comment":"The architecture diagrams are small and the text within them is difficult to read; higher-resolution figures or vector graphics would help.","section":"Figures 1 and 2"},{"comment":"Reference formatting is inconsistent: some entries include DOIs and some only URLs, and several arXiv papers lack the arXiv identifier in the visible text; aligning all references to the journal style would improve readability.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper's reported headline improvement over the external CXR-only baseline is not supported by the current experimental design. The two main problems, episode-level splitting and the lack of any same-data unimodal baseline, are fixable through re-analysis: the authors can retrain with a patient-level split and add CXR-only, EHR-only, and ECG-only models. I therefore recommend major revision rather than rejection, but the revision must include these new experiments; a text-only rebuttal would not suffice. The authors' public code and use of MIMIC data should make the additional experiments feasible. I would also ask the editor to monitor whether the revised paper addresses the temporal labeling issue, since the word 'screening' may be too strong for a model that uses the same hospitalization's ICD codes as labels."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nTwo things to know upfront. The paper is the first to combine chest X-rays with time-series EHRs for T2DM screening, and it releases a genuinely useful preprocessing pipeline for MIMIC-IV. That is real. The headline claim—a 2.3% AUROC gain of joint ResNet-LSTM over CXR-only screening—is not supported by the evaluation as written.\n\nThe only CXR-only baseline is Pyrros et al.'s ResNet34, trained on 271,065 CXRs from a different cohort and evaluated on its own test set. The authors train no CXR-only or EHR-only model on the same DE+C or DE+C+G splits. So the 2.3% gap could reflect cohort, label prevalence, or image-acquisition differences rather than multimodal fusion. The missing-CXR ablation zeroes the image input; it tells us about robustness to missing modalities but does not provide the needed unimodal baseline. This is the primary problem.\n\nSecondary, but real: the split is by episode, not by patient. With 14,091 episodes from 7,861 patients, and the paper itself noting that subsequent ICU stays of a patient share the same CXR and ECG, patients leak across train and test. That inflates absolute AUROC and makes the 0.86 an optimistic estimate. To the authors' credit, they acknowledge this in the conclusion, so the flaw is not hidden.\n\nWhat the paper does well: clear writing, public data, bootstrapped confidence intervals, ablations for pre-training, noise, and missing modalities, and a released pipeline. The early-fusion vs joint-fusion comparison is a sensible use of existing architectures; the novelty is the application and the data curation, not the models themselves.\n\nFor a clinical screening claim, this needs a patient-level split and same-dataset unimodal baselines before the 2.3% number is meaningful. I would not cite the AUROC claim as is; I would cite the released pipeline if I were doing multimodal ICU work.\n\nRecommendation: send it to review, because the question and the released dataset justify referee time and the issues are fixable. But the abstract should be read as a proposal, not a verified result.","headline":"First CXR+EHR T2DM screening study with a useful released pipeline; the 2.3% AUROC claim is not supported because the only CXR baseline comes from a different dataset and no same-data unimodal baseline is trained.","tokens_in":22235,"tokens_out":4287,"would_cite":true,"duration_ms":35329,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Combining chest X-ray images with time-series electronic health records in an end-to-end trained multimodal model screens for type 2 diabetes more accurately (AUROC 0.86) than chest X-rays alone, using far fewer training samples.","keywords":["type 2 diabetes screening","chest X-ray","electronic health records","multimodal deep learning","time series EHR","ResNet-LSTM","early fusion transformer","noninvasive screening"],"falsifier":"Train and test the same models on a patient-level split (no patient in more than one partition) or on an external cohort; if the EHR+CXR AUROC drops to roughly the CXR-only level (0.84), the claimed 2.3% improvement is an artifact of episode-level data leakage rather than genuine multimodal signal.","tokens_in":21207,"feed_emoji":"🩻","tokens_out":9147,"duration_ms":70325,"temperature":0.7,"pith_summary":"This paper claims that combining chest X-ray images with time-series electronic health records—two noninvasive data sources already captured in routine hospital care—can screen for type 2 diabetes more accurately than chest X-rays alone. The best-performing model, an end-to-end trained ResNet-LSTM that fuses a pre-trained ResNet50 image encoder with LSTM encoders for 48-hour vital-sign sequences, reaches an AUROC of 0.8616 (0.8592 without ECG), surpassing the CXR-only baseline's 0.84 by about 2.3 percentage points using only 9863 training episodes. The authors also compare an early-fusion multimodal transformer (ViLT), which reaches 0.8481 but is less robust to missing CXR inputs. If the claim holds, opportunistic, noninvasive T2DM screening could piggyback on chest radiographs that are already being performed, with no additional blood tests or patient visits.","feed_headline":"Fusing chest X-rays and EHR lifts diabetes screening to 0.86 AUROC","feed_subtitle":"A multimodal ResNet-LSTM reaches 0.86 AUROC with far fewer training images than the X-ray-only baseline.","key_machinery":"The load-bearing mechanism is the joint-fusion ResNet-LSTM encoder architecture: a pre-trained ResNet50 encodes each chest radiograph, while two single-layer LSTMs with 256 hidden units encode time-series EHR inputs (96 timesteps × 11 features, sampled every 30 minutes for 48 hours) and ECG matrices (100×12); the three representations are concatenated and passed to a fully connected binary classifier, with all encoders and the classifier trained synchronously. The paper contrasts this with the ViLT early-fusion transformer, which linearly projects EHR and ECG rows and CXR patches into 768-dimensional tokens that share a single transformer encoder. The end-to-end joint training, the authors argue, lets the model exploit cross-modal interactions dynamically instead of freezing pre-trained encoders, which is what they credit for the performance edge over the stage-wise trained variant.","core_discovery":"The paper's central discovery is that joint, end-to-end fusion of chest X-ray images with time-series EHR data improves T2DM screening over a strong image-only baseline. On the test set of 2819 episodes, the joint-fusion ResNet-LSTM attains AUROC 0.8592 (95% CI 0.8425–0.8751) with EHR+CXR, and 0.8616 (95% CI 0.8469–0.8757) when 12-lead ECG is added, against the CXR-only classifier's 0.84 (95% CI 0.83–0.85). These gains come with far fewer training samples (9863 versus 271,065 CXRs). Ablation experiments show that zeroing out the CXR in 30–70% of test samples degrades AUROC substantially (to 0.68–0.81), confirming the image modality carries signal, and that the early-fusion ViLT transformer is more noise-robust but overall weaker than the ResNet-LSTM.","pith_inferences":["The apparent gain over the image-only baseline may be driven largely by demographic and vital-sign features in the EHR (age, weight, heart rate) rather than by image content; the paper reports a missing-CXR ablation but no EHR-only baseline, so the unique contribution of imaging remains untested.","Because each patient's episodes share the same CXR and ECG, a random episode split lets the model see the exact same image in training and test; a patient-level split could shrink the reported advantage.","If patient-level validation confirms the result, the most natural deployment is in inpatient and ICU settings where CXRs are routine, before any outpatient screening program is considered."],"forward_implications":["Hospitals that already perform chest radiographs could flag patients at high T2DM risk without extra procedures or blood tests.","The 2.3% AUROC gain over a baseline trained on 271,065 CXRs, obtained with 9863 episodes, suggests multimodal fusion is substantially more data-efficient for this screening task.","Adding ECG to the EHR+CXR combination yields only a small AUROC increment (0.8616 versus 0.8592), so ECG's marginal value for this screening use case is limited.","The released preprocessing pipeline allows other groups to reconstruct comparable multimodal T2DM datasets from public intensive-care databases, enabling external validation."],"supporting_citations":[{"why":"This citation provides the CXR-only baseline (AUROC 0.84) that the paper's multimodal models are compared against.","marker":"[20]"},{"why":"This citation supplies the EHR time-series preprocessing approach (episode extraction, vital-sign selection) that the paper extends.","marker":"[28]"},{"why":"This citation provides the modular joint-fusion architecture and the early/joint encoder-training strategies used to build the ResNet-LSTM models.","marker":"[63]"},{"why":"This citation is the source of the electronic health records used to construct the EHR modality.","marker":"[52]"},{"why":"This citation is the source of the chest X-ray images used to construct the CXR modality.","marker":"[54]"},{"why":"This citation is the source of the 12-lead ECG signals used to construct the ECG modality.","marker":"[56]"},{"why":"This citation is the pre-trained ViLT vision-language transformer that the paper adapts for early-fusion multimodal T2DM screening.","marker":"[67]"}],"fun_headline_variants":["CXR plus EHR: deep learning screens diabetes at 0.86 AUROC","Multimodal fusion of X-ray and EHR boosts diabetes screening","AI uses chest X-ray and EHR to noninvasively flag diabetes","Fewer images, same accuracy: X-ray+EHR diabetes screening hits 0.86","Deep learning combines CXR and EHR for early diabetes detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that randomly splitting ICU episodes into train, validation, and test sets keeps patients independent; the paper itself acknowledges that subsequent ICU stays of a patient are treated as separate samples even though they share the same CXR and ECG, so if the same patient appears in multiple splits, the reported AUROC is inflated.","fun_headline_variants_meta":{"raw":{"variants":["CXR plus EHR: deep learning screens diabetes at 0.86 AUROC","Multimodal fusion of X-ray and EHR boosts diabetes screening","AI uses chest X-ray and EHR to noninvasively flag diabetes","Fewer images, same accuracy: X-ray+EHR diabetes screening hits 0.86","Deep learning combines CXR and EHR for early diabetes detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000577,"raw_usage":{"total_tokens":2757,"prompt_tokens":1019,"completion_tokens":1738,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":635,"completion_tokens_details":{"reasoning_tokens":1640}},"tokens_in":635,"tokens_out":1738,"duration_ms":12414,"temperature":1.0,"reasoning_tokens":1640,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:26:57.261827+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train and test the same models on a patient-level split (no patient in more than one partition) or on an external cohort; if the EHR+CXR AUROC drops to roughly the CXR-only level (0.84), the claimed 2.3% improvement is an artifact of episode-level data leakage rather than genuine multimodal signal.","supporting_citations":[{"cited_title":"Pyrros et al., ‘‘Opportunistic detection of type 2 diabetes using deep learning from frontal chest radiographs,’’Nat","cited_arxiv_id":null,"evidence_quote":"This citation provides the CXR-only baseline (AUROC 0.84) that the paper's multimodal models are compared against."},{"cited_title":"Hayat, K","cited_arxiv_id":null,"evidence_quote":"This citation provides the modular joint-fusion architecture and the early/joint encoder-training strategies used to build the ResNet-LSTM models."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"This citation is the pre-trained ViLT vision-language transformer that the paper adapts for early-fusion multimodal T2DM screening."}],"review_version":1}