{"id":"6616e07b-a006-4e10-8cfe-e50fd4fd2937","arxiv_id":"2508.13072","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Fusing labs, ECG, and echo through a text-guided transformer beats single-modality models for heart failure diagnosis (AUC 0.91) and incident-HF risk (C-index 0.61) on a newly curated MIMIC-IV cohort.","lead":"Researchers built a new aligned dataset from MIMIC-IV that pairs lab results, 12-lead ECGs, and echocardiograms for 1,524 heart failure patients, and a framework called TGMM that combines all three for diagnosis, risk prediction, and retrieval tasks. Generalists may read it because multimodal fusion of routine cardiac tests is the direction of practical clinical AI, and this paper supplies both a reusable benchmark and a cautionary case about feature leakage.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Labs text includes same-admission drug and medical history, so the reported HF diagnosis gains may be label leakage; no ablated control is reported.","rationale":"This is the load-bearing premise because the strongest claim is that combining Labs, ECGs, and ECHOs improves HF diagnosis by up to 10% over single modalities and that TGMM outperforms prior fusion baselines. Unless the Labs modality is free of variables caused by the outcome, the comparison cannot establish multimodal synergy. The concern is not about the framework's internal consistency; the architecture may be sound and the dataset artifact useful. It is about whether the headline quantitative result measures what it claims. The reader's weakest_assumption already identifies drug/medical history leakage, and I agree. The reader's CONDITIONAL verdict is appropriate; I would not tighten or loosen it without the ablated control, and with the control the verdict can be upgraded or downgraded accordingly.","tokens_in":25267,"tokens_out":4161,"duration_ms":44720,"concrete_test":"Re-run the HF diagnosis experiments in Table 2 with an ablated Labs input that excludes the Medical and Drug history fields (and, as a second variant, replaces those tokens with random tokens from the same vocabulary), holding every other setting fixed. Compare Labs-only, Lab+ECG, Lab+ECHO, and Lab+ECG+ECHO AUC/accuracy. If trimodal AUC falls from 0.91 by more than about 0.03–0.05, or if the trimodal advantage over ECG+ECHO disappears, the reported fusion benefit is substantially explained by target leakage rather than by complementary modality integration.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.3 lists 'medical history' and 'drug history' as part of the Labs modality, and Fig. 1(a) shows these fields feed the model. Section 4.2.1 textualizes every column as '<column name> of the <object> is <value>', so medication names become BERT tokens. The diagnosis label is prevalent HF defined by ICD codes for the same admission (Section 3.2), so for an HF patient the medication list is a consequence of the diagnosis; 'medical history' may even contain the HF diagnosis itself. Fig. 9(a) shows tokens such as 'antithrombotics' and 'antiplatelets' are major SHAP contributors. No experiment removes or permutes drug/medical history, so the central claim that trimodal fusion adds up to 10% diagnostic AUC is not insulated from target leakage; the Limitation paragraph acknowledges data size, missing values, and single-centre validation but not this risk. If the leak is real, the reported 0.82 unimodal Labs AUC and 0.91 trimodal AUC largely measure the model's ability to recognize HF treatment rather than complementary multimodal physiology.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces HFTri-MIMIC, a curated multimodal dataset from MIMIC-IV containing same-admission laboratory results, 12-lead ECGs, and echocardiograms for 1,524 patients, with a follow-up subset of 455 patients for risk stratification. It then proposes TGMM, an end-to-end framework combining modality-specific foundation encoders, the MedFlexFusion module, a textual guidance module, and a transformer-decoder response module, supporting unimodal, bimodal, and trimodal inputs for heart failure diagnosis, risk prediction, and cross-modal retrieval. The experiments claim that trimodal fusion improves HF diagnosis and risk prediction by up to 10% and 8% over single modalities, that labs plus ECG improves MI diagnosis by up to 5%, and that ablation and explainability analyses support the contribution of each component.","tokens_in":25409,"tokens_out":5907,"duration_ms":62278,"significance":"If the leakage concern is controlled, the paper would make a useful contribution: HFTri-MIMIC is a patient- and time-aligned multimodal resource that appears genuinely absent from the public literature; TGMM is flexible over modality subsets and tasks; the fusion baselines share the same encoders; and external validation on an MI dataset is included. The explainability analyses (attention matrices, SHAP, Grad-CAM++) are also a strength. However, the headline quantitative claims currently rest on an unaddressed label-leakage risk and on significance statements that are internally inconsistent with the reported confidence intervals, so the evidentiary value of the results is substantially weaker than the text asserts.","major_comments":[{"comment":"The Labs modality includes 'medical history' and 'drug history' from the same admission (§3.3), and §4.2.1 textualizes every column as '<column name> of the <object> is <value>', so these fields are fed directly into Bio-Clinical BERT. Since the diagnosis label is prevalent HF defined by same-admission ICD codes (§3.2), a patient's medication list and medical history are consequences of the target label. The SHAP analysis in Fig. 9(a) indeed shows tokens such as 'antithrombotics' and 'antiplatelets' driving predictions. Without an ablation that removes or permutes these columns, the unimodal Labs AUC of 0.82 and the trimodal AUC of 0.91 in Table 2 cannot be interpreted as evidence of complementary multimodal physiology; the central claim of synergistic fusion may reduce to label leakage. Please rerun the diagnosis experiments (and ideally the risk experiments) excluding medical and drug history, report the resulting metrics, and add an explicit discussion of this risk in the Limitation section.","section":"§3.2–3.3, §4.2.1, Fig. 9(a)"},{"comment":"The note 'All results differ significantly under the same modality setting (p<0.05)' is internally contradicted by the table: in the Lab ACC column, the no-TGM linear-response row and the full TGMM row both report 0.77 (0.72, 0.82), so a paired test cannot be significant on those values. More generally, many rows in Table 6 have overlapping 95% confidence intervals, and Section 5.2 reports Wilcoxon signed-rank tests without specifying the pairing units, the number of comparisons, or any multiple-testing correction. Please provide per-comparison p-values, confidence intervals for differences, and a precise description of the test procedure; otherwise the component-wise ablation claims are not supported.","section":"§5.3, Table 6"},{"comment":"The prognosis results rest on only 130 incident events (Table 1) with a best C-index of 0.61±0.05, and several reported 'significant' differences (e.g., TGMM trimodal 0.61 vs Janus-Pro 0.54; TGMM Lab+ECHO 0.59 vs Meancat 0.53) have overlapping standard deviations under the repeated two-fold procedure. The paper does not state whether Wilcoxon tests were applied to fold-level metrics, patient-level scores, or repeated runs, nor does it report C-index confidence intervals or event counts by fold. Please specify the exact test procedure and report uncertainty measures compatible with the paired design.","section":"§5.2, Tables 2, 4, and 5"}],"minor_comments":[{"comment":"The phrase 'relatives weightings' contains a typo and should read 'relative weightings'.","section":"Abstract"},{"comment":"The statement 'all compared approaches employed the same foundation models to ensure a fair comparison' is inaccurate for Qwen2.5-VL and Janus-Pro, which are large multimodal models with their own encoders; please clarify that the claim applies to the fusion baselines that use the same modality encoders as TGMM.","section":"§5.1"},{"comment":"The row labels use checkmarks without an explicit legend, making it hard to tell which ablation configuration each row represents; please add a clear key or describe each row configuration in the caption.","section":"Table 6"},{"comment":"The gating weights αk1, αv1, αk2, αv2 are defined through sums of projections; please specify the dimensions of the projection outputs and how the sums are computed if the K and V projections have different shapes.","section":"§4.2.2, Eqs. (3)–(5)"},{"comment":"C-index values are reported with standard deviations but no confidence intervals; bootstrap confidence intervals would better convey the uncertainty given the small number of incident events.","section":"§5.1"},{"comment":"The term 'adversarial comparison' is a suggestive name for comparing candidate likelihoods; please clarify whether any min-max or adversarial training is actually used, since none is described in the loss formulation.","section":"§4.2.4"},{"comment":"The phrase 'patient- and time-aligned' should clarify whether each patient contributes exactly one admission or whether multiple admissions per patient are possible, since the exclusion criteria are only summarized in Appendix A.","section":"§3.1–3.2"},{"comment":"The Discussion mentions 'AUC 0.97 vs. 0.96, p<0.05' for Qwen2.5-VL without specifying which modality setting or task this refers to; please tie this statement to Table 4 or state the comparison explicitly.","section":"§6"}],"recommendation":"major_revision","confidential_remarks":"The main risk is the same-admission medical and drug history leakage in the HF diagnosis task. I recommend that the authors be required to run leakage-control experiments (removing or permuting those columns) before acceptance; if those experiments show the fusion gains vanish, the central claim collapses. The statistical reporting also needs substantial revision, particularly the impossible significance statement in Table 6."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a heavy-lifting engineering paper building something genuinely new: HFTri-MIMIC, a time-aligned Labs-ECG-Echo cohort from MIMIC-IV with 1,524 patients for diagnosis and 455 for risk, plus raw echoes and follow-up outcomes. The TGMM framework is a sensible assembly: frozen external foundation models, a gated cross-attention fusion module, hybrid human-plus-learnable textual guidance, and a decoder-based response head. The ablation coverage is extensive, and the external MI validation on PTB-XL, while using demographic metadata as Labs rather than lab tests, shows the framework can transfer. The dataset artifact alone is valuable to the community if released. The main soft spot is exactly where the reader put it. Section 3.3 lists medical history and drug history inside the Labs table, and the textualization template in Section 4.2.1 turns drug names into BERT tokens. For a patient coded with prevalent HF on the same admission, the medication list is a consequence of the diagnosis. The paper's own SHAP analysis (Fig. 9a) highlights antithrombotics and antiplatelets as top contributors, and there is no control that removes or permutes these fields. So the trimodal AUC of 0.91 and the unimodal Labs AUC of 0.82 may substantially reflect detection of HF treatment rather than complementary multimodal physiology. The limitation paragraph names data size and missing values, but not this risk. That needs to be fixed with a straightforward ablation before the central claim about synergy can be accepted. Second, the statistics are over-claimed. Table 6 says all paired comparisons are significant at p<0.05 even where the 95% CIs overlap, and the Wilcoxon procedure is never described--paired across which folds, how many tests, any correction? That should be corrected for publication. Third, the risk-stratification result rests on 130 incident events with C-index 0.61 +/- 0.05. That is close to uninformative, and the text should acknowledge it more plainly rather than framing it as a strength. Fourth, the appendices referenced for preprocessing and hyperparameters are not present here, and code is only promised, so the work is not yet reproducible. Proportionately, this is not a desk-reject. It deserves a serious referee who asks for a leakage ablation, a transparent statistical comparison, and the missing appendices. If those come back clean, HFTri-MIMIC and TGMM would be a solid contribution to multimodal clinical decision-support research.","headline":"A serious multimodal cardiac engineering paper with a genuinely useful new dataset, but the headline gains may be inflated by same-admission drug history leaking the HF label; referee it, require a leak-control ablation.","tokens_in":743,"tokens_out":788,"would_cite":false,"duration_ms":42355,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that fusing laboratory tests, ECGs, and echocardiograms in one framework improves cardiac diagnosis and risk prediction, and it releases a patient-aligned multimodal cohort to test this.","keywords":["multimodal learning","multitask learning","heart failure diagnosis","risk stratification","electrocardiogram","echocardiography","laboratory tests","model explainability"],"falsifier":"Mask out every medication-history and medical-history token from the Labs text and retrain the model on the same split; if the trimodal AUC falls from 0.91 toward the 0.85 ECG+ECHO baseline, the complementarity claim is an artifact of label leakage.","tokens_in":24896,"feed_emoji":"🫀","tokens_out":6386,"duration_ms":58886,"temperature":0.7,"pith_summary":"The paper tries to establish that a single multimodal architecture can improve cardiac diagnosis and prognosis by fusing laboratory text, 12-lead ECGs, and echocardiogram video, rather than relying on one data type or on coarse feature concatenation. To test this, the authors build HFTri-MIMIC, a patient- and time-aligned cohort of 1,524 MIMIC-IV patients with all three modalities and follow-up for 455 patients. Their TGMM framework uses pretrained modality backbones, a flexible fusion module that gates both shared and modality-specific features, and task-specific text guidance. The core claim is that trimodal fusion raises heart-failure diagnosis to AUC 0.91 and risk-stratification C-index to 0.61, beating single-modality and prior-fusion baselines, and that the same design transfers to a myocardial infarction task. The reader should care because it proposes a reusable data resource and a concrete recipe for how complementary cardiac signals could be integrated in one system.","feed_headline":"Three data types beat single tests at heart failure diagnosis","feed_subtitle":"Lab values, ECG, and echocardiogram together reach 0.91 AUC for heart failure, beating every single-modality baseline.","key_machinery":"The load-bearing mechanism is the MedFlexFusion module: it computes query/key/value projections for each modality, lets one modality query all others, and combines the local fused features with a sigmoid global gate per modality, so the model can decide how much each source contributes. A textual guidance module adds semi-soft textual content, a task-specific human-written query concatenated with learnable tokens, encoded by a one-layer transformer, and uses that content to gate the concatenated specific-plus-shared features. The response module then turns prediction into a comparison of candidate answers with a small transformer decoder. The argument that this machinery works is carried by ablations showing that removing the learned text guidance or replacing the response module with a linear probe degrades performance.","core_discovery":"The central discovery, as the paper states it, is that trimodal integration of laboratory test values, ECGs, and echocardiograms improves heart failure diagnosis and risk prediction by up to 10% and 8%, respectively, over single modalities, with the full TGMM model reaching an AUC of 0.91 (95% CI 0.87–0.94), accuracy 0.86, and a C-index of 0.61 ± 0.05 on the HFTri-MIMIC cohort. The paper attributes this gain to a fusion design that takes complementarity seriously: the MedFlexFusion module gives local cross-modal attention plus a global gating signal, so shared and modality-specific features are both retained, and a textual guidance module filters those features toward the current task using a mix of human-written and learned prompts. It further claims that the same framework improves myocardial infarction diagnosis by up to 5% when labs and ECGs are fused, and that the trimodal model remains stable across missing-modality scenarios. Wilcoxon signed-rank tests are reported to support the significance of the modality-combination differences.","pith_inferences":["Beyond the paper, a direct next test is to run the same fusion on an independent multi-centre cohort with outcomes defined prospectively rather than by ICD codes, which would separate true fusion gains from coding artifacts.","Beyond the paper, the explanation figures suggest the model attends to shared Lab-ECHO features most; one testable extension is to ablate only the shared features and measure how much of the 0.91 AUC comes from cross-modal agreement rather than unique signal.","Beyond the paper, a deployment-oriented extension would calibrate the risk scores and set a decision threshold, since a C-index of 0.61 is unlikely to change clinical management without calibration."],"forward_implications":["If TGMM generalizes as reported, a single modular network can cover diagnosis and prognosis with whatever subset of labs, ECG, and echo is available, matching clinical settings where not every test is done for every patient.","The ECG+ECHO combination being weaker than other pairs suggests that merely adding modalities is not enough; the fusion mechanism must exploit complementary portions, so future systems should expect pair-specific gains.","Because the risk-stratification C-index (0.61) exceeds all compared baselines, the textual-guidance design is a viable direction for prognosis even if absolute predictive strength remains moderate.","On the external MI dataset, the same architecture makes lab+ECG diagnosis nearly match ECG-alone performance, implying ECG is the dominant signal for MI while labs add little."],"supporting_citations":[{"why":"Supplies the MIMIC-IV source records from which the curated HFTri-MIMIC cohort is built.","marker":"Johnson et al. (2024)"},{"why":"Provides the Bio-Clinical BERT encoder used to turn textualized lab results into modality-specific representations.","marker":"Alsentzer et al. (2019)"},{"why":"Provides ST-MEM, the spatio-temporal ECG foundation model used to encode 12-lead signals.","marker":"Na et al. (2024)"},{"why":"Provides EchoPrime, the multi-view echocardiogram foundation model used as the echo backbone and also as a baseline.","marker":"Vukadinovic et al. (2024)"},{"why":"Supplies the external myocardial infarction dataset used to validate transfer of the framework.","marker":"Wagner et al. (2020)"},{"why":"Provides the SHAP framework used to attribute predictions to text tokens in the explainability analysis.","marker":"Lundberg and Lee (2017)"},{"why":"Defines the concordance index used to evaluate risk-stratification performance.","marker":"Harrell et al. (1982)"},{"why":"Provides the negative partial likelihood loss used for the risk-prediction task.","marker":"Katzman et al. (2018)"}],"fun_headline_variants":["Trimodal fusion lifts heart failure AUC to 0.91","Labs, ECG, echo together improve cardiac diagnosis","Multimodal fusion outperforms single tests in heart care","TGMM fuses labs, ECG, echo for better heart diagnosis"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The heart-failure label for training comes from ICD codes, and the lab-text input includes the patient's medication and medical history from the same admission; the whole trimodal advantage assumes that this history does not already reveal which patients carry the heart-failure diagnosis.","fun_headline_variants_meta":{"raw":{"variants":["Trimodal fusion lifts heart failure AUC to 0.91","Labs, ECG, echo together improve cardiac diagnosis","Multimodal fusion outperforms single tests in heart care","TGMM fuses labs, ECG, echo for better heart diagnosis"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000517,"raw_usage":{"total_tokens":2561,"prompt_tokens":1053,"completion_tokens":1508,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":669,"completion_tokens_details":{"reasoning_tokens":1439}},"tokens_in":669,"tokens_out":1508,"duration_ms":10911,"temperature":1.0,"reasoning_tokens":1439,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:17:36.203298+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Mask out every medication-history and medical-history token from the Labs text and retrain the model on the same split; if the trimodal AUC falls from 0.91 toward the 0.85 ECG+ECHO baseline, the complementarity claim is an artifact of label leakage.","supporting_citations":[{"cited_title":", author Califf, R.M","cited_arxiv_id":null,"evidence_quote":"Defines the concordance index used to evaluate risk-stratification performance."}],"review_version":2}