{"id":"e0c4f8c5-290c-4604-8a00-adfc07159bc6","arxiv_id":"2411.09822","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A CLIP-style multimodal model with image-tabular matching achieves modest AUC gains over unimodal baselines for pre-stroke stroke risk prediction on a small UK Biobank test set.","lead":"This paper combines pre-stroke brain MRI scans and clinical records in a self-supervised contrastive model to predict future stroke risk in UK Biobank participants. The authors report small improvements over standard single-modality models, but the test set has only 93 patients and lacks external validation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Statistical basis for the headline gains is not established: Table 5's bootstrap at 0.52% prevalence on n=93 is internally inconsistent, and Table 4 lacks confidence intervals.","rationale":"The reader's conditional verdict is reasonable, but the weakest_assumption identified is not the most load-bearing threat to the central claim. The paper's headline is about outperforming baselines; that claim stands or falls on the statistical comparison. Table 4 gives no confidence intervals, and Table 5, intended to supply statistical support, fails a basic arithmetic check: n=93 at 0.52% prevalence gives an expected 0.48 stroke cases per bootstrap sample, incompatible with the reported stable sensitivity and F1 values. This is an internal inconsistency, not a matter of external consensus. The healthy-control definition could affect generalizability, but even with perfectly defined controls we would still not know whether the reported 2.1–2.8% AUC margin reflects signal or noise. Because the main numerical claim cannot currently be evaluated, the verdict should move from CONDITIONAL to UNVERDICTED: the paper may be correct, but the published evidence does not yet support a determination. The proposed check—reproducing Table 5 with the public repository and stated prevalence—would settle whether the concern lands. If the bootstrap protocol is clarified and the result survives, the reader's CONDITIONAL verdict would be appropriate again, pending external validation.","tokens_in":17080,"tokens_out":4583,"duration_ms":48617,"concrete_test":"Run the released code to reproduce Table 5: perform 1,000 bootstrap resamples of the 93-patient test set at the stated 0.52% UKB stroke prevalence and recompute sensitivity, F1, and AUC for the proposed model. If the number of stroke-positive cases per resample is typically zero or one, then the stable sensitivity and F1 values in Table 5 cannot be genuine, and the Wilcoxon significance claim is invalid. If the code reveals a different prevalence or resampling scheme that makes sensitivity computable, the concern is resolved and the remaining issue is external validation.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The central claim is a comparative performance advantage, so it depends on the significance evidence. Table 4 reports point estimates on a single 93-patient test set with no confidence intervals or repeated-seed variation. Table 5 is offered as significance evidence, but its protocol is internally inconsistent: with n=93 and a stated UKB stroke prevalence of 0.52%, the expected number of stroke cases per bootstrap sample is about 0.48. Under a binomial with p=0.0052 and n=93, sensitivity and F1 are undefined or extremely unstable for most resamples, yet Table 5 reports stable values such as sensitivity 0.846 [0.770–0.910] and F1 0.744 [0.723–0.765]. This can only happen if the bootstrap actually used a different prevalence, a much larger sample, or a rebalanced subset; the paper does not specify the resampling procedure or how paired predictions across models were constructed. Moreover, even if Table 5 is taken at face value, the AUC confidence intervals overlap substantially (Ours 0.748 [0.696–0.798]; CLIP 0.734 [0.680–0.787]; SCARF 0.727 [0.672–0.780]), so the reported Wilcoxon p-values < 0.01 are not consistent with the displayed intervals for the primary metric unless the paired test was applied to a different quantity. The result is that the claimed 2.1–2.8% AUC improvements and larger balanced-accuracy gains have not been shown to be statistically reliable. The healthy-control definition is a real but secondary generalizability concern; the immediate load-bearing issue is that the numerical superiority claim lacks a valid statistical foundation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a self-supervised multimodal framework that aligns 3D T2-FLAIR brain MRI and tabular clinical/imaging-derived features in a shared latent space using a CLIP-style contrastive loss together with an image-tabular matching (ITM) loss. The method is pre-trained on 5,000 UK Biobank samples and fine-tuned on 278 samples, then evaluated on a 93-patient hold-out set for pre-stroke risk prediction. The authors compare against supervised and self-supervised unimodal and multimodal baselines, reporting higher ROC-AUC and balanced accuracy for the proposed model, and support the result with interpretability analyses (UMAP embeddings, integrated gradients, and GradCAM heatmaps). The paper also includes a bootstrap-based significance analysis in Table 5 and a discussion of clinical plausibility.","tokens_in":17343,"tokens_out":6617,"duration_ms":59929,"significance":"If the reported gains are statistically reliable, the paper would provide a useful demonstration that self-supervised multimodal pretraining can combine imaging and tabular data for a low-prevalence clinical outcome such as pre-stroke risk prediction. The work has practical strengths: a released code repository, a reasonably comprehensive set of baselines, and interpretability analyses that connect model attention to white-matter hyperintensities, which are clinically relevant. However, the central comparative claim is not yet convincingly supported: the primary benchmarking table lacks confidence intervals, the bootstrap significance analysis is internally inconsistent with its own stated protocol, and the reported effect sizes in the abstract and conclusion do not match Table 4. The contribution is promising but needs substantial statistical and reporting revisions before the headline claims can be accepted.","major_comments":[{"comment":"The key comparison of the proposed model against baselines is presented as point estimates on a single 93-patient test set. No confidence intervals, standard errors, or repeated-seed results are reported. Given the small test set and the number of models compared, the claimed improvements (e.g., ROC-AUC 74.42 vs 73.82 for DAFT; balanced accuracy 71.11 vs 67.85 for SCARF T T) are not shown to be statistically reliable. The authors should report confidence intervals or resampling-based intervals for all models in Table 4, and should specify how many random seeds were used and how the Youden-index operating point selected on the validation set affects the reported test metrics.","section":"Section 4.1, Table 4"},{"comment":"The bootstrap procedure is underdescribed and internally inconsistent. With n=93 and the stated 0.52% stroke prevalence, each bootstrap resample would contain about 0.48 stroke patients, so sensitivity and F1 would be undefined for most resamples under binomial sampling; yet Table 5 reports tight intervals such as sensitivity 0.846 [0.770–0.910] and F1 0.744 [0.723–0.765]. The authors must specify the exact resampling protocol: whether whole patients are resampled, what prevalence is used, how undefined metrics are handled, and how paired predictions are constructed for the Wilcoxon signed-rank test. As written, Table 5 cannot serve as evidence of significance, and the overlapping AUC confidence intervals (Ours 0.748 [0.696–0.798]; CLIP 0.734 [0.680–0.787]; SCARF 0.727 [0.672–0.780]) further undermine the claimed p<0.01 for the primary metric.","section":"Section 2.2.2, Table 5"},{"comment":"The reported improvement magnitudes are inconsistent across the manuscript. The abstract states gains of 2.6% (2.6%) in ROC-AUC and 3.3% (5.6%) in balanced accuracy over self-supervised tabular (image) methods, while Section 5 states 2.1% (2.8%) in ROC-AUC and 10.6% (12.8%) in balanced accuracy, and 7.6% accuracy over the best multimodal supervised model. Neither set of numbers is fully reproducible from Table 4: for example, Ours T T has AUC 74.42 versus best SCARF 72.16 (difference 2.26) and best SimCLR 72.11 (difference 2.31), while balanced accuracy differences versus SCARF T T and SimCLR T T are 3.26 and 5.55, respectively. The authors should recompute and harmonize all reported effect sizes under a clearly stated comparison convention.","section":"Abstract, Section 5, Table 4"},{"comment":"The healthy control definition—participants who were still stroke-free at a second imaging visit—may introduce selection bias: controls who return for follow-up may differ from the general population in health-seeking behavior and unmeasured risk, and the model may partly learn to distinguish 'returned for follow-up' rather than stroke risk. This concern should be analyzed, for example by comparing the control group's risk-factor distribution with the broader UKB non-stroke population or by using all stroke-free participants as controls, or at least discussed explicitly as a limitation affecting generalization.","section":"Section 2.1.1, Table 1"}],"minor_comments":[{"comment":"The column header 'Acc' is used interchangeably with 'balanced accuracy' in the text; please use a single term consistently throughout the manuscript.","section":"Section 3.3, Table 4"},{"comment":"The Mann-Whitney U test on GradCAM activations is a single unadjusted comparison on 93 test scans; please report the effect size and clarify that this analysis is exploratory.","section":"Section 4.2.2, Figure 6"},{"comment":"Several references have formatting errors or incomplete bibliographic details (e.g., [63] contains a duplicate DOI/URL pattern and placeholder text); please check the reference list against the publisher's style.","section":"References"},{"comment":"The confidence intervals in Table 5 use inconsistent decimal formatting (F1 intervals have four digits while other metrics have three); please unify the formatting.","section":"Table 5"}],"recommendation":"major_revision","confidential_remarks":"The paper has a promising core idea and a reasonable experimental setup, but the statistical evidence for the headline claims is currently not solid. The bootstrap analysis in Table 5 appears internally inconsistent, and the effect sizes in the abstract and conclusion do not match Table 4. These issues are fixable with additional analysis and careful rewriting, so I recommend major revision rather than rejection. The healthy-control definition is a secondary concern but should be addressed for clinical credibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a genuinely new application — pre-stroke T2-FLAIR brain MRI plus tabular clinical records through a CLIP-style contrastive loss with ITM — on UK Biobank, and the authors compare it against a sensible set of unimodal and multimodal baselines. The method itself is mostly a reapplication of known pieces (CLIP, ALBEF/TIP, cross-attention), so novelty is in the domain and the setup, not the architecture. Credit where due: they ablate tabular feature sets, report frozen vs trainable, provide code, and the GradCAM/integrated-gradients analysis is a reasonable attempt to connect predictions to white matter hyperintensities. The study is clearly relevant for anyone working on multimodal medical ML.\n\nThe soft spot is the statistical foundation of the headline claim. Table 4 gives point estimates on a 93-patient test set with no confidence intervals. Table 5 is meant to fix that, but the protocol is internally inconsistent: with n=93 and 0.52% prevalence, the expected number of stroke cases per bootstrap resample is about 0.5, so sensitivity and F1 would be undefined or absurdly unstable for most resamples. The reported stable CIs and tight Wilcoxon p-values cannot come from that protocol as described. The paper needs to spell out exactly what was resampled. On top of that, the abstract reports different gains from the conclusion (2.6%/2.6% vs 2.1%/2.8% in AUC), which needs reconciliation. The healthy-control definition — stroke-free only at a second imaging visit — is a real selection-bias worry, though secondary. And there's no comparison to established clinical risk scores (Framingham, QRISK), which makes it hard to know whether the model adds clinical value.\n\nBottom line: the idea deserves attention, and the negative-control analyses (e.g., checking age as a proxy) are thoughtful, but the central \"outperforms\" claim is not demonstrated as written. If the bootstrap is corrected and the numbers made internally consistent, this could be a solid application paper. As-is, a serious referee should ask for major revision, not rejection out of hand.","headline":"New application, old building blocks: the multimodal setup and cohort work are real, but the statistical basis for the performance claims doesn't hold.","tokens_in":18008,"tokens_out":2328,"would_cite":false,"duration_ms":22272,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that jointly aligning brain MRI and clinical tabular encoders with a contrastive loss and an image-tabular matching loss predicts pre-stroke risk better than any uni-modal or supervised multimodal baseline on the UK…","keywords":["stroke risk prediction","self-supervised learning","contrastive learning","multimodal foundation model","brain MRI","T2-FLAIR","clinical tabular data","UK Biobank"],"falsifier":"Retrain the model with healthy controls sampled from stroke-free UK Biobank participants who did not return for a second imaging visit, matched for age and sex; if the ROC-AUC advantage over DAFT disappears or falls below significance, the central claim is confounded.","tokens_in":16819,"feed_emoji":"🧠","tokens_out":6568,"duration_ms":59161,"temperature":0.7,"pith_summary":"The paper's aim is to show that stroke risk can be predicted before onset by jointly learning from a patient's pre-stroke brain MRI and clinical records, without requiring annotated training data. It proposes a self-supervised framework that aligns image and tabular embeddings in a shared latent space using a contrastive objective plus an image-tabular matching task. On the UK Biobank test set, the combined model reaches a ROC-AUC of 74.42 and a balanced accuracy of 71.11, ahead of all uni-modal baselines and of the best supervised multimodal baseline (DAFT at 73.82). The authors present this as the first pre-onset stroke risk model built from multimodal clinical data, with heatmaps showing that the model focuses on white-matter hyperintensity regions tied to aging and stroke.","feed_headline":"Aligned MRI and health records beat single-modality stroke-risk models","feed_subtitle":"A CLIP-and-ITM aligned MRI/clinical model beats uni-modal and supervised baselines on UK Biobank.","key_machinery":"The load-bearing mechanism is the joint alignment of two encoders in a shared latent space: a CLIP-style contrastive loss defined between image and tabular projections, combined with an image-tabular matching (ITM) loss over hard negative pairs and a cross-attention transformer module that produces a joint [CLS] representation. The CLIP loss maximizes cosine similarity between matching image-tabular pairs while minimizing similarity to in-batch negatives, and the ITM loss adds a binary match/no-match prediction on mined hard negatives; together they force the two modalities to share a common patient representation. The shared space is what allows the downstream classifier to draw on both modalities, and the paper's UMAP analysis shows that uni-modal pretraining leaves image and tabular embeddings separated while the multimodal pretraining overlaps them.","core_discovery":"The central discovery is that aligning 3D T2-FLAIR brain MRIs with structured clinical data in a shared embedding space produces representations that transfer better to pre-stroke risk classification than either modality alone or than supervised fusion. The method pre-trains a ResNet-50 image encoder and an MLP tabular encoder with a CLIP-style contrastive loss that pulls matching image-tabular pairs together and pushes mismatched pairs apart, together with an image-tabular matching (ITM) loss that uses hard negative pairs and a cross-attention transformer to force genuine cross-modal interaction. After fine-tuning on 278 labeled examples, the model reaches ROC-AUC 74.42 and balanced accuracy 71.11 on a 93-patient test set, surpassing the best self-supervised tabular baseline by 2.1% in ROC-AUC and 10.6% in balanced accuracy, the best self-supervised image baseline by 2.8% and 12.8%, and the best supervised multimodal baseline (DAFT) by 7.6% in balanced accuracy. The authors report that the improvement over all self-supervised baselines is statistically significant when evaluated at the real 0.52% stroke prevalence with bootstrapping, and that removing the ITM loss lowers performance.","pith_inferences":["If the healthy-control definition (stroke-free at a second imaging visit) is replaced by stroke-free participants who did not return for follow-up, the model may lose part of its advantage, because the current controls could let the model learn follow-up attendance or health-seeking behavior rather than pure stroke risk.","The CLIP-plus-ITM recipe is modality-agnostic; the same shared-latent alignment with hard-negative matching could be transferred to other large biobanks that pair imaging with structured health records for different disease outcomes.","The finding that accuracy peaks for strokes occurring 2-3 years after the scan suggests a possible imaging-detectable prodromal window; a testable extension would stratify by time-to-onset in a larger cohort and check whether the signal remains stable.","The 93-patient test set is small, so a practical follow-up beyond ranking metrics would be to examine calibration and threshold choice at realistic prevalence before any clinical screening use."],"forward_implications":["If correct, pre-stroke imaging and routine clinical data contain complementary signals that a contrastively aligned model can exploit without expert annotations.","Self-supervised pretraining on unlabeled biobank-scale data can substitute for scarce pre-onset stroke labels; frozen representations nearly match trainable ones on the small fine-tuning set.","The ITM loss contributes measurable gains over pure CLIP alignment, implying that hard-negative cross-modal matching adds signal beyond simple embedding alignment.","The model's GradCAM activations concentrate in periventricular and deep white-matter hyperintensities, a pattern consistent with known vascular aging and stroke pathology, and could guide imaging biomarker discovery.","Evaluated at the real 0.52% stroke prevalence, the model's balance-accuracy lead over self-supervised baselines persists under bootstrap reweighting, supporting its potential as a screening aid rather than a diagnostic."],"supporting_citations":[{"why":"Supplies the CLIP contrastive loss that aligns image and tabular projections in the shared latent space.","marker":"[31]"},{"why":"Supplies the image-tabular matching (ITM) loss and hard-negative mining strategy.","marker":"[32]"},{"why":"Provides the tabular-image pretraining design and the transformer interaction module the authors adapt for multimodal encoding.","marker":"[28]"},{"why":"Defines SimCLR, the projection head and augmentation scheme used for the image encoder and as a uni-modal image baseline.","marker":"[22]"},{"why":"Defines SCARF, the random feature corruption method for tabular data and the principal self-supervised tabular baseline.","marker":"[26]"},{"why":"Defines DAFT, the best supervised multimodal baseline the proposed model is compared against.","marker":"[20]"},{"why":"Describes the UK Biobank imaging pipeline that produced the registered MRIs and image-derived phenotypes.","marker":"[41]"}],"fun_headline_variants":["Self-supervised MRI+clinical model beats single-modality stroke risk","Aligned brain MRI and health records improve pre-stroke risk prediction","Multimodal foundation model fuses imaging and tabular data for stroke risk","Self-supervised model aligns MRI and clinical data to predict stroke before onset"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is the definition of healthy controls as participants who were still stroke-free at a second imaging visit; if returning for follow-up is itself a marker of different health or health-seeking behavior, the reported gains could partly reflect that difference rather than stroke risk.","fun_headline_variants_meta":{"raw":{"variants":["Self-supervised MRI+clinical model beats single-modality stroke risk","Aligned brain MRI and health records improve pre-stroke risk prediction","Multimodal foundation model fuses imaging and tabular data for stroke risk","Self-supervised model aligns MRI and clinical data to predict stroke before onset"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000718,"raw_usage":{"total_tokens":3306,"prompt_tokens":1104,"completion_tokens":2202,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":720,"completion_tokens_details":{"reasoning_tokens":2124}},"tokens_in":720,"tokens_out":2202,"duration_ms":15161,"temperature":1.0,"reasoning_tokens":2124,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:17:02.516900+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the model with healthy controls sampled from stroke-free UK Biobank participants who did not return for a second imaging visit, matched for age and sex; if the ROC-AUC advantage over DAFT disappears or falls below significance, the central claim is confounded.","supporting_citations":[{"cited_title":"A simple framework for contrastive learning of visual representations","cited_arxiv_id":null,"evidence_quote":"Defines SimCLR, the projection head and augmentation scheme used for the image encoder and as a uni-modal image baseline."},{"cited_title":"DAFT: A universal module to interweave tabular data and 3D images in CNNs","cited_arxiv_id":null,"evidence_quote":"Defines DAFT, the best supervised multimodal baseline the proposed model is compared against."}],"review_version":1}