{"id":"de419fef-5ff5-471a-a41e-2c1bc0e19c03","arxiv_id":"2501.13277","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"MEDFORM combines SimCLR on CT slices with cross-modal contrastive alignment to clinical numeric data, reporting modest AUROC gains that are not consistently supported by the presented statistics.","lead":"This paper presents MEDFORM, a multimodal pretraining method that aligns CT imaging with clinical numeric data through contrastive learning. The authors report better cancer classification and few-shot performance across three tumor types, but unclear data splits and possible label leakage weaken the evidence.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Data split overlap in Section 4.1: the same 'rest' set is used for both multimodal alignment training and testing, so reported gains may reflect memorization rather than generalization.","rationale":"The reader's weakest assumption was label leakage through the clinical features. I agree that this is a serious risk, but the paper's own text in Section 4.1 suggests an even more direct and load-bearing flaw: the same data are described as being used for both training the multimodal alignment and as the test set. If this reading is correct, the evaluation protocol is invalid regardless of what the clinical columns contain, because the model has seen the test patients' CT slices and the corresponding clinical vectors during pre-training. This would make the reported AUROC/ACC measures of memorization, not generalization. The label-leakage concern is a special case of this broader evaluative failure: if the clinical vector contains the target label, the alignment step can encode the label into the CT representation, making the downstream probe trivially successful. But the split overlap is sufficient on its own to undercut the central claim. I also note that the 'consistently outperforms' statement is not supported by Table 1, where the feature-concatenation baseline exceeds the contrastive model on NSCLC accuracy, and most differences are within reported standard deviations. These issues reinforce the reader's rejection, so I do not change the verdict.","tokens_in":6737,"tokens_out":4138,"duration_ms":46029,"concrete_test":"Request the exact patient-level or slice-level identifiers used for (a) CT-SimCLR pre-training, (b) multimodal alignment training, and (c) evaluation in Section 4.1. Check for any overlap between sets (b) and (c). If overlap exists, rerun the full pipeline with a disjoint hold-out cohort from the same TCIA collections; if the contrastive model no longer outperforms the unimodal and feature-concatenation baselines, the central claim is not supported. As a secondary check, obtain the list of clinical variables used in the clinical encoder and verify that the target labels (T-stage, bilobar disease, histologic stage) are not among them.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.1 describes the data split for each cancer type with wording such as 'the rest ... was used in training multimodal alignment and as test set.' This is most naturally read as saying the same patients/slices are used both to train the cross-modal alignment and to serve as the test set. If that is the case, the linear probe in Section 4.2 evaluates on data whose labels were indirectly accessible during pre-training: the cross-modal contrastive loss pairs CT slices with clinical vectors, and if those clinical vectors encode the target label (or a proxy), the CT representation can learn to read the label directly. Even without label leakage, using the same data for alignment training and evaluation means the reported AUROC/ACC reflect memorization of the training set, not generalization. This is a more fundamental threat to the central claim than label leakage alone, and it is not addressed anywhere in the paper. Additionally, the clinical feature list is never provided, so the label-leakage mechanism remains a live secondary concern. The claim that the contrastive model 'consistently outperforms all baselines' is also contradicted by Table 1: for NSCLC histologic stage, the feature-concatenation model achieves ACC 0.8420 while the contrastive model achieves 0.8208, and most AUROC differences are within one standard deviation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MEDFORM, a multimodal pre-training framework that combines CT imaging with clinical numeric data for multi-cancer classification. The method has three components: a ResNet-50 slice encoder trained with SimCLR, an attention-based multiple instance learning (ABMIL) module that aggregates slice embeddings into a volumetric CT representation, and a cross-modal contrastive learning objective (symmetric CLIP-style loss) that aligns CT representations with representations of clinical numeric features encoded by an MLP. The authors evaluate on three cancer datasets from TCIA: breast cancer T-stage, colorectal cancer bilobar disease, and NSCLC histologic stage, comparing unimodal CT, feature concatenation, and their contrastive model. They also report few-shot learning results. The central claim is that the dual pre-training strategy 'consistently outperforms all baselines' (Section 4.2) and 'maintains robust performance in few-shot learning scenarios' (Abstract).","tokens_in":6961,"tokens_out":3802,"duration_ms":45067,"significance":"If the claims were valid, MEDFORM would be a useful contribution to multimodal medical foundation models for CT and tabular clinical data, and the use of MIL for volumetric CT aggregation is a reasonable design choice. The paper makes code available and uses public TCIA data, which are positive features. However, the reported experimental evaluation is not reliable: the same data are used for multimodal alignment training and for testing, the clinical variables are undisclosed so label leakage cannot be ruled out, and the paper's own Table 1 contradicts the 'consistently outperforms' claim. As a result, the current evidence does not support the central claims, and the significance of the contribution cannot be assessed from the reported results.","major_comments":[{"comment":"The data split is not a held-out split. For each cancer type, the text states that 'the rest ... was used in training multimodal alignment and as test set' (NSCLC, breast, and colorectal descriptions). This means the same patients/slices used to train the cross-modal alignment are also used to evaluate the downstream linear probe in Section 4.2. Any reported AUROC/ACC can therefore reflect memorization of the training data rather than generalization. This is a load-bearing flaw: without a proper train/validation/test split, none of the performance numbers in Tables 1 and 2 support the paper's claims.","section":"Section 4.1"},{"comment":"The paper never lists the clinical variables used as input to the clinical encoder. The downstream tasks are T-stage, bilobar disease, and histologic stage. If any of these target labels (or direct proxies, such as stage or grade) appear as a column in the clinical data, the cross-modal contrastive objective lets the CT embedding learn to encode the label from the paired clinical vector. The downstream linear probe could then read the label from the CT representation, making the multimodal gain circular. The authors must disclose the full clinical feature list and explicitly exclude label columns, or demonstrate that no such leakage is possible.","section":"Sections 3.2 and 4.1"},{"comment":"The claim that the contrastive model 'consistently outperforms all baselines' is contradicted by the reported numbers. For NSCLC histologic stage, the feature concatenation model achieves ACC 0.8420 while the contrastive model achieves ACC 0.8208. Moreover, most AUROC differences in Table 1 are within one standard deviation, and the colorectal unimodal AUROC of 0.4825 is below the chance level of 0.5. The paper provides no statistical significance testing. These inconsistencies undermine the central claim even before considering the split and leakage issues.","section":"Table 1 and Section 4.2"},{"comment":"The few-shot experiments report only the proposed contrastive model; there are no unimodal or feature-concatenation baselines for the same few-shot protocol, and the evaluation again uses the overlapping data split. Without a comparison baseline and a proper held-out split, the statement that the model 'maintains robust performance in few-shot learning scenarios' is not supported.","section":"Table 2"}],"minor_comments":[{"comment":"The number of colorectal CT slices used for CT-SimCLR pre-training is reported as 10,393 in the Abstract and Section 4.1 but as 10,323 in Section 3.1; these should be reconciled.","section":"Abstract and Section 3.1"},{"comment":"Equation (1) is not typeset correctly: the temperature τ appears inside the exponent without a clear definition of normalized cosine similarity. The authors should specify that embeddings are L2-normalized and define the temperature parameter explicitly.","section":"Equation (1)"},{"comment":"There are numerous typographical errors, including 'CT slide' instead of 'CT slice' in Section 3.3 and grammatical issues in Sections 4.1 and 5. A careful language edit is needed.","section":"Throughout"},{"comment":"The few-shot table does not report standard deviations for the k=1 breast cancer AUROC or provide a baseline such as training a linear probe on the unimodal CT representation; adding these would help interpret the results.","section":"Table 2"},{"comment":"The description of the CT-SimCLR pre-training dataset is inconsistent: the paper says '141,171 NSCLC slices, 8,100 breast cancer slices, and 10,323 colorectal cancer images' in Section 3.1, but Section 4.1 reports different cohort totals. The exact composition of the pre-training set should be clarified.","section":"Section 3.1"}],"recommendation":"reject","confidential_remarks":"The paper has a potentially interesting idea, but the experimental evaluation is fundamentally invalid due to the train/test overlap and undisclosed clinical features, and the reported results contradict the central claim. I would consider a resubmission only if the authors rerun all experiments with a proper held-out test set, disclose the full clinical feature list and exclude labels, and provide statistical significance testing. As it stands, the manuscript does not meet the bar for publication in a serious journal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a straightforward adaptation of TANGLE to CT slices plus clinical numeric data, not a new mechanism. The paper is worth a look because the problem is real — using tabular clinical data as weak supervision for CT representation learning is a sensible idea — and the authors put together multi-cancer data and released code. But the experimental section, as written, does not support the headline claim that MEDFORM “consistently outperforms all baselines.”\n\nWhat’s good: the MIL treatment of volumetric CT is appropriate, the combination of SimCLR pretraining with cross-modal contrastive alignment is clearly described, and the choice to benchmark against feature concatenation is the right baseline. The code availability is a plus.\n\nWhere it falls down. First, the data split description in Section 4.1 strongly suggests that the same set of patients/slices was used both for multimodal alignment training and for testing. If so, the linear probe evaluates on data whose representations were aligned during pretraining, which can inflate results through memorization even without label leakage. That is a load-bearing flaw. Second, the clinical feature list is never given, so the possibility that the target label (T-stage, histologic stage, bilobar disease) appears among the input columns remains live. That would make the multimodal gains circular. Third, Table 1 itself contradicts the “consistently outperforms” claim: on NSCLC histologic stage, feature concatenation beats contrastive learning on accuracy (0.8420 vs 0.8208), and most AUROC differences are within one standard deviation. The zero standard deviation for that baseline is also a red flag. Fourth, Table 2 is labeled as the authors’ model but the model column says TANGLE, which suggests a copy-paste error and leaves the few-shot results ambiguous.\n\nIn short, the core idea is plausible but the evidence is not. The paper needs a corrected train/test split, a full disclosure of the clinical variables, and a more honest reading of Table 1. The novelty is limited, but the flaws, not the novelty, are what sink the current version.\n\nWho this is for: someone working on multimodal medical imaging pre-training, especially those considering clinical tabular data as a supervision source. They might learn from the method’s design and from the pitfalls this paper illustrates.\n\nRecommendation for peer review: send it out. The methodological concerns are serious but could be addressed in revision, and the paper touches a question worth refereeing. My own verdict would be reject in the present form, but with an invitation to resubmit if the data split and variable disclosure are fixed.","headline":"Adapts TANGLE to CT + clinical data, but the reported evidence is undercut by a likely train/test overlap and undisclosed clinical features.","tokens_in":7525,"tokens_out":2194,"would_cite":false,"duration_ms":23733,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-stage contrastive pretraining scheme can align CT scans with clinical numeric data, improving cancer classification and few-shot performance across breast, colorectal, and lung cancer tasks.","keywords":["medical foundation model","computed tomography","contrastive learning","multiple instance learning","cross-modal alignment","cancer classification","few-shot learning","clinical tabular data"],"falsifier":"Audit the clinical feature columns used in the alignment experiment and remove any column that is the target label or a deterministic transform of it; if the AUROC gap over the unimodal CT baseline collapses, the reported advantage rests on label information in the clinical input rather than on cross-modal alignment.","tokens_in":6516,"feed_emoji":"🩻","tokens_out":7341,"duration_ms":69614,"temperature":0.7,"pith_summary":"MEDFORM tests a two-stage recipe for building a medical foundation model from CT images and tabular clinical records without expert annotations. First, a CT slice encoder is pretrained with self-supervised contrastive learning on unlabeled slices; then the slice embeddings are pooled by an attention-based multiple-instance-learning module into a patient-level representation, and that representation is aligned with a clinical-data embedding through a symmetric cross-modal contrastive loss. The paper reports that this aligned representation outperforms a unimodal CT encoder and a feature-concatenation baseline across three cancer-specific tasks, and that the gains survive with only a handful of labeled examples. If the result holds, large archives of paired CT scans and routine clinical records could serve as cheap supervision for label-efficient cancer classifiers.","feed_headline":"Dual pretraining aligns CT and clinical data for sharper cancer staging","feed_subtitle":"Contrastive CT–clinical alignment beats both baselines on three cancer tasks, even with 5–10 labels.","key_machinery":"The load-bearing object is the aligned embedding space produced by the dual pretraining schedule. Stage one trains a ResNet-50 slice encoder with SimCLR-style contrastive learning on 2D axial CT slices, using circular masking and Hounsfield-unit windowing as preprocessing. Stage two computes a patient-level CT vector by attention-based multiple-instance learning pooling over the slice embeddings, then minimizes a symmetric cross-modal contrastive loss against an MLP embedding of clinical numeric features, so matched CT-clinical pairs have high cosine similarity in a shared space. At inference, only the CT side is used for downstream linear-probe classification; the clinical encoder is present to shape the representation.","core_discovery":"The central claim is that guiding CT image representation learning with clinical numeric data, rather than simply concatenating the two modalities at decision time, produces a shared embedding with more usable signal for cancer classification. Using the dual pretraining strategy, MEDFORM reports higher AUROC than both baselines on all three tasks, with breast T-stage rising from 0.6390 to 0.7042, colorectal bilobar disease from 0.4825 to 0.5336, and lung histologic stage from 0.6000 to 0.6658. The paper interprets these results as showing that contrastive cross-modal alignment integrates imaging and clinical information more effectively than feature concatenation, and that the learned CT representation retains useful structure when only a few labeled examples are available.","pith_inferences":["The paper never lists the clinical variables entering the encoder, so the strongest check on the result is to audit those columns; if T-stage, bilobar status, or histologic stage appear as inputs, the contrastive alignment could simply copy the label into the CT embedding and the reported multimodal gain would be circular.","The experiments are restricted to the three cancers used in pretraining, so the phrase foundation model should be read narrowly; a genuine transfer test on an unseen cancer type or an unseen task remains an open extension.","A direct ablation that removes all outcome-related clinical columns and reruns the Table 1 comparisons would separate genuine multimodal signal from label leakage; the authors do not report such an ablation."],"forward_implications":["Because the CT side alone is used for downstream classification, the pretraining promises a label-efficient pipeline: a hospital with CT archives and matching clinical records could align representations once, then reuse them for several downstream tasks.","The attention-based multiple-instance aggregation removes the need for slice-level annotations, so the same encoder can consume volumetric CTs of arbitrary slice count.","The reported few-shot numbers imply that with 5 to 10 labeled examples per task, the aligned representation still gives usable accuracy, which could reduce expert annotation cost for new cancer staging tasks.","The gap between contrastive alignment and feature concatenation suggests that learning a shared space changes the representation itself, not just the input to the final classifier."],"supporting_citations":[{"why":"Supplies the SimCLR instance-discrimination objective used to pretrain the CT slice encoder from unlabeled slices.","marker":"[1]"},{"why":"Defines the multiple-instance learning bag assumption that lets a patient be treated as a bag of CT slices without slice-level labels.","marker":"[10]"},{"why":"Provides the attention-based MIL pooling that aggregates slice embeddings into a patient-level CT representation.","marker":"[11]"},{"why":"Contributes the transcriptomics-guided slide representation strategy that the paper adapts for aligning CT volumes with clinical numeric data.","marker":"[22]"},{"why":"Supplies the symmetric cross-modal contrastive objective that aligns CT and clinical embeddings in a shared space.","marker":"[23]"},{"why":"Provides the TabNet-based clinical encoder used in the feature-concatenation baseline that MEDFORM is compared against.","marker":"[3]"}],"fun_headline_variants":["Contrastive CT-clinical pretraining improves multi-cancer staging","Aligning CT with clinical data ups cancer classification AUROC","MEDFORM: contrastive CT-clinical learning sharpens cancer staging","Dual pretraining on CT and clinical data aids few-shot cancer"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The computer's clinical input contains no hidden copy of the exact answer, such as tumor stage or bilobar status, that it is later asked to predict.","fun_headline_variants_meta":{"raw":{"variants":["Contrastive CT-clinical pretraining improves multi-cancer staging","Aligning CT with clinical data ups cancer classification AUROC","MEDFORM: contrastive CT-clinical learning sharpens cancer staging","Dual pretraining on CT and clinical data aids few-shot cancer"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000687,"raw_usage":{"total_tokens":3089,"prompt_tokens":894,"completion_tokens":2195,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":510,"completion_tokens_details":{"reasoning_tokens":2123}},"tokens_in":510,"tokens_out":2195,"duration_ms":14831,"temperature":1.0,"reasoning_tokens":2123,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T16:17:57.012309+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Audit the clinical feature columns used in the alignment experiment and remove any column that is the target label or a deterministic transform of it; if the AUROC gap over the unimodal CT baseline collapses, the reported advantage rests on label information in the clinical input rather than on cross-modal alignment.","supporting_citations":[{"cited_title":"A simple framework for contrastive learning of visual representations","cited_arxiv_id":null,"evidence_quote":"Supplies the SimCLR instance-discrimination objective used to pretrain the CT slice encoder from unlabeled slices."},{"cited_title":"Lathrop, and T","cited_arxiv_id":null,"evidence_quote":"Defines the multiple-instance learning bag assumption that lets a patient be treated as a bag of CT slices without slice-level labels."},{"cited_title":"Tomczak, and M","cited_arxiv_id":null,"evidence_quote":"Provides the attention-based MIL pooling that aggregates slice embeddings into a patient-level CT representation."},{"cited_title":"Transcriptomics-guided slide representation learning in computational pathology","cited_arxiv_id":null,"evidence_quote":"Contributes the transcriptomics-guided slide representation strategy that the paper adapts for aligning CT volumes with clinical numeric data."},{"cited_title":"Learning transferable visual models from natural language supervision","cited_arxiv_id":null,"evidence_quote":"Supplies the symmetric cross-modal contrastive objective that aligns CT and clinical embeddings in a shared space."}],"review_version":1}