{"id":"971cebd1-fb33-4104-9066-ffaec7754c1c","arxiv_id":"2501.02778","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"ICFNet fuses histology images, genomics, demographics, and treatment text with optimal-transport attention and reports higher C-index than prior methods on five TCGA cancer datasets.","lead":"This paper proposes a neural network called ICFNet that combines four types of patient data: tissue images, gene expression, demographics, and treatment plans, to predict cancer survival. A generalist might read it because adding routine clinical variables to image-based models appears to improve survival predictions on five public cancer datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The treatment variable is actual treatment received, so the §4.5 counterfactual claim is confounded and part of the reported C-index gain may reflect treatment-by-indication rather than fusion quality.","rationale":"The paper's central claim has two parts: (i) ICFNet improves C-index by fusing additional modalities, and (ii) it can support treatment decisions by allowing clinicians to vary treatment input. Part (ii) is explicitly argued in Section 4.5 and the conclusion, and part (i) is what makes (ii) plausible. Both depend on what the treatment input means. If it is actual treatment received, then any learned association between treatment text/tensor and survival is observational and confounded by indication. The SOTA gain from adding such a variable is not proof of better multimodal fusion or better treatment understanding; it may simply be access to a strong prognostic correlate. The reader's weakest_assumption identifies exactly this. No formal verification, calibration, or causal-inference analysis is present, and the ablation in Table 4 shows treatment input helps, which is consistent with either a real treatment effect or an artifact of confounding. The proposed permutation test would settle whether the treatment branch is using treatment identity or just outcome-correlated assignment; the propensity-overlap analysis would settle the counterfactual reading of Fig. 6. Therefore the paper should be accepted only conditionally on addressing this distinction; the empirical architecture and ablations are otherwise a reasonable extension of MOTCat.","tokens_in":15533,"tokens_out":8354,"duration_ms":86827,"concrete_test":"Use the released code to first confirm what is actually encoded in xtreat_text and xt_tensor (e.g., whether the prompt is literally 'Treatments is/are applied.' or contains radiation/pharmaceutical labels). Then run a permutation check on each of the five fold splits: retrain ICFNet with the treatment tensor/text randomly shuffled across patients while keeping WSI, genomics, and demographics fixed, and compare C-index to both the no-treatment model and the true-treatment model. If permuted-treatment performance stays close to true-treatment performance, the reported treatment contribution is not causal information. To test the §4.5 counterfactual, restrict Fig.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing assumption is that treatment can be treated as a controllable input whose effect can be read off by changing the text prompt (§4.5). Section 3.1 defines treatment as 'whether radiation or pharmaceutical therapy was applied to the patients,' i.e., the treatment actually received. On observational TCGA data, treatment assignment is not randomized; it is driven by disease severity, stage, comorbidities, and prognosis. A model trained on actual treatment can therefore increase C-index by learning the selection mechanism (e.g., sicker patients receive more aggressive therapy) rather than a generalizable treatment–outcome relation. The Fig. 6 analysis in §4.5 changes the treatment input for an already-treated patient and interprets risk differences as prognostic consequences of treatment choice, but the model has never observed the same patient under an alternative treatment; this is a counterfactual extrapolation without propensity overlap or causal adjustment. Consequently, the clinical decision-support promise in the abstract and conclusion is not supported by the experiments. The benchmark claim is also affected: compared with MOTCat, which does not use treatment, ICFNet has access to a post-diagnosis variable that is correlated with outcome, so the 5.29% average gain conflates added information with better fusion. The paper's limitation section does not mention this identifiability issue; a propensity-stratified or randomized evaluation would be needed to separate these.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ICFNet, a multimodal survival prediction network that combines histopathology whole-slide images, genomic expression profiles, patient demographics, and treatment information. WSIs are encoded with a pretrained ResNet, genomics with six self-normalizing networks, and demographic/treatment data with a CLIP text encoder plus an MLP on a tensorized array. Cross-modal interaction uses optimal-transport-based co-attention, followed by a residual orthogonal decomposition (ROD) module and a unification fusion module. Training uses a dense supervision scheme and a proposed Balanced Negative Log-Likelihood Loss (BNLLLoss). The method is evaluated with C-index on five TCGA datasets (BLCA, BRCA, GBMLGG, LUAD, UCEC) under 5-fold cross-validation, reporting an average C-index gain of 5.29% over MOTCat, along with ablations, Kaplan-Meier curves, GradCAM visualizations, and a discussion of clinical decision support. The code is publicly available.","tokens_in":15751,"tokens_out":4545,"duration_ms":47105,"significance":"If the reported results are robust, the paper makes a useful empirical contribution: it shows that adding cheap clinical text (demographics and treatment received) to histology and genomics can improve prognostic C-index on several TCGA cohorts. The manuscript includes open code, external validation on five datasets, ablation studies, and KM-curve analyses, all of which support the internal consistency of the experimental pipeline. However, the clinical decision-support claim in Section 4.5 requires causal interpretation of treatment, which the observational study design does not support. The benchmark comparison also conflates added input information with better fusion, because the treatment variable is post-diagnosis and correlated with outcome via treatment-by-indication. The BNLL loss derivation in Section 3.3 is internally inconsistent with Eq. 10, and the SOTA comparison in Table 2 mixes reproduced and quoted baselines. These issues do not necessarily invalidate the predictive accuracy claim, but they do affect the strength of the paper's central conclusions.","major_comments":[{"comment":"The clinical decision-support claim is not supported by the experimental design. Section 3.1 defines treatment as 'whether radiation or pharmaceutical therapy was applied to the patients,' i.e., the treatment actually received, which on observational TCGA data is confounded with disease severity, stage, and prognosis. The model is trained on actual treatment, so it can exploit treatment-by-indication (e.g., sicker patients receiving more aggressive therapy) to improve C-index. Changing the treatment text in Fig. 6 for an already-treated patient is a counterfactual extrapolation without propensity overlap or causal adjustment; the model has never observed the same patient under alternative treatments. The abstract and conclusion therefore overstate the decision-support value. The authors should either remove the causal claims, reframe them as descriptive associations, or provide a propensity-stratified or otherwise causally adjusted evaluation.","section":"Section 4.5, Fig. 6; Section 3.1"},{"comment":"The motivation and derivation of BNLLLoss are internally inconsistent. Eq. (10) defines Ssurv,i(yi) as a product of (1 - hazard) terms, so Ssurv decreases as yi increases, not 'disproportionately larger for higher values of yi' as claimed in the text. The standard NLL in Eq. (9) is larger for larger yi because it sums more negative log-survival terms, which is the actual imbalance being addressed. However, the paper does not show that the added factor (1 - Shaz,i(yi))^{Nb-1-yi} in Eq. (11) equalizes per-class weights; no derivation or class-weight analysis is provided, and the exponent depends on the bin index in a way that does not obviously yield constant per-bin loss. Since BNLL is a stated contribution and is supported by an ablation row in Table 3, the authors need to provide a correct derivation or an empirical demonstration of the balancing effect.","section":"Section 3.3, Eq. (10)-(11)"},{"comment":"The state-of-the-art comparison mixes reproduced baselines (marked with '*') with numbers quoted from prior papers. If the quoted baselines were evaluated under different dataset splits, patch extraction protocols, or preprocessing steps, the reported 5.29% average gain over MOTCat is not a controlled comparison. In addition, MOTCat+Text is a simple concatenation baseline, and ICFNet has access to treatment information that is post-diagnosis and outcome-correlated; therefore the gain over MOTCat conflates added input information with superior fusion. The authors should either reproduce all baselines under the same protocol, clearly state which numbers are quoted and from which papers, and add a controlled comparison that isolates the fusion mechanism from the added treatment variable.","section":"Table 2, Section 4.3"}],"minor_comments":[{"comment":"The text says 'α and β are hyper-parameters which are set as 0.1', but Eq. (12) contains only α; β is never defined or used. Please clarify.","section":"Eq. (12)"},{"comment":"There is a formatting error in the BRCA column of one ablation row: '0.722±0.0.007' should likely be '0.722±0.007'.","section":"Table 3"},{"comment":"The introduction and contribution list refer to 'three modalities' of data, but the method actually uses four types of input: WSIs, genomics, demographics, and treatment text/tensor. Please make the terminology consistent.","section":"Section 1 and Section 3.1"},{"comment":"The treatment text template 'Treatments is/are applied.' is vague; it is unclear whether it captures only the binary presence of any treatment or distinguishes radiation from pharmaceutical therapy, and whether dose or regimen details are encoded. This matters for interpreting Fig. 6, where 'various treatment methods' are discussed.","section":"Section 3.1, treatment template"},{"comment":"Reference [5] is cited as 'CLIP' in Section 3.1, but the reference is Dosovitskiy et al., 'An image is worth 16x16 words' (ViT), not CLIP. Please correct the citation or the reference list.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The main risk to the paper's central claim is the treatment confounding issue: the reported SOTA gain may partly reflect the model's access to a post-diagnosis, outcome-correlated variable rather than improved fusion. If the authors cannot provide a propensity-stratified analysis or a controlled comparison, the clinical decision-support claims should be removed or substantially weakened. The BNLL derivation issue and the mixed-baseline comparison are also important but appear fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a reasonable extension of MOTCat: it adds demographic and treatment text to the usual histology-plus-genomics inputs, throws in a redundancy-reduction module (ROD), and reweights the NLL loss. The empirical work is real—five TCGA datasets, ablations, KM curves, and code. The headline result, a 5.29% average C-index gain over MOTCat, is a new benchmark that deserves attention, even if it is an incremental contribution.\n\nThe strongest part is the ablation study. The incremental gains from adding text, OT attention, dense supervision, ROD, and the unified fusion module are visible and make the design choices legible. The KM curves with lower p-values are also a nice sanity check. If I were building a multimodal survival model, I would study this paper's architecture.\n\nThe soft spots are material but fixable. The biggest one is the treatment variable. Section 3.1 defines treatment as \"whether radiation or pharmaceutical therapy was applied,\" i.e., actual treatment received. Section 4.5 then says clinicians can \"adjust the text input for ICFNet\" to compare prognoses under different treatments. That is a counterfactual claim the data cannot support. Treatment assignment on TCGA is observational and confounded with severity, stage, and other prognostic factors. The model may be learning treatment-by-indication rather than a treatment effect. This also taints the comparison with MOTCat: ICFNet has access to a post-diagnosis variable correlated with outcome, so part of the gain may reflect extra information rather than better fusion. The paper's limitation section does not mention this identifiability problem.\n\nThe BNLL loss is another concern. The paper claims it balances weights across classes, but the formula multiplies survival terms by (1 - hazard)^(Nb-1-yi). Since survival is a decreasing product of (1 - hazard) terms, this factor makes shorter-survival patients contribute less, not more. No derivation is given for the \"same weights\" claim. This looks wrong, though it might be a typo. The comparison table also mixes reproduced baselines with numbers quoted from prior papers, and there is a typo in Table 3 (0.722±0.0.007). These are secondary issues but need cleanup.\n\nThe paper is for researchers in computational pathology and prognostic modeling. It is a competent engineering contribution, not a paradigm shift. I would send it to peer review, but I would insist the authors either drop the causal language in §4.5 or add a proper propensity-stratified analysis, and either fix or properly motivate the BNLL loss. The benchmark comparison should also be run under the same split for all baselines.\n\nOverall: worth engaging with, but needs revision before I would trust the clinical claims.","headline":"A competent four-modality extension of MOTCat with plausible C-index gains, but the causal reading of treatment effects in §4.5 is not supported and the proposed BNLL loss has a suspicious balancing argument.","tokens_in":16295,"tokens_out":1929,"would_cite":true,"duration_ms":20687,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes ICFNet, a multimodal network that combines histopathology slides, genomic profiles, demographics, and treatment text to predict cancer patient survival, reporting an average 5.29% C-index improvement over the previous…","keywords":["multi-modal fusion","survival prediction","whole slide images","genomics","optimal transport","multiple instance learning","clinical text","TCGA"],"falsifier":"Retrain ICFNet with treatment labels randomly permuted across patients while keeping all other inputs identical; if the C-index gain over the no-treatment model persists under permutation, the reported improvement cannot be attributed to the treatment text's content. Alternatively, flip the treatment prompt for each patient and check whether the risk-score change agrees with outcomes in patient subgroups matched on stage and genomic profile.","tokens_in":15301,"feed_emoji":"🧬","tokens_out":5431,"duration_ms":51663,"temperature":0.7,"pith_summary":"The paper tries to establish that survival prediction improves when patient demographics and treatment protocols are added to the histopathology and genomic data that current models use. On five TCGA cancer cohorts, the proposed ICFNet network reports new state-of-the-art concordance indices, with an average 5.29% C-index gain over MOTCat, the strongest prior model. The gain is argued to come from cross-modal fusion: optimal-transport co-attention links image patches to genomic and text features, while a residual orthogonal decomposition removes redundant cross-modal information and a balanced loss treats short- and long-survival patients fairly. If the claim holds, adding two cheap clinical text fields to existing histology-plus-genomics pipelines is a robust benchmark improvement.","feed_headline":"Survival prediction gains 5.29% by adding patient text","feed_subtitle":"ICFNet fuses slides, genomics, demographics and treatment text to improve prognosis on five cancer datasets.","key_machinery":"The machinery is a modular fusion stack. A ResNet50 encodes WSI patches, six self-normalizing networks encode gene groups, and a CLIP-based text encoder plus MLP encode demographics and treatment. An optimal-transport co-attention module computes a matching flow between image features and genomic/text features, preserving global structure while aligning modalities. A residual orthogonal decomposition (ROD) projects patch features orthogonal to the cross-modal features to strip redundancy, a unification fusion module maps all five features into a shared latent space, and a balanced negative log-likelihood loss reweights survival bins so short-survival patients are trained as much as long-survival ones.","core_discovery":"ICFNet's central claim is that prognosis is better modeled by four modalities than by two. The authors report that on BLCA, BRCA, GBMLGG, LUAD, and UCEC, ICFNet achieves C-indices of 0.709, 0.724, 0.854, 0.697, and 0.702 respectively, with an overall 0.737, outperforming MOTCat by 5.29% on average. The model encodes whole slide images with a ResNet50, genomics with six self-normalizing networks, and demographics/treatment with a CLIP-based text encoder plus an MLP, then fuses them through optimal-transport co-attention. The paper also shows that changing only the treatment text changes the predicted risk, which the authors present as a step toward using the model to compare treatment options in clinical decision-making.","pith_inferences":["The paper's clinical counterfactual reading—that editing the treatment prompt reveals the effect of a treatment—is not supported by the training setup, because treatment is the treatment the patient actually received, and its assignment is likely confounded with prognosis. The reported gains may partly reflect this coupling.","One testable extension is to use the same architecture with free-text clinical notes instead of the short template prompts; if the gains are driven by treatment detail, richer notes should improve C-index further.","The uneven gains across cancers (1.43% on GBMLGG vs 11.40% on BRCA) suggest the value of text is disease-dependent; a cohort-level analysis of which clinical variables matter could guide where multimodal fusion pays off.","The ROD orthogonalization is a general recipe: explicitly enforcing that modality-specific features are orthogonal to cross-modal summaries may help any multi-modal medical model that suffers from redundancy in small-sample regimes."],"forward_implications":["Adding demographic and treatment text to histology-plus-genomics models yields consistent C-index gains on all five cohorts, with the largest gains on BRCA (11.40%) and BLCA (6.48%).","Because treatment is encoded as editable text, the model offers a way to compare treatment options by changing the prompt, which the paper presents as a clinical decision-support tool.","The balanced loss ensures patients with short survival times are not underweighted during training, which should make risk estimates more reliable for high-risk patients.","The text-only experiments show that demographics and treatment alone can already drive prediction, especially for breast cancer, so the model's gain is not purely from image or genomic features.","The modular design means new modalities (e.g., radiology or lab values) can be added with a new encoder and an OT interaction branch."],"supporting_citations":[{"why":"Provides the MOTCat baseline that ICFNet reproduces and reports beating by 5.29% average C-index, and supplies the optimal-transport co-attention idea.","marker":"[10]"},{"why":"The earlier WSI-genomics co-attention fusion method whose setup ICFNet extends with text and demographics.","marker":"[9]"},{"why":"Supplies the self-normalizing neural network used as the genomic encoder for the six gene groups.","marker":"[6]"},{"why":"Supplies the foreground segmentation and multi-instance patch-extraction pipeline, and provides CLAM as a baseline survival method.","marker":"[3]"},{"why":"Supplies the ImageNet-pretrained ResNet50 that encodes whole slide image patches.","marker":"[44]"},{"why":"Supplies the CLIP-based text encoder with an adapter used to encode demographic and treatment prompts.","marker":"[43]"},{"why":"The negative log-likelihood survival loss that the paper modifies into a balanced version to handle label imbalance.","marker":"[11]"}],"fun_headline_variants":["ICFNet fuses four data types to sharpen cancer prognosis","Cross-modal fusion improves survival prediction by 5.29%","ICFNet beats state-of-the-art on five cancer datasets","Treatment text alters predicted risk in ICFNet"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The treatment information in the input is the treatment each patient actually received, and the model treats that text as a knob that can be turned in hypothetical scenarios; if treatment choice is entangled with how sick a patient already is, the model's response to a changed prompt may not measure the treatment's true effect.","fun_headline_variants_meta":{"raw":{"variants":["ICFNet fuses four data types to sharpen cancer prognosis","Cross-modal fusion improves survival prediction by 5.29%","ICFNet beats state-of-the-art on five cancer datasets","Treatment text alters predicted risk in ICFNet"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0008,"raw_usage":{"total_tokens":3487,"prompt_tokens":886,"completion_tokens":2601,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":502,"completion_tokens_details":{"reasoning_tokens":2533}},"tokens_in":502,"tokens_out":2601,"duration_ms":19791,"temperature":1.0,"reasoning_tokens":2533,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:04:55.248168+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain ICFNet with treatment labels randomly permuted across patients while keeping all other inputs identical; if the C-index gain over the no-treatment model persists under permutation, the reported improvement cannot be attributed to the treatment text's content. Alternatively, flip the treatment prompt for each patient and check whether the risk-score change agrees with outcomes in patient subgroups matched on stage and genomic profile.","supporting_citations":[{"cited_title":"Multimodal optimal transport-based co-attention transformer with global structure consistency for survival prediction,","cited_arxiv_id":null,"evidence_quote":"Provides the MOTCat baseline that ICFNet reproduces and reports beating by 5.29% average C-index, and supplies the optimal-transport co-attention idea."},{"cited_title":"Multimodal co-attention trans- former for survival prediction in gigapixel whole slide images,","cited_arxiv_id":null,"evidence_quote":"The earlier WSI-genomics co-attention fusion method whose setup ICFNet extends with text and demographics."},{"cited_title":"Self- normalizing neural networks,","cited_arxiv_id":null,"evidence_quote":"Supplies the self-normalizing neural network used as the genomic encoder for the six gene groups."},{"cited_title":"Data-efficient and weakly supervised computational pathology on whole-slide images,","cited_arxiv_id":null,"evidence_quote":"Supplies the foreground segmentation and multi-instance patch-extraction pipeline, and provides CLAM as a baseline survival method."},{"cited_title":"Clip-adapter: Better vision-language models with feature adapters,","cited_arxiv_id":null,"evidence_quote":"Supplies the CLIP-based text encoder with an adapter used to encode demographic and treatment prompts."},{"cited_title":"Bias in cross-entropy-based training of deep survival networks,","cited_arxiv_id":null,"evidence_quote":"The negative log-likelihood survival loss that the paper modifies into a balanced version to handle label imbalance."}],"review_version":1}