Pith. sign in

REVIEW 3 major objections 5 minor 47 references

ICFNet: Integrated Cross-modal Fusion Network for Survival Prediction

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proposes ICFNet, a multimodal network that combines histopathology slides, genomic profiles, demographics, and treatment text to predict cancer patient survival, reporting an average 5.29% C-index improvement over the previous…

desk verdict A competent four-modality extension of MOTCat with plausible C-index gains, but the causal reading of treatment effects in §4.5 is not supported and the proposed BNLL loss has a suspicious balancing argument. read the letter →

arxiv 2501.02778 v1 pith:FBUDFOF6 submitted 2025-01-06 eess.IV cs.AIcs.CV

Binyu Zhang , Zhu Meng , Junhao Dong , Fei Su , Zhicheng Zhao This is my paper · ORCID
classification eess.IVcs.AIcs.CV
keywords multi-modalfusionsurvivalpredictionwholeslideimagesgenomicsoptimaltransportmultipleinstancelearningclinicaltextTCGA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that survival prediction improves when patient demographics and treatment protocols are added to the histopathology and genomic data that current models use. On five TCGA cancer cohorts, the proposed ICFNet network reports new state-of-the-art concordance indices, with an average 5.29% C-index gain over MOTCat, the strongest prior model. The gain is argued to come from cross-modal fusion: optimal-transport co-attention links image patches to genomic and text features, while a residual orthogonal decomposition removes redundant cross-modal information and a balanced loss treats short- and long-survival patients fairly. If the claim holds, adding two cheap clinical text fields to existing histology-plus-genomics pipelines is a robust benchmark improvement.

What carries the argument

The machinery is a modular fusion stack. A ResNet50 encodes WSI patches, six self-normalizing networks encode gene groups, and a CLIP-based text encoder plus MLP encode demographics and treatment. An optimal-transport co-attention module computes a matching flow between image features and genomic/text features, preserving global structure while aligning modalities. A residual orthogonal decomposition (ROD) projects patch features orthogonal to the cross-modal features to strip redundancy, a unification fusion module maps all five features into a shared latent space, and a balanced negative log-likelihood loss reweights survival bins so short-survival patients are trained as much as long-survival ones.

What would settle it

Retrain ICFNet with treatment labels randomly permuted across patients while keeping all other inputs identical; if the C-index gain over the no-treatment model persists under permutation, the reported improvement cannot be attributed to the treatment text's content. Alternatively, flip the treatment prompt for each patient and check whether the risk-score change agrees with outcomes in patient subgroups matched on stage and genomic profile.

Watch

Extended reading notes

Core claim

ICFNet's central claim is that prognosis is better modeled by four modalities than by two. The authors report that on BLCA, BRCA, GBMLGG, LUAD, and UCEC, ICFNet achieves C-indices of 0.709, 0.724, 0.854, 0.697, and 0.702 respectively, with an overall 0.737, outperforming MOTCat by 5.29% on average. The model encodes whole slide images with a ResNet50, genomics with six self-normalizing networks, and demographics/treatment with a CLIP-based text encoder plus an MLP, then fuses them through optimal-transport co-attention. The paper also shows that changing only the treatment text changes the predicted risk, which the authors present as a step toward using the model to compare treatment options in clinical decision-making.

Load-bearing premise

The treatment information in the input is the treatment each patient actually received, and the model treats that text as a knob that can be turned in hypothetical scenarios; if treatment choice is entangled with how sick a patient already is, the model's response to a changed prompt may not measure the treatment's true effect.

Editorial extensions

If this is right

  • Adding demographic and treatment text to histology-plus-genomics models yields consistent C-index gains on all five cohorts, with the largest gains on BRCA (11.40%) and BLCA (6.48%).
  • Because treatment is encoded as editable text, the model offers a way to compare treatment options by changing the prompt, which the paper presents as a clinical decision-support tool.
  • The balanced loss ensures patients with short survival times are not underweighted during training, which should make risk estimates more reliable for high-risk patients.
  • The text-only experiments show that demographics and treatment alone can already drive prediction, especially for breast cancer, so the model's gain is not purely from image or genomic features.
  • The modular design means new modalities (e.g., radiology or lab values) can be added with a new encoder and an OT interaction branch.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's clinical counterfactual reading—that editing the treatment prompt reveals the effect of a treatment—is not supported by the training setup, because treatment is the treatment the patient actually received, and its assignment is likely confounded with prognosis. The reported gains may partly reflect this coupling.
  • One testable extension is to use the same architecture with free-text clinical notes instead of the short template prompts; if the gains are driven by treatment detail, richer notes should improve C-index further.
  • The uneven gains across cancers (1.43% on GBMLGG vs 11.40% on BRCA) suggest the value of text is disease-dependent; a cohort-level analysis of which clinical variables matter could guide where multimodal fusion pays off.
  • The ROD orthogonalization is a general recipe: explicitly enforcing that modality-specific features are orthogonal to cross-modal summaries may help any multi-modal medical model that suffers from redundancy in small-sample regimes.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes ICFNet, a multimodal survival prediction network that combines histopathology whole-slide images, genomic expression profiles, patient demographics, and treatment information. WSIs are encoded with a pretrained ResNet, genomics with six self-normalizing networks, and demographic/treatment data with a CLIP text encoder plus an MLP on a tensorized array. Cross-modal interaction uses optimal-transport-based co-attention, followed by a residual orthogonal decomposition (ROD) module and a unification fusion module. Training uses a dense supervision scheme and a proposed Balanced Negative Log-Likelihood Loss (BNLLLoss). The method is evaluated with C-index on five TCGA datasets (BLCA, BRCA, GBMLGG, LUAD, UCEC) under 5-fold cross-validation, reporting an average C-index gain of 5.29% over MOTCat, along with ablations, Kaplan-Meier curves, GradCAM visualizations, and a discussion of clinical decision support. The code is publicly available.

Significance. If the reported results are robust, the paper makes a useful empirical contribution: it shows that adding cheap clinical text (demographics and treatment received) to histology and genomics can improve prognostic C-index on several TCGA cohorts. The manuscript includes open code, external validation on five datasets, ablation studies, and KM-curve analyses, all of which support the internal consistency of the experimental pipeline. However, the clinical decision-support claim in Section 4.5 requires causal interpretation of treatment, which the observational study design does not support. The benchmark comparison also conflates added input information with better fusion, because the treatment variable is post-diagnosis and correlated with outcome via treatment-by-indication. The BNLL loss derivation in Section 3.3 is internally inconsistent with Eq. 10, and the SOTA comparison in Table 2 mixes reproduced and quoted baselines. These issues do not necessarily invalidate the predictive accuracy claim, but they do affect the strength of the paper's central conclusions.

major comments (3)
  1. [Section 4.5, Fig. 6; Section 3.1] The clinical decision-support claim is not supported by the experimental design. Section 3.1 defines treatment as 'whether radiation or pharmaceutical therapy was applied to the patients,' i.e., the treatment actually received, which on observational TCGA data is confounded with disease severity, stage, and prognosis. The model is trained on actual treatment, so it can exploit treatment-by-indication (e.g., sicker patients receiving more aggressive therapy) to improve C-index. Changing the treatment text in Fig. 6 for an already-treated patient is a counterfactual extrapolation without propensity overlap or causal adjustment; the model has never observed the same patient under alternative treatments. The abstract and conclusion therefore overstate the decision-support value. The authors should either remove the causal claims, reframe them as descriptive associations, or provide a propensity-stratified or otherwise causally adjusted evaluation.
  2. [Section 3.3, Eq. (10)-(11)] The motivation and derivation of BNLLLoss are internally inconsistent. Eq. (10) defines Ssurv,i(yi) as a product of (1 - hazard) terms, so Ssurv decreases as yi increases, not 'disproportionately larger for higher values of yi' as claimed in the text. The standard NLL in Eq. (9) is larger for larger yi because it sums more negative log-survival terms, which is the actual imbalance being addressed. However, the paper does not show that the added factor (1 - Shaz,i(yi))^{Nb-1-yi} in Eq. (11) equalizes per-class weights; no derivation or class-weight analysis is provided, and the exponent depends on the bin index in a way that does not obviously yield constant per-bin loss. Since BNLL is a stated contribution and is supported by an ablation row in Table 3, the authors need to provide a correct derivation or an empirical demonstration of the balancing effect.
  3. [Table 2, Section 4.3] The state-of-the-art comparison mixes reproduced baselines (marked with '*') with numbers quoted from prior papers. If the quoted baselines were evaluated under different dataset splits, patch extraction protocols, or preprocessing steps, the reported 5.29% average gain over MOTCat is not a controlled comparison. In addition, MOTCat+Text is a simple concatenation baseline, and ICFNet has access to treatment information that is post-diagnosis and outcome-correlated; therefore the gain over MOTCat conflates added input information with superior fusion. The authors should either reproduce all baselines under the same protocol, clearly state which numbers are quoted and from which papers, and add a controlled comparison that isolates the fusion mechanism from the added treatment variable.
minor comments (5)
  1. [Eq. (12)] The text says 'α and β are hyper-parameters which are set as 0.1', but Eq. (12) contains only α; β is never defined or used. Please clarify.
  2. [Table 3] There is a formatting error in the BRCA column of one ablation row: '0.722±0.0.007' should likely be '0.722±0.007'.
  3. [Section 1 and Section 3.1] The introduction and contribution list refer to 'three modalities' of data, but the method actually uses four types of input: WSIs, genomics, demographics, and treatment text/tensor. Please make the terminology consistent.
  4. [Section 3.1, treatment template] The treatment text template 'Treatments is/are applied.' is vague; it is unclear whether it captures only the binary presence of any treatment or distinguishes radiation from pharmaceutical therapy, and whether dose or regimen details are encoded. This matters for interpreting Fig. 6, where 'various treatment methods' are discussed.
  5. [References] Reference [5] is cited as 'CLIP' in Section 3.1, but the reference is Dosovitskiy et al., 'An image is worth 16x16 words' (ViT), not CLIP. Please correct the citation or the reference list.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ICFNet's benchmark claims rest on held-out TCGA evaluations, and its borrowed components and self-citations are not load-bearing in the derivation.

full rationale

The paper's central claim is an empirical performance comparison on five TCGA datasets. The reported C-index values are computed on held-out test folds under a fixed dataset partition, so the state-of-the-art claim is not constructed from the model's fitted parameters or from the target result. The methodology borrows components from prior work—SNN encoding [6], optimal-transport co-attention [10], CLIP-based text encoding [43], and the MOTCat evaluation protocol—but borrowing is dependence, not circularity; none of these components is invoked as a uniqueness theorem or as an unverified self-citation that forces the conclusion. The authors' self-references [1] and [2] appear only as general examples of prior medical image work in the introduction and do not carry any load-bearing argument. The BNLLLoss is a modified loss function, and the risk score in Eq. (13) is simply the negative sum of predicted survival probabilities; neither reduces by definition to the reported C-index improvements. The only passage with a causal flavor is Section 4.5, where changing the treatment text input is interpreted as observing prognostic differences across treatment protocols; that interpretation may be confounded because the treatment variable reflects therapy actually received, but confounding is a validity risk, not circularity. No equation in the paper equates a prediction to a fitted input, and no parameter is fitted to the evaluation endpoint and then renamed as a prediction. Accordingly, no specific circular step can be exhibited, and the honest finding is no significant circularity.

Assumptions & free parameters 2 free parameters · 2 assumptions · 0 invented entities

The method is an empirical composition of existing ML components and introduces no new physical entities. It does introduce modeling choices such as alpha and Nb that the performance claims depend on, and it relies on the unproved balancing property of the new loss.

free parameters (2)
  • loss weight alpha = 0.1
    Eq. 12 weights the six uni-modal losses and two cosine losses with alpha=0.1; the paper says alpha and beta are 0.1, but beta does not appear in the equation.
  • number of survival time bins Nb = not stated numerically
    Predetermined from training survival times according to Section 3.2; the BNLL exponent (Nb-1-yi) in Eq. 11 depends on it, and the paper does not report how Nb is chosen for each dataset.
assumptions (2)
  • ad hoc to paper The BNLL formula in Eq. 11 balances the training weight across survival classes.
    Asserted in Section 3.3 without a formal derivation; the stated reason that Ssurv is larger for larger yi conflicts with Eq. 10's cumulative product, so the balancing effect is an unproved premise.
  • domain assumption Treatment text is available at prediction time and is causally adjustable.
    Section 3.1 treats applied treatment as input; Section 4.5 uses ICFNet to compare treatment plans. This requires that the treatment variable is a planned intervention rather than a confounded outcome of clinical decisions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ICFNet: Integrated Cross-modal Fusion Network for Survival Prediction." pith.science (2026). https://pith.science/paper/FBUDFOF6

@misc{pith2026250102778,
  author       = {Pith},
  title        = {Pith review of: ICFNet: Integrated Cross-modal Fusion Network for Survival Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FBUDFOF6}},
  note         = {Machine review of arXiv:2501.02778}
}
read the original abstract

Survival prediction is a crucial task in the medical field and is essential for optimizing treatment options and resource allocation. However, current methods often rely on limited data modalities, resulting in suboptimal performance. In this paper, we propose an Integrated Cross-modal Fusion Network (ICFNet) that integrates histopathology whole slide images, genomic expression profiles, patient demographics, and treatment protocols. Specifically, three types of encoders, a residual orthogonal decomposition module and a unification fusion module are employed to merge multi-modal features to enhance prediction accuracy. Additionally, a balanced negative log-likelihood loss function is designed to ensure fair training across different patients. Extensive experiments demonstrate that our ICFNet outperforms state-of-the-art algorithms on five public TCGA datasets, including BLCA, BRCA, GBMLGG, LUAD, and UCEC, and shows its potential to support clinical decision-making and advance precision medicine. The codes are available at: https://github.com/binging512/ICFNet.

Figures

Figures reproduced from arXiv: 2501.02778 by the authors.

Figure 1
Figure 1. The ICFNet consists of three different feature extractors, a feature interaction [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. The structure of ROD module. An average pooling and projector are applied [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. The unification fusion module consists two multi-head attention based blocks. [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The Kaplan-Meier survival curves of MOTCat [10] and ICFNet on five cancer [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]
Figure 5
Figure 5. Figure 5: The visualization of CAM from ICFNet. The red region represents the area [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: The analysis of the survival predictions on the related datasets. (a) is the [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 42 canonical work pages

  1. [1]

    Mtcsnet: One-stage learning and two-point labeling are sufficient for cell segmentation,

    B. Zhang, Z. Meng, H. Li, Z. Zhao, and F. Su, “Mtcsnet: One-stage learning and two-point labeling are sufficient for cell segmentation,” IEEE Transactions on Medical Imaging , 2024. 20

    work page 2024

  2. [2]

    Nusea: Nuclei segmentation with ellipse annotations,

    Z. Meng, J. Dong, B. Zhang, S. Li, R. Wu, F. Su, G. Wang, L. Guo, and Z. Zhao, “Nusea: Nuclei segmentation with ellipse annotations,” IEEE Journal of Biomedical and Health Informatics , 2024

    work page 2024

  3. [3]

    Data-efficient and weakly supervised computational pathology on whole-slide images,

    M. Y. Lu, D. F. Williamson, T. Y. Chen, R. J. Chen, M. Barbieri, and F. Mahmood, “Data-efficient and weakly supervised computational pathology on whole-slide images,” Nature biomedical engineering, vol. 5, no. 6, pp. 555–570, 2021

    work page 2021

  4. [4]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017

    2017

  5. [5]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in International Conference on Learning Representations , 2020

    2020

  6. [6]

    Self- normalizing neural networks,

    G. Klambauer, T. Unterthiner, A. Mayr, and S. Hochreiter, “Self- normalizing neural networks,” Advances in neural information process- ing systems , vol. 30, 2017

    work page 2017

  7. [7]

    Ghrelin upregulates oncogenic aurora a to promote renal cell carcinoma invasion,

    T.-C. Lin, Y.-M. Yeh, W.-L. Fan, Y.-C. Chang, W.-M. Lin, T.-Y. Yang, and M. Hsiao, “Ghrelin upregulates oncogenic aurora a to promote renal cell carcinoma invasion,” Cancers, vol. 11, no. 3, p. 303, 2019

    work page 2019

  8. [8]

    Mmp-1 is overexpressed in triple-negative breast cancer tissues and the knockdown of mmp-1 expression inhibits tumor cell malignant behaviors in vitro,

    Q.-M. Wang, L. Lv, Y. Tang, L. Zhang, and L.-F. Wang, “Mmp-1 is overexpressed in triple-negative breast cancer tissues and the knockdown of mmp-1 expression inhibits tumor cell malignant behaviors in vitro,” Oncology letters, vol. 17, no. 2, pp. 1732–1740, 2019

    work page 2019

Show all 47 references
  1. [9]

    Multimodal co-attention trans- former for survival prediction in gigapixel whole slide images,

    R. J. Chen, M. Y. Lu, W.-H. Weng, T. Y. Chen, D. F. Williamson, T. Manz, M. Shady, and F. Mahmood, “Multimodal co-attention trans- former for survival prediction in gigapixel whole slide images,” in Pro- ceedings of the IEEE/CVF international conference on computer vision, pp....

    2021

  2. [10]

    Multimodal optimal transport-based co-attention transformer with global structure consistency for survival prediction,

    Y. Xu and H. Chen, “Multimodal optimal transport-based co-attention transformer with global structure consistency for survival prediction,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 21241–21251, 2023. 21

    2023

  3. [11]

    Bias in cross-entropy-based training of deep survival networks,

    S. G. Zadeh and M. Schmid, “Bias in cross-entropy-based training of deep survival networks,” IEEE transactions on pattern analysis and ma- chine intelligence , vol. 43, no. 9, pp. 3126–3137, 2020

    2020

  4. [12]

    Attention-based deep multi- ple instance learning,

    M. Ilse, J. Tomczak, and M. Welling, “Attention-based deep multi- ple instance learning,” in International conference on machine learning, pp. 2127–2136, PMLR, 2018

    2018

  5. [13]

    Graph attention multi-instance learning for accurate colorectal cancer stag- ing,

    A. Raju, J. Yao, M. M. Haq, J. Jonnagaddala, and J. Huang, “Graph attention multi-instance learning for accurate colorectal cancer stag- ing,” in Medical Image Computing and Computer Assisted Intervention– MICCAI 2020: 23rd International Conference, Lima, Peru, October 4– 8, 2...

    2020

  6. [14]

    Cluster-to-conquer: A framework for end-to-end multi- instance learning for whole slide image classification,

    Y. Sharma, A. Shrivastava, L. Ehsan, C. A. Moskaluk, S. Syed, and D. Brown, “Cluster-to-conquer: A framework for end-to-end multi- instance learning for whole slide image classification,” in Medical Imag- ing with Deep Learning , pp. 682–698, PMLR, 2021

    2021

  7. [15]

    Dual-stream multiple instance learn- ing network for whole slide image classification with self-supervised con- trastive learning,

    B. Li, Y. Li, and K. W. Eliceiri, “Dual-stream multiple instance learn- ing network for whole slide image classification with self-supervised con- trastive learning,” in Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pp. 14318–14328, 2021

    2021

  8. [16]

    Trans- mil: Transformer based correlated multiple instance learning for whole slide image classification,

    Z. Shao, H. Bian, Y. Chen, Y. Wang, J. Zhang, X. Ji, et al. , “Trans- mil: Transformer based correlated multiple instance learning for whole slide image classification,” Advances in neural information processing systems, vol. 34, pp. 2136–2147, 2021

    2021

  9. [17]

    Dtfd-mil: Double-tier feature distillation multiple instance learning for histopathology whole slide image classification,

    H. Zhang, Y. Meng, Y. Zhao, Y. Qiao, X. Yang, S. E. Coupland, and Y. Zheng, “Dtfd-mil: Double-tier feature distillation multiple instance learning for histopathology whole slide image classification,” in Pro- ceedings of the IEEE/CVF conference on computer vision and pattern r...

    2022

  10. [18]

    Sparse and hierarchical transformer for survival analysis on whole slide images,

    R. Yan, Z. Lv, Z. Yang, S. Lin, C. Zheng, and F. Zhang, “Sparse and hierarchical transformer for survival analysis on whole slide images,” IEEE Journal of Biomedical and Health Informatics , 2023

    2023

  11. [19]

    dmil-transformer: Multiple instance learning 22 via integrating morphological and spatial information for lymph node metastasis classification,

    Y. Chen, Z. Shao, H. Bian, Z. Fang, Y. Wang, Y. Cai, H. Wang, G. Liu, X. Li, and Y. Zhang, “dmil-transformer: Multiple instance learning 22 via integrating morphological and spatial information for lymph node metastasis classification,” IEEE Journal of Biomedical and Health In...

    2023

  12. [20]

    Tgmil: A hybrid multi-instance learning model based on the transformer and the graph attention network for whole- slide images classification of renal cell carcinoma,

    X. Sun, W. Li, B. Fu, Y. Peng, J. He, L. Wang, T. Yang, X. Meng, J. Li, J. Wang, et al. , “Tgmil: A hybrid multi-instance learning model based on the transformer and the graph attention network for whole- slide images classification of renal cell carcinoma,” Computer Methods a...

    2023

  13. [21]

    Cross-scale multi- instance learning for pathological image diagnosis,

    R. Deng, C. Cui, L. W. Remedios, S. Bao, R. M. Womick, S. Chi- ron, J. Li, J. T. Roland, K. S. Lau, Q. Liu, et al. , “Cross-scale multi- instance learning for pathological image diagnosis,” Medical image anal- ysis, vol. 94, p. 103124, 2024

    2024

  14. [22]

    Unveil- ing the power of model-agnostic multiscale analysis for enhancing artifi- cial intelligence models in breast cancer histopathology images,

    N. Tsiknakis, G. Manikis, E. Tzoras, D. Salgkamis, J. M. Vidal, K. Wang, D. Zaridis, E. Sifakis, I. Zerdes, J. Bergh, et al. , “Unveil- ing the power of model-agnostic multiscale analysis for enhancing artifi- cial intelligence models in breast cancer histopathology images,” I...

    2024

  15. [23]

    Si-mil: Taming deep mil for self-interpretability in gigapixel histopathology,

    S. Kapse, P. Pati, S. Das, J. Zhang, C. Chen, M. Vakalopoulou, J. Saltz, D. Samaras, R. R. Gupta, and P. Prasanna, “Si-mil: Taming deep mil for self-interpretability in gigapixel histopathology,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognit...

    2024

  16. [24]

    Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks,

    J. Yao, X. Zhu, J. Jonnagaddala, N. Hawkins, and J. Huang, “Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks,” Medical Image Analysis , vol. 65, p. 101789, 2020

    2020

  17. [25]

    Whole slide images are 2d point clouds: Context-aware survival prediction using patch-based graph convolu- tional networks,

    R. J. Chen, M. Y. Lu, M. Shaban, C. Chen, T. Y. Chen, D. F. Williamson, and F. Mahmood, “Whole slide images are 2d point clouds: Context-aware survival prediction using patch-based graph convolu- tional networks,” in Medical Image Computing and Computer Assisted Intervention–M...

    2021

  18. [26]

    Sur- vival prediction in triple negative breast cancer using multiple instance learning of histopathological images,

    P. Sandarenu, E. K. Millar, Y. Song, L. Browne, J. Beretov, J. Lynch, P. H. Graham, J. Jonnagaddala, N. Hawkins, J. Huang, et al. , “Sur- vival prediction in triple negative breast cancer using multiple instance learning of histopathological images,” Scientific Reports, vol. 1...

    2022

  19. [27]

    Surformer: An interpretable pattern-perceptive survival transformer for cancer survival prediction from histopathology whole slide images,

    Z. Wang, Q. Gao, X. Yi, X. Zhang, Y. Zhang, D. Zhang, P. Li` o, C. Bain, R. Bassed, S. Li, et al., “Surformer: An interpretable pattern-perceptive survival transformer for cancer survival prediction from histopathology whole slide images,” Computer Methods and Programs in Biom...

    2023

  20. [28]

    Lnpl-mil: Learning from noisy pseudo labels for promoting multiple instance learning in whole slide image,

    Z. Shao, Y. Wang, Y. Chen, H. Bian, S. Liu, H. Wang, and Y. Zhang, “Lnpl-mil: Learning from noisy pseudo labels for promoting multiple instance learning in whole slide image,” inProceedings of the IEEE/CVF International Conference on Computer Vision , pp. 21495–21505, 2023

    2023

  21. [29]

    Multi-instance multi-task learning for joint clinical outcome and genomic profile predictions from the histopatho- logical images,

    W. Shao, H. Shi, J. Liu, Y. Zuo, L. Sun, T. Xia, W. Chen, P. Wan, J. Sheng, Q. Zhu, et al. , “Multi-instance multi-task learning for joint clinical outcome and genomic profile predictions from the histopatho- logical images,” IEEE Transactions on Medical Imaging , 2024

    2024

  22. [30]

    Pathomic fusion: an integrated frame- work for fusing histopathology and genomic features for cancer diagnosis and prognosis,

    R. J. Chen, M. Y. Lu, J. Wang, D. F. Williamson, S. J. Rodig, N. I. Lindeman, and F. Mahmood, “Pathomic fusion: an integrated frame- work for fusing histopathology and genomic features for cancer diagnosis and prognosis,” IEEE Transactions on Medical Imaging , vol. 41, no. 4, ...

    2020

  23. [31]

    Q. He, X. Li, D. N. Kim, X. Jia, X. Gu, X. Zhen, and L. Zhou, “Feasi- bility study of a multi-criteria decision-making based hierarchical model for multi-modality feature and multi-classifier fusion: Applications in medical prognosis prediction,” Information Fusion, vol. 55, p...

    2020

  24. [32]

    Modeling dense multimodal interactions between biologi- cal pathways and histology for survival prediction,

    G. Jaume, A. Vaidya, R. J. Chen, D. F. Williamson, P. P. Liang, and F. Mahmood, “Modeling dense multimodal interactions between biologi- cal pathways and histology for survival prediction,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , p...

    2024

  25. [33]

    Cross-modal translation and alignment for sur- vival analysis,

    F. Zhou and H. Chen, “Cross-modal translation and alignment for sur- vival analysis,” in Proceedings of the IEEE/CVF International Confer- ence on Computer Vision , pp. 21485–21494, 2023

    2023

  26. [34]

    Prototypical infor- mation bottlenecking and disentangling for multimodal cancer survival prediction,

    Y. Zhang, Y. Xu, J. Chen, F. Xie, and H. Chen, “Prototypical infor- mation bottlenecking and disentangling for multimodal cancer survival prediction,” (Hybrid, Vienna, Austria), 2024. Discriminability;Genomic data;Image data;Information bottleneck;Multi-modal;Multi-modal data;...

    2024

  27. [35]

    Multimodal prototyping for cancer survival prediction,

    A. H. Song, R. J. Chen, G. Jaume, A. Vaidya, A. S. Baras, and F. Mahmood, “Multimodal prototyping for cancer survival prediction,” vol. 235, (Vienna, Austria), pp. 46050 – 46073, 2024. ’current;Gene groups;Interpretability;Multi-modal;Multimodal prototyping;Small patches;Survi...

    2024

  28. [36]

    Cohort-individual cooperative learning for multimodal cancer survival analysis,

    H. Zhou, F. Zhou, and H. Chen, “Cohort-individual cooperative learning for multimodal cancer survival analysis,” IEEE Transactions on Medical Imaging, pp. 1–1, 2024

    2024

  29. [37]

    Histo- genomic knowledge distillation for cancer prognosis from histopathology whole slide images,

    Z. Wang, Y. Zhang, Y. Xu, S. Imoto, H. Chen, and J. Song, “Histo- genomic knowledge distillation for cancer prognosis from histopathology whole slide images,” arXiv preprint arXiv:2403.10040 , 2024

    2024 arXiv

  30. [38]

    Survival prediction via hierar- chical multimodal co-attention transformer: A computational histology- radiology solution,

    Z. Li, Y. Jiang, M. Lu, R. Li, and Y. Xia, “Survival prediction via hierar- chical multimodal co-attention transformer: A computational histology- radiology solution,” IEEE Transactions on Medical Imaging , vol. 42, no. 9, pp. 2678–2689, 2023

    2023

  31. [39]

    D. Y. Jeong, J. Park, H. Song, J. Moon, T. Lee, C. Ahn, S. Park, S.-H. Lee, C.-Y. Ock, and H. Y. Lee, “Artificial intelligence (ai)-based multi-modal approach using h&e and ct image for predicting treatment response of immune checkpoint inhibitor (ici) in non-small cell lung c...

    2024

  32. [40]

    The molecular signatures database hallmark gene set collection,

    A. Liberzon, C. Birger, H. Thorvaldsd´ ottir, M. Ghandi, J. P. Mesirov, and P. Tamayo, “The molecular signatures database hallmark gene set collection,” Cell systems , vol. 1, no. 6, pp. 417–425, 2015. 25

    2015

  33. [41]

    Llama 2: Open foundation and fine-tuned chat models,

    M. GenAI, “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288 , 2023

    2023 arXiv

  34. [42]

    Blip: Bootstrapping language- image pre-training for unified vision-language understanding and gen- eration,

    J. Li, D. Li, C. Xiong, and S. Hoi, “Blip: Bootstrapping language- image pre-training for unified vision-language understanding and gen- eration,” in International conference on machine learning , pp. 12888– 12900, PMLR, 2022

    2022

  35. [43]

    Clip-adapter: Better vision-language models with feature adapters,

    P. Gao, S. Geng, R. Zhang, T. Ma, R. Fang, Y. Zhang, H. Li, and Y. Qiao, “Clip-adapter: Better vision-language models with feature adapters,” International Journal of Computer Vision , vol. 132, no. 2, pp. 581–595, 2024

    2024

  36. [44]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016

    2016

  37. [45]

    On the translocation of masses.,

    L. V. Kantorovich, “On the translocation of masses.,” Journal of math- ematical sciences, vol. 133, no. 4, 2006

    2006

  38. [46]

    Pan-cancer integra- tive histology-genomic analysis via multimodal deep learning,

    R. J. Chen, M. Y. Lu, D. F. Williamson, T. Y. Chen, J. Lipkova, Z. Noor, M. Shaban, M. Shady, M. Williams, B. Joo, et al., “Pan-cancer integra- tive histology-genomic analysis via multimodal deep learning,” Cancer Cell, vol. 40, no. 8, pp. 865–878, 2022

    2022

  39. [47]

    Grad-cam: Visual explanations from deep networks via gradient-based localization,

    R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” in Proceedings of the IEEE international conference on computer vision , pp. 618–626, 2017. 26

    2017

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.