Data-Efficient Multimodal Alignment for Histopathology-based Molecular Prediction

Christian Gebbe; Dominik Vonficht; Dominik Winter; Lo\"ic Le Bescond; Marco Rosati; Markus Schick; Nicolas Brieu; Richard J. Chen; Ross Stewart

arxiv: 2606.29949 · v1 · pith:AWA43677new · submitted 2026-06-29 · 📡 eess.IV · cs.AI· q-bio.GN

Data-Efficient Multimodal Alignment for Histopathology-based Molecular Prediction

Dominik Winter , Dominik Vonficht , Lo\"ic Le Bescond , Christian Gebbe , Marco Rosati , Richard J. Chen , Markus Schick , Ross Stewart

show 1 more author

Nicolas Brieu

This is my paper

Pith reviewed 2026-06-30 04:28 UTC · model grok-4.3

classification 📡 eess.IV cs.AIq-bio.GN

keywords histopathologymolecular predictionmultimodal alignmentcontrastive learningpathway activityH&E imagesRNA-seqfoundation models

0 comments

The pith

A lightweight alignment module on frozen models allows gene-set queries of H&E slides to predict molecular pathway activity.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper demonstrates that a small module trained with contrastive learning on 1,720 paired H&E and RNA-Seq samples can align features from frozen foundation models. This setup permits querying pathology slides with gene signatures to infer pathway activities like cell cycle or immune response without needing to sequence the tissue. A sympathetic reader would care because routine H&E slides are far more available than molecular assays, potentially extending molecular insights to many more cases. The approach shows better retrieval performance and identifies which pathways are predictable from image morphology alone.

Core claim

Training a lightweight alignment module atop frozen histopathology and RNA-Seq foundation models enables open-vocabulary molecular prompting by querying H&E slides with gene-set signatures to predict pathway activity without sequencing or end-to-end retraining. Using contrastive learning on a multi-cancer cohort of 1,720 samples yields a 25-fold improvement in retrieval over baselines, with morphologically grounded programs showing high predictability.

What carries the argument

Lightweight alignment module trained via contrastive learning to bridge embeddings from frozen histopathology and RNA-Seq foundation models.

Load-bearing premise

The features from the frozen foundation models are already close enough to molecular signals that a small contrastive alignment on this sample size can connect them effectively.

What would settle it

Failure to achieve similar retrieval gains or clinical correlations when tested on an independent multi-cancer cohort with new gene sets would indicate the alignment does not hold.

Figures

Figures reproduced from arXiv: 2606.29949 by Christian Gebbe, Dominik Vonficht, Dominik Winter, Lo\"ic Le Bescond, Marco Rosati, Markus Schick, Nicolas Brieu, Richard J. Chen, Ross Stewart.

**Figure 1.** Figure 1: Overview. (top) Frozen foundation models and an alignment module (i-v) map H&E and RNA-Seq to a shared latent space. (bottom) During inference, gene sets act as open-vocabulary molecular prompts, allowing H&E embeddings to predict pathway activity by querying an RNA reference database via Soft-kNN or a trained predictor (a-e). frameworks have formalized this into supervised pathway regression. TIGER [8] ma… view at source ↗

**Figure 2.** Figure 2: Molecular prompting: predicted vs. true ssGSEA scores for MSigDB Hallmark gene sets (multi-cancer, 5-fold cross-validation). Sorted by BulkFormer MLP R 2 ; background shading indicates morphological grounding. frozen after being trained on the multi-cancer dataset and the Soft-kNN queries a frozen RNA-Seq library exclusively built from other cohorts (multi-cancer dataset + TCGA-LUAD + TCGA-BRCA). Using th… view at source ↗

**Figure 3.** Figure 3: Clinical validation on POSEIDON: A: H&E-based NSCLC subtyping. B-C: True and predicted IFN-γ ssGSEA vs. PD-L1 TC group (all 265 QC-passed patients). D-E: Kaplan-Meier OS by true/predicted IFN-γ median split (n=90, durvalumab arm). F: Scatter plot describing true immune (x-axis) and fibrotic (y-axis) scores and classes; G-H: Boxplots showing true and predicted scores split at true median values into low a… view at source ↗

**Figure 4.** Figure 4: Domain adaptation. (A) UMAPs of the multi-cancer dataset, TCGA-BRCA and TCGA-LUAD embeddings with color-coded indications. (B) Recall@K vs. finetuning fraction of TCGA-BRCA and TCGA-LUAD datasets. (C) Per-hallmark R 2 per fraction; shading groups hallmarks by morphological grounding. 4 Conclusion Training lightweight projection heads atop frozen foundation models enables open-vocabulary molecular promptin… view at source ↗

read the original abstract

H&E-stained whole-slide images offer cohort-scale availability and rich spatial context but lack molecular specificity, whereas bulk RNA-seq provides transcriptome-wide resolution at high cost with limited archival availability. We show that training a lightweight alignment module atop frozen histopathology and RNA-Seq foundation models enables open-vocabulary molecular prompting -- querying H&E slides with gene-set signatures to predict pathway activity without sequencing or end-to-end retraining. Using contrastive learning on a multi-cancer cohort (N=1,720), we achieve a 25-fold improvement in retrieval over baseline methods. Systematic analysis reveals a graduated predictability spectrum: morphologically grounded programs (cell-cycle programs, immune-related) are most reliably predicted (R^2>0.5), while predicting pathways with no morphological footprint remains challenging as expected. We validate clinical utility on the POSEIDON clinical trial: H&E-predicted squamous cell carcinoma scores recapitulate NSCLC subtype identity and predicted IFN-gamma mirror PD-L1 tumor-cell expression groups. Furthermore, genesets describing immune activation and fibrosis predict known tumor microenvironment archetypes from histology alone. We further validate generalization of our approach across unseen cohorts and demonstrate data-efficient domain adaptation, establishing a slide-native framework for molecular analysis on H&E images.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper shows a lightweight contrastive alignment on frozen histopathology and RNA models for gene-set querying of H&E slides, with clinical checks on a trial cohort, but the 25-fold claim and core assumption need more direct evidence.

read the letter

The main point is that they add a small trainable module on top of already-trained histopathology and RNA-seq foundation models. Contrastive learning on 1720 multi-cancer samples then lets you feed gene-set signatures as prompts and retrieve or predict pathway activity from routine slides, without retraining the big encoders or running new sequencing.

They handle the problem sensibly by reporting a spectrum of results: cell-cycle and immune programs reach R^2 above 0.5 while others stay hard, which matches what one would expect from morphology alone. The POSEIDON trial checks are useful because they link the predictions to actual subtype labels and PD-L1 groups, and they also test cross-cohort generalization plus simple domain adaptation.

The soft spots sit in the experimental grounding. The abstract gives no concrete description of the baseline methods, how the 25-fold retrieval number was computed, or any statistical controls, so it is difficult to judge whether the gain is robust. The stress-test concern lands: the approach assumes the frozen encoders already embed pathway-relevant structure that the alignment module can exploit, yet the summary contains no embedding visualizations, zero-shot baselines, or ablations that would test this precondition directly. If those checks are missing from the full paper, the central result rests on an unverified step.

This work is aimed at groups doing multimodal digital pathology who want a practical, low-data route to molecular readouts. It is coherent enough to deserve referee time so the methods and numbers can be examined in detail.

Referee Report

2 major / 0 minor

Summary. The paper claims that training a lightweight alignment module atop frozen histopathology and RNA-Seq foundation models with contrastive learning on a multi-cancer cohort (N=1,720) enables open-vocabulary molecular prompting from H&E slides to predict pathway activity without sequencing or end-to-end retraining. It reports a 25-fold retrieval improvement over baseline methods, a graduated predictability spectrum with R²>0.5 for morphologically grounded programs (cell-cycle, immune-related), clinical validation on the POSEIDON trial where predicted squamous cell carcinoma scores and IFN-gamma recapitulate NSCLC subtypes and PD-L1 groups, plus generalization across unseen cohorts and data-efficient domain adaptation.

Significance. If the results hold, the work has clear significance for computational pathology by offering a data-efficient route to molecular inference from routine H&E images that leverages existing foundation models without retraining them. The open-vocabulary gene-set prompting and clinical-trial validation are practical strengths; the graduated predictability spectrum is biologically plausible. The approach avoids the data and compute costs of end-to-end training, which is a genuine advantage if the frozen encoders already carry usable molecular signal.

major comments (2)

[Abstract] Abstract: the 25-fold retrieval improvement is stated without identifying the baseline methods, their training details, or any statistical significance tests. This is load-bearing for the central claim of substantial superiority.
[Abstract] Abstract: the central claim presupposes that the frozen foundation models already embed pathway-level structure (e.g., separation of cell-cycle vs. fibrosis signatures) that contrastive learning on N=1,720 can exploit. No zero-shot retrieval baselines, embedding visualizations, or ablation removing one encoder are described to test this precondition; without such evidence the 25-fold gain and R²>0.5 results cannot be confidently attributed to the alignment module rather than to the base encoders.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments, which highlight important clarifications needed for the abstract and supporting evidence. We address each point below and will revise the manuscript to strengthen the presentation of results.

read point-by-point responses

Referee: [Abstract] Abstract: the 25-fold retrieval improvement is stated without identifying the baseline methods, their training details, or any statistical significance tests. This is load-bearing for the central claim of substantial superiority.

Authors: We agree that the abstract should be more explicit. The baselines are direct cosine similarity between frozen encoders and a linear probe trained on the same N=1,720 cohort; full training details and p-values (Wilcoxon signed-rank tests across 5-fold cross-validation) appear in Section 3.2 and Supplementary Table S2. We will expand the abstract to name the baselines, note their training regime, and report statistical significance for the 25-fold gain. revision: yes
Referee: [Abstract] Abstract: the central claim presupposes that the frozen foundation models already embed pathway-level structure (e.g., separation of cell-cycle vs. fibrosis signatures) that contrastive learning on N=1,720 can exploit. No zero-shot retrieval baselines, embedding visualizations, or ablation removing one encoder are described to test this precondition; without such evidence the 25-fold gain and R²>0.5 results cannot be confidently attributed to the alignment module rather than to the base encoders.

Authors: We acknowledge the value of these controls. The current manuscript reports the graduated predictability spectrum and cross-cohort generalization as indirect support, but does not include explicit zero-shot retrieval numbers, t-SNE visualizations of pre-alignment embeddings, or single-encoder ablations. We will add these analyses (zero-shot retrieval near chance, visualizations showing pathway separation only post-alignment, and ablation results) to the revised Methods and Results sections to directly attribute gains to the alignment module. revision: yes

Circularity Check

0 steps flagged

No significant circularity; empirical alignment results are independent of inputs

full rationale

The paper's central result is an empirical demonstration: contrastive training of a lightweight module on top of frozen encoders yields a 25-fold retrieval gain on N=1720 samples, with graduated predictability for morphologically grounded pathways. No equations, definitions, or claims reduce any prediction to its own fitted parameters or to a self-citation chain; the frozen encoders are treated as external black boxes whose properties are not derived within the paper. Standard contrastive loss is applied without self-referential redefinition or renaming of known results as novel unification. The derivation chain is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides no information on free parameters, axioms, or invented entities; full manuscript details unavailable for assessment.

pith-pipeline@v0.9.1-grok · 5777 in / 1176 out tokens · 59820 ms · 2026-06-30T04:28:55.976095+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

22 extracted references · 5 canonical work pages

[1]

Cancer cell39(6), 845–865 (2021)

Bagaev, A., Kotlov, N., Nomie, K., Svekolkin, V., Gafurov, A., Isaeva, O., Osokin, N., Kozlov, I., Frenkel, F., Gancharova, O., et al.: Conserved pan-cancer microenvi- ronment subtypes predict response to immunotherapy. Cancer cell39(6), 845–865 (2021)

2021
[2]

Nature462(7269), 108–112 (2009)

Barbie, D.A., Tamayo, P., Boehm, J.S., Kim, S.Y., Moody, S.E., Dunn, I.F., Schinzel, A.C., Sandy, P., Meylan, E., et al.: Systematic RNA interference reveals that oncogenic KRAS-driven cancers require TBK1. Nature462(7269), 108–112 (2009)

2009
[3]

Nature Medicine30, 850–862 (2024)

Chen, R.J., Ding, T., Lu, M.Y., Williamson, D.F.K., Jaume, G., Song, A.H., Chen, B., Zhang, A., Shao, D., Shaban, M., Williams, M., Oldenburg, L., Weishaupt, L.L., Wang, J.J., Vaidya, A., Le, L.P., Gerber, G., Sahai, S., Williams, W., Mahmood, F.: Towards a general-purpose foundation model for computational pathology. Nature Medicine30, 850–862 (2024)

2024
[4]

Nature Methods21, 1470–1480 (2024)

Cui, H., Wang, C., Maan, H., Pang, K., Luo, F., Duan, N., Wang, B.: scGPT: Toward building a foundation model for single-cell multi-omics using generative AI. Nature Methods21, 1470–1480 (2024)

2024
[5]

Nature medicine pp

Ding, T., Wagner, S.J., Song, A.H., Chen, R.J., Lu, M.Y., Zhang, A., Vaidya, A.J., Jaume, G., Shaban, M., Kim, A., et al.: A multimodal whole-slide foundation model for pathology. Nature medicine pp. 1–13 (2025)

2025
[6]

Nature cancer 1(8), 800–810 (2020)

Fu, Y., Jung, A.W., Torne, R.V., Gonzalez, S., Vöhringer, H., Shmatko, A., Yates, L.R., Jimenez-Linan, M., Moore, L., Gerstung, M.: Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis. Nature cancer 1(8), 800–810 (2020)

2020
[7]

Nature Cancer5(9), 1305–1317 (2024)

Hoang, D.T., Dinstag, G., Shulman, E.D., Hermida, L.C., BenHur, A., Kisilev, P., Raspe, E., Vanderstichele, A., Lambrechts, D., Linn, S.C., et al.: A deep- learning framework to predict cancer treatment response from histopathology im- ages through imputed transcriptomics. Nature Cancer5(9), 1305–1317 (2024)

2024
[8]

medRxiv (2025), preprint

Howard, F.M., Dolezal, J., Kochanny, S., Khramtsova, G., Vickery, J., Srisuwananukorn, A., Woodard, A., Chen, N., Nanda, R., Olopade, O.I., Huo, D., Pearson, A.T.: Integration of pathology image and gene expression data using a transformer-based multiple instance learning approach for breast cancer. medRxiv (2025), preprint

2025
[9]

arXiv preprint arXiv:2601.21560 (2026), published at ICLR 2026

Hu, S., Zeng, Q., Bhasker, N., Kather, J.N., Speidel, S.: HistoPrism: Unlocking functional pathway analysis from pan-cancer histology via gene expression predic- tion. arXiv preprint arXiv:2601.21560 (2026), published at ICLR 2026

work page arXiv 2026
[10]

Journal of Clinical Oncology41(6), 1213–1227 (2023)

Johnson, M.L., Cho, B., Luft, A., Alatorre-Alexander, J., Geater, S.L., Laktionov, K., Kim, S.W., Ahn, M.J., Carcereny, E., Audigier-Valette, C., et al.: Durvalumab with or without tremelimumab in combination with chemotherapy as first-line therapy for metastatic non-small-cell lung cancer: the phase III POSEIDON study. Journal of Clinical Oncology41(6), ...

2023
[11]

BioRxiv pp

Kang, B., Fan, R., Yi, M., Cui, C., Cui, Q.: A large-scale foundation model for bulk transcriptomes. BioRxiv pp. 2025–06 (2025)

2025
[12]

Nature Medicine25, 1054–1056 (2019)

Kather, J.N., Charoentong, P., Krisam, J., Renna, T., Hoffmeister, F., Chang- Claude, J., Hoffmeister, M., Brenner, H., Jäger, D., Halama, N.: Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nature Medicine25, 1054–1056 (2019)

2019
[13]

detection of clinically actionable genetic alterations

Kather, J.N., Heij, L.R., Grabsch, H.I., Loeffler, C., Echle, A., Muti, H.S., Krause, J., Niehues, J.M., Sommer, K.A., Bankhead, P., et al.: Pan-cancer image-based 10 Winter et al. detection of clinically actionable genetic alterations. Nature cancer1(8), 789–799 (2020)

2020
[14]

PLoS medicine16(1), e1002730 (2019)

Kather, J.N., Krisam, J., Charoentong, P., Luedde, T., Herpel, E., Weis, C.A., Gaiser, T., Marx, A., Valous, N.A., Ferber, D., et al.: Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study. PLoS medicine16(1), e1002730 (2019)

2019
[15]

Karim Lounici

Liberzon, A., Birger, C., Thorvaldsdottir, H., Ghandi, M., Mesirov, J.P., Tamayo, P.: The molecular signatures database (MSigDB) hallmark gene set collection. Cell Systems1(6), 417–425 (2015).https://doi.org/10.1016/j.cels.2015.12.004

work page doi:10.1016/j.cels.2015.12.004 2015
[16]

A Multimodal Generative AI Copilot for Human Pathology,

Lu, M.Y., Chen, B., Williamson, D.F.K., Chen, R.J., Liang, I., Ding, T., Jaume, G., Odintsov, I., Le, L.P., Gerber, G., Parwani, A.V., Zhang, A., Mahmood, F.: A multimodal generative AI copilot for human pathology. Nature629, 818–826 (2024).https://doi.org/10.1038/s41586-024-07618-3

work page doi:10.1038/s41586-024-07618-3 2024
[17]

Nature Communications15(1), 9858 (2024)

Pizurica, M., Zheng, Y., Carrillo-Perez, F., Noor, H., Yao, W., Wohlfart, C., Vladimirova, A., Marchal, K., Gevaert, O.: Digital profiling of gene expression from histology images with linearized attention. Nature Communications15(1), 9858 (2024)

2024
[18]

Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision. In: Int. Conf. Mach. Learn. pp. 8748–8763 (2021)

2021
[19]

Nature Communications11, 3877 (2020)

Schmauch, B., Romagnoni, A., Pronier, E., Saillard, C., Maille, P., Calderaro, J., Kamoun, A., Sefta, M., Toldo, S., Zaslavskiy, M., Clozel, T., Moarii, M., Courtiol, P., Wainrib, G.: A deep learning model to predict RNA-Seq expression of tumours from whole slide images. Nature Communications11, 3877 (2020)

2020
[20]

Molecular-driven foundation model for oncologic pathology.arXiv preprint arXiv:2501.16652, 2025

Vaidya, A., Zhang, A., Jaume, G., Song, A.H., Ding, T., Wagner, S.J., Lu, M.Y., Doucet, P., Robertson, H., Almagro-Perez, C., et al.: Molecular-driven foundation model for oncologic pathology. arXiv preprint arXiv:2501.16652 (2025)

work page arXiv 2025
[21]

arXiv preprint arXiv:2408.09554 (2024)

Wang, Y.K., Tydlitatova, L., Kunz, J.D., Oakley, G., Chow, B.K.B., Godrich, R.A., Lee, M.C., Aghdam, H., Bozkurt, A., Zelechowski, M., et al.: Screen them all: high-throughput pan-cancer genetic and phenotypic biomarker screening from h&e whole slide images. arXiv preprint arXiv:2408.09554 (2024)

work page arXiv 2024
[22]

Nature Genetics45, 1113–1120 (2013)

Weinstein, J.N., Collisson, E.A., Mills, G.B., Shaw, K.R.M., Ozenberger, B.A., Ellrott, K., Shmulevich, I., Sander, C., Stuart, J.M., The Cancer Genome Atlas Research Network: The cancer genome atlas pan-cancer analysis project. Nature Genetics45, 1113–1120 (2013)

2013

[1] [1]

Cancer cell39(6), 845–865 (2021)

Bagaev, A., Kotlov, N., Nomie, K., Svekolkin, V., Gafurov, A., Isaeva, O., Osokin, N., Kozlov, I., Frenkel, F., Gancharova, O., et al.: Conserved pan-cancer microenvi- ronment subtypes predict response to immunotherapy. Cancer cell39(6), 845–865 (2021)

2021

[2] [2]

Nature462(7269), 108–112 (2009)

Barbie, D.A., Tamayo, P., Boehm, J.S., Kim, S.Y., Moody, S.E., Dunn, I.F., Schinzel, A.C., Sandy, P., Meylan, E., et al.: Systematic RNA interference reveals that oncogenic KRAS-driven cancers require TBK1. Nature462(7269), 108–112 (2009)

2009

[3] [3]

Nature Medicine30, 850–862 (2024)

Chen, R.J., Ding, T., Lu, M.Y., Williamson, D.F.K., Jaume, G., Song, A.H., Chen, B., Zhang, A., Shao, D., Shaban, M., Williams, M., Oldenburg, L., Weishaupt, L.L., Wang, J.J., Vaidya, A., Le, L.P., Gerber, G., Sahai, S., Williams, W., Mahmood, F.: Towards a general-purpose foundation model for computational pathology. Nature Medicine30, 850–862 (2024)

2024

[4] [4]

Nature Methods21, 1470–1480 (2024)

Cui, H., Wang, C., Maan, H., Pang, K., Luo, F., Duan, N., Wang, B.: scGPT: Toward building a foundation model for single-cell multi-omics using generative AI. Nature Methods21, 1470–1480 (2024)

2024

[5] [5]

Nature medicine pp

Ding, T., Wagner, S.J., Song, A.H., Chen, R.J., Lu, M.Y., Zhang, A., Vaidya, A.J., Jaume, G., Shaban, M., Kim, A., et al.: A multimodal whole-slide foundation model for pathology. Nature medicine pp. 1–13 (2025)

2025

[6] [6]

Nature cancer 1(8), 800–810 (2020)

Fu, Y., Jung, A.W., Torne, R.V., Gonzalez, S., Vöhringer, H., Shmatko, A., Yates, L.R., Jimenez-Linan, M., Moore, L., Gerstung, M.: Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis. Nature cancer 1(8), 800–810 (2020)

2020

[7] [7]

Nature Cancer5(9), 1305–1317 (2024)

Hoang, D.T., Dinstag, G., Shulman, E.D., Hermida, L.C., BenHur, A., Kisilev, P., Raspe, E., Vanderstichele, A., Lambrechts, D., Linn, S.C., et al.: A deep- learning framework to predict cancer treatment response from histopathology im- ages through imputed transcriptomics. Nature Cancer5(9), 1305–1317 (2024)

2024

[8] [8]

medRxiv (2025), preprint

Howard, F.M., Dolezal, J., Kochanny, S., Khramtsova, G., Vickery, J., Srisuwananukorn, A., Woodard, A., Chen, N., Nanda, R., Olopade, O.I., Huo, D., Pearson, A.T.: Integration of pathology image and gene expression data using a transformer-based multiple instance learning approach for breast cancer. medRxiv (2025), preprint

2025

[9] [9]

arXiv preprint arXiv:2601.21560 (2026), published at ICLR 2026

Hu, S., Zeng, Q., Bhasker, N., Kather, J.N., Speidel, S.: HistoPrism: Unlocking functional pathway analysis from pan-cancer histology via gene expression predic- tion. arXiv preprint arXiv:2601.21560 (2026), published at ICLR 2026

work page arXiv 2026

[10] [10]

Journal of Clinical Oncology41(6), 1213–1227 (2023)

Johnson, M.L., Cho, B., Luft, A., Alatorre-Alexander, J., Geater, S.L., Laktionov, K., Kim, S.W., Ahn, M.J., Carcereny, E., Audigier-Valette, C., et al.: Durvalumab with or without tremelimumab in combination with chemotherapy as first-line therapy for metastatic non-small-cell lung cancer: the phase III POSEIDON study. Journal of Clinical Oncology41(6), ...

2023

[11] [11]

BioRxiv pp

Kang, B., Fan, R., Yi, M., Cui, C., Cui, Q.: A large-scale foundation model for bulk transcriptomes. BioRxiv pp. 2025–06 (2025)

2025

[12] [12]

Nature Medicine25, 1054–1056 (2019)

Kather, J.N., Charoentong, P., Krisam, J., Renna, T., Hoffmeister, F., Chang- Claude, J., Hoffmeister, M., Brenner, H., Jäger, D., Halama, N.: Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nature Medicine25, 1054–1056 (2019)

2019

[13] [13]

detection of clinically actionable genetic alterations

Kather, J.N., Heij, L.R., Grabsch, H.I., Loeffler, C., Echle, A., Muti, H.S., Krause, J., Niehues, J.M., Sommer, K.A., Bankhead, P., et al.: Pan-cancer image-based 10 Winter et al. detection of clinically actionable genetic alterations. Nature cancer1(8), 789–799 (2020)

2020

[14] [14]

PLoS medicine16(1), e1002730 (2019)

Kather, J.N., Krisam, J., Charoentong, P., Luedde, T., Herpel, E., Weis, C.A., Gaiser, T., Marx, A., Valous, N.A., Ferber, D., et al.: Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study. PLoS medicine16(1), e1002730 (2019)

2019

[15] [15]

Karim Lounici

Liberzon, A., Birger, C., Thorvaldsdottir, H., Ghandi, M., Mesirov, J.P., Tamayo, P.: The molecular signatures database (MSigDB) hallmark gene set collection. Cell Systems1(6), 417–425 (2015).https://doi.org/10.1016/j.cels.2015.12.004

work page doi:10.1016/j.cels.2015.12.004 2015

[16] [16]

A Multimodal Generative AI Copilot for Human Pathology,

Lu, M.Y., Chen, B., Williamson, D.F.K., Chen, R.J., Liang, I., Ding, T., Jaume, G., Odintsov, I., Le, L.P., Gerber, G., Parwani, A.V., Zhang, A., Mahmood, F.: A multimodal generative AI copilot for human pathology. Nature629, 818–826 (2024).https://doi.org/10.1038/s41586-024-07618-3

work page doi:10.1038/s41586-024-07618-3 2024

[17] [17]

Nature Communications15(1), 9858 (2024)

Pizurica, M., Zheng, Y., Carrillo-Perez, F., Noor, H., Yao, W., Wohlfart, C., Vladimirova, A., Marchal, K., Gevaert, O.: Digital profiling of gene expression from histology images with linearized attention. Nature Communications15(1), 9858 (2024)

2024

[18] [18]

Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision. In: Int. Conf. Mach. Learn. pp. 8748–8763 (2021)

2021

[19] [19]

Nature Communications11, 3877 (2020)

Schmauch, B., Romagnoni, A., Pronier, E., Saillard, C., Maille, P., Calderaro, J., Kamoun, A., Sefta, M., Toldo, S., Zaslavskiy, M., Clozel, T., Moarii, M., Courtiol, P., Wainrib, G.: A deep learning model to predict RNA-Seq expression of tumours from whole slide images. Nature Communications11, 3877 (2020)

2020

[20] [20]

Molecular-driven foundation model for oncologic pathology.arXiv preprint arXiv:2501.16652, 2025

Vaidya, A., Zhang, A., Jaume, G., Song, A.H., Ding, T., Wagner, S.J., Lu, M.Y., Doucet, P., Robertson, H., Almagro-Perez, C., et al.: Molecular-driven foundation model for oncologic pathology. arXiv preprint arXiv:2501.16652 (2025)

work page arXiv 2025

[21] [21]

arXiv preprint arXiv:2408.09554 (2024)

Wang, Y.K., Tydlitatova, L., Kunz, J.D., Oakley, G., Chow, B.K.B., Godrich, R.A., Lee, M.C., Aghdam, H., Bozkurt, A., Zelechowski, M., et al.: Screen them all: high-throughput pan-cancer genetic and phenotypic biomarker screening from h&e whole slide images. arXiv preprint arXiv:2408.09554 (2024)

work page arXiv 2024

[22] [22]

Nature Genetics45, 1113–1120 (2013)

Weinstein, J.N., Collisson, E.A., Mills, G.B., Shaw, K.R.M., Ozenberger, B.A., Ellrott, K., Shmulevich, I., Sander, C., Stuart, J.M., The Cancer Genome Atlas Research Network: The cancer genome atlas pan-cancer analysis project. Nature Genetics45, 1113–1120 (2013)

2013