pith. sign in

arxiv: 2605.00925 · v1 · submitted 2026-04-30 · 💻 cs.LG · cs.CV· q-bio.QM

Linking spatial biology and clinical histology via Haiku

Yan Cui , Jacob S. Leiby , Wenhui Lei , Dokyoon Kim , Yanxiang Deng , Aaron T. Mayer , Zhenqin Wu , Alexandro E. Trevino
show 1 more author
Zhi Huang
This is my paper

Pith reviewed 2026-05-09 19:52 UTC · model grok-4.3

classification 💻 cs.LG cs.CVq-bio.QM
keywords tri-modal contrastive learningspatial proteomicscross-modal retrievalzero-shot biomarker inferencecounterfactual predictionhistologyclinical metadatasurvival prediction
0
0 comments X

The pith

Haiku aligns spatial proteomics, H&E histology, and clinical metadata in a shared embedding space to enable three-way retrieval, improved predictions, and zero-shot biomarker inference.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Haiku is a contrastive learning model trained on 26.7 million spatial proteomics patches from mIF, matched H&E images, and clinical metadata across 1,606 patients and 11 organs. It learns a joint representation that supports retrieving any modality from the others and outperforms single-modality baselines on downstream tasks. The alignment also permits inferring molecular biomarker levels directly from text descriptions of clinical metadata and running counterfactual tests that change only the metadata to surface associated molecular shifts. These capabilities matter because they connect data types routinely collected in clinics with high-dimensional molecular measurements that are harder to obtain.

Core claim

Haiku is a tri-modal contrastive model that embeds multiplexed immunofluorescence spatial proteomics patches, hematoxylin and eosin histology, and clinical metadata into one space, achieving Recall@50 of 0.611 for cross-modal retrieval, C-index of 0.737 for survival prediction, and mean Pearson correlation of 0.718 for zero-shot biomarker inference across 52 markers, plus exploratory counterfactual molecular predictions from metadata changes alone.

What carries the argument

Tri-modal contrastive learning that aligns mIF proteomics patches, H&E images, and clinical metadata text in a shared embedding space.

Load-bearing premise

The shared embedding space encodes genuine predictive relationships between clinical metadata and molecular features rather than dataset-specific correlations.

What would settle it

In an independent patient cohort, apply the counterfactual metadata modifications and check whether the predicted molecular shifts match the actual measured biomarker levels in those patients.

Figures

Figures reproduced from arXiv: 2605.00925 by Aaron T. Mayer, Alexandro E. Trevino, Dokyoon Kim, Jacob S. Leiby, Wenhui Lei, Yan Cui, Yanxiang Deng, Zhenqin Wu, Zhi Huang.

Figure 1
Figure 1. Figure 1: Dataset composition and overview of the Haiku framework. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Cross-modality alignment, retrieval, and zero-shot evaluation. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Downstream linear probing and slice-level prediction tasks. [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Zero-shot fusion retrieval–based biomarker inference. [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Metadata-only counterfactual analysis of breast cancer progression dynamics. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Metadata-only counterfactual analysis of survival outcome in lung cancer. a, Overview of the zero-shot counterfactual workflow for an in silico survival intervention on a single lung adenocarcinoma patient (Deceased, sur￾vival 25 months). For each H&E query patch, fusion retrieval is performed using the original metadata-only text (“Deceased”) against a reference atlas containing original deceased patients… view at source ↗
read the original abstract

Integrating molecular, morphological, and clinical data is essential for basic and translational biomedical research, yet systematic frameworks for jointly modeling these modalities remain limited. Here we present Haiku, a tri-modal contrastive learning model trained on multiplexed immunofluorescence (mIF). It comprises 26.7 million spatial proteomics patches from 3,218 tissue sections across 1,606 patients spanning 11 organ types, with matched hematoxylin and eosin (H&E) histology and clinical metadata aligned in a shared embedding space. Haiku enables three-way cross-modal retrieval, improves downstream classification and clinical prediction tasks over unimodal baselines, and supports zero-shot biomarker inference through fusion retrieval conditioned on clinical metadata-only text descriptions. Across tasks, Haiku outperforms competing approaches, achieving cross-modal retrieval (Recall@50 up to 0.611 versus near-zero baseline), survival prediction (C-index 0.737, +7.91% relative improvement), and zero-shot biomarker inference (mean Pearson correlation 0.718 across 52 biomarkers). Furthermore, we introduce a counterfactual prediction framework in which modifying only clinical metadata while fixing tissue morphology surfaces niche-specific molecular shifts associated with breast cancer stage progression and lung cancer survival outcomes. In a lung adenocarcinoma case study, the counterfactual analysis recovers niche-specific shifts characterized by increased CD8 and granzyme B, reduced PD-L1, and decreased Ki67, broadly consistent with patterns reported for favorable outcomes. We present these counterfactual results as exploratory, hypothesis-generating signals rather than mechanistic claims. These capabilities demonstrate that tri-modal alignment via Haiku enables integrative analysis of spatial biology, bridging molecular measurements with clinical context for biological exploration.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper introduces Haiku, a tri-modal contrastive learning model trained on 26.7 million mIF spatial proteomics patches from 3,218 tissue sections across 1,606 patients and 11 organs, with aligned H&E histology and clinical metadata. It claims to enable three-way cross-modal retrieval, outperform unimodal baselines on classification and survival prediction, support zero-shot biomarker inference via metadata-conditioned fusion retrieval, and generate exploratory counterfactual molecular predictions by editing only clinical text while holding morphology fixed. Key results include Recall@50 up to 0.611 for retrieval, C-index 0.737 (+7.91%) for survival, and mean Pearson correlation 0.718 across 52 biomarkers, with niche-specific shifts (e.g., increased CD8/granzyme B, reduced PD-L1/Ki67) in lung/breast case studies presented as hypothesis-generating.

Significance. If the alignment is robust and the counterfactuals hold under proper validation, Haiku would represent a meaningful advance in multi-modal spatial biology by providing a scalable shared embedding for integrative retrieval and prediction across molecular, morphological, and clinical data at unprecedented scale. The tri-modal contrastive approach and large patient cohort are strengths that could facilitate hypothesis generation in oncology beyond current unimodal or bimodal methods.

major comments (3)
  1. [Abstract] Abstract: The headline performance numbers (Recall@50 = 0.611, C-index = 0.737, Pearson = 0.718) are reported without any description of training procedures, patient-level validation splits, statistical testing, or controls for data leakage across the 1,606 patients; this absence directly undermines assessment of whether the gains are robust or artifactual.
  2. [Abstract] Abstract (counterfactual prediction framework): The claim that editing clinical metadata alone produces meaningful niche-specific molecular shifts (e.g., CD8/granzyme B up, PD-L1/Ki67 down in lung adenocarcinoma) rests on the embedding capturing predictive rather than confounded co-occurrence structure, yet no disentanglement loss, confounder adjustment, or causal identification strategy is described; the observational nature of the 11-organ dataset makes this a load-bearing assumption for the zero-shot inference results.
  3. [Abstract] Abstract: The zero-shot biomarker inference via 'fusion retrieval conditioned on clinical metadata-only text descriptions' is presented as achieving mean Pearson 0.718, but without details on how the conditioning is implemented or how held-out biomarkers are selected, it is impossible to evaluate whether this reduces to in-sample correlation rather than true generalization.
minor comments (2)
  1. [Methods] Clarify the precise encoder architectures and contrastive loss formulation for the three modalities (mIF, H&E, text metadata) to allow reproducibility.
  2. [Abstract] The term 'fusion retrieval' is used without a formal definition or pseudocode; a brief algorithmic description would improve clarity.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for their constructive and detailed feedback. We address each major comment point by point below. The full manuscript contains the requested methodological details, but we agree that the abstract would benefit from additional clarification and will revise accordingly.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The headline performance numbers (Recall@50 = 0.611, C-index = 0.737, Pearson = 0.718) are reported without any description of training procedures, patient-level validation splits, statistical testing, or controls for data leakage across the 1,606 patients; this absence directly undermines assessment of whether the gains are robust or artifactual.

    Authors: We appreciate the referee's emphasis on transparency for the headline metrics. While the abstract is length-constrained, the full manuscript details the training procedures (including contrastive loss and optimization), patient-level cross-validation splits across the 1,606 patients to prevent leakage, statistical testing, and controls in the Methods and Experimental Setup sections. We will revise the abstract to include a brief clause noting the use of patient-level validation and statistical evaluation. revision: yes

  2. Referee: [Abstract] Abstract (counterfactual prediction framework): The claim that editing clinical metadata alone produces meaningful niche-specific molecular shifts (e.g., CD8/granzyme B up, PD-L1/Ki67 down in lung adenocarcinoma) rests on the embedding capturing predictive rather than confounded co-occurrence structure, yet no disentanglement loss, confounder adjustment, or causal identification strategy is described; the observational nature of the 11-organ dataset makes this a load-bearing assumption for the zero-shot inference results.

    Authors: We agree that the dataset is observational and that no explicit disentanglement loss or causal identification strategy is employed. The counterfactuals are generated by holding the morphology embedding fixed while varying only the clinical text embedding; the manuscript already qualifies these results as 'exploratory, hypothesis-generating signals rather than mechanistic claims.' We will expand the Discussion to further articulate the assumptions, potential confounding, and limitations of this framework. revision: partial

  3. Referee: [Abstract] Abstract: The zero-shot biomarker inference via 'fusion retrieval conditioned on clinical metadata-only text descriptions' is presented as achieving mean Pearson 0.718, but without details on how the conditioning is implemented or how held-out biomarkers are selected, it is impossible to evaluate whether this reduces to in-sample correlation rather than true generalization.

    Authors: The fusion retrieval implementation, metadata-only conditioning mechanism, and selection of held-out biomarkers for zero-shot evaluation are described in the Methods section of the full manuscript. We will add a short clarifying phrase to the abstract describing the conditioning approach and held-out evaluation to make this explicit without exceeding length limits. revision: yes

Circularity Check

0 steps flagged

No significant circularity; model trained on data and evaluated on held-out tasks

full rationale

The derivation consists of training a tri-modal contrastive embedding on 26.7 M patches and then measuring empirical performance on retrieval, classification, survival prediction, and zero-shot biomarker tasks using held-out data. No equation or claim reduces a prediction to a fitted parameter by construction, no self-citation supplies a load-bearing uniqueness result, and the counterfactual framework is explicitly labeled exploratory rather than mechanistic. All reported metrics (Recall@50, C-index, Pearson correlation) are external evaluations, not definitional identities.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on standard contrastive learning assumptions and the representativeness of the curated multi-organ dataset; no new physical entities are introduced.

axioms (1)
  • domain assumption Contrastive learning can produce a semantically meaningful shared embedding space across molecular, imaging, and metadata modalities
    Core premise of the Haiku model; standard in multimodal contrastive frameworks but unproven for this specific tri-modal spatial biology setting.

pith-pipeline@v0.9.0 · 5630 in / 1338 out tokens · 47321 ms · 2026-05-09T19:52:54.004736+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

57 extracted references · 57 canonical work pages

  1. [1]

    J., Ding, T., Lu, M

    Chen, R. J., Ding, T., Lu, M. Y., Williamson, D. F. K., Jaume, G., Song, A. H., Chen, B., Zhang, A., Shao, D., Shaban, M., Williams, M., Oldenburg, L., Weishaupt, L. L., Wang, J. J., Vaidya, A., Le, L. P ., Gerber, G., Sahai, S., Williams, W. & Mahmood, F. Towards a general-purpose foundation model for computational pathology. en.Nat. Med.30,850–862 (Mar. 2024)

    work page 2024

  2. [2]

    J., Song, A

    Ding, T., Wagner, S. J., Song, A. H., Chen, R. J., Lu, M. Y., Zhang, A., Vaidya, A. J., Jaume, G., Shaban, M., Kim, A., Williamson, D. F. K., Robertson, H., Chen, B., Almagro-PÃl’rez, C., Doucet, P ., Sahai, S., Chen, C., Chen, C. S., Komura, D., Kawabe, A., Ochi, M., Sato, S., Yokose, T., Miyagi, Y., Ishikawa, S., Gerber, G., Peng, T., Le, L. P . & Mahmo...

    work page 2025

  3. [3]

    & Huang, Z

    Li, S., Xu, J., Bao, T., Liu, Y., Liu, Y., Liu, Y., Wang, L., Lei, W., Wang, S., Xu, Y., Cui, Y., Yao, J., Koga, S. & Huang, Z. A co-evolving agentic AI system for medical imaging analysis.arXiv [cs.CV](Sept. 2025)

    work page 2025

  4. [4]

    L., Pak, J

    Swanson, K., Wu, W., Bulaong, N. L., Pak, J. E. & Zou, J. The Virtual Lab of AI agents designs new SARS-CoV-2 nanobodies. en.Nature646,716–723 (Oct. 2025)

    work page 2025

  5. [5]

    N., Zhou, X., Wheeler, M., Bernstein, J

    Huang, K., Zhang, S., Wang, H., Qu, Y., Lu, Y., Roohani, Y., Li, R., Qiu, L., Li, G., Zhang, J., Yin, D., Marwaha, S., Carter, J. N., Zhou, X., Wheeler, M., Bernstein, J. A., Wang, M., He, P ., Zhou, J., Snyder, M., Cong, L., Regev, A. & Leskovec, J. Biomni: A general-purpose biomedical AI agent. en.bioRxivorg, 2025.05.30.656746 (June 2025)

    work page 2025

  6. [6]

    G., Geneslaw, L., Miraflor, A., Werneck Krauss Silva, V ., Busam, K

    Campanella, G., Hanna, M. G., Geneslaw, L., Miraflor, A., Werneck Krauss Silva, V ., Busam, K. J., Brogi, E., Reuter, V . E., Klimstra, D. S. & Fuchs, T. J. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. en.Nat. Med.25,1301–1309 (Aug. 2019)

    work page 2019

  7. [7]

    H., Choi, H.-K., Kim, D., Lee, J.-Y., Nam, E

    Ahn, B., Moon, D., Kim, H.-S., Lee, C., Cho, N. H., Choi, H.-K., Kim, D., Lee, J.-Y., Nam, E. J., Won, D., An, H. J., Kwon, S. Y., Shin, S.-J., Jung, H. R., Kwon, D., Park, H., Kim, M., Cha, Y. J., Park, H., Lee, Y., Noh, S., Lee, Y.-M., Choi, S.-E., Kim, J. M., Sung, S. H. & Park, E. Histopathologic image-based deep learning classifier for predicting pla...

    work page 2024

  8. [8]

    & Huang, J

    Yao, J., Zhu, X., Jonnagaddala, J., Hawkins, N. & Huang, J. Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks. en.Med. Image Anal.65, 101789 (Oct. 2020)

    work page 2020

  9. [9]

    W., Kennedy-Darling, J., Venkataraaman, V

    Black, S., Phillips, D., Hickey, J. W., Kennedy-Darling, J., Venkataraaman, V . G., Samusik, N., Golt- sev, Y., SchÃijrch, C. M. & Nolan, G. P . CODEX multiplexed tissue imaging with DNA-conjugated antibodies. en.Nat. Protoc.16,3802–3835 (Aug. 2021)

    work page 2021

  10. [10]

    & Keren, L

    Elhanani, O., Ben-Uri, R. & Keren, L. Spatial profiling technologies illuminate the tumor microenvi- ronment. en.Cancer Cell41,404–420 (Mar. 2023)

    work page 2023

  11. [11]

    Shaitelman, S. F. & Woodward, W. A. Neoadjuvant radioimmunotherapy synergy in triple-negative breast cancer: Is microenvironment-guided patient selection on the horizon? en.Cancer Cell42,10–12 (Jan. 2024)

    work page 2024

  12. [12]

    S., Januszyk, M., Delitto, D., Yost, K

    Foster, D. S., Januszyk, M., Delitto, D., Yost, K. E., Griffin, M., Guo, J., Guardino, N., Delitto, A. E., Chinta, M., Burcham, A. R., Nguyen, A. T., Bauer-Rowe, K. E., Titan, A. L., Salhotra, A., Jones, R. E., da Silva, O., Lindsay, H. G., Berry, C. E., Chen, K., Henn, D., Mascharak, S., Talbott, H. E., Kim, A., Nosrati, F., Sivaraj, D., Ransom, R. C., M...

    work page 2022

  13. [13]

    & Cao, Z

    Chen, H., Deng, C., Gao, J., Wang, J., Fu, F., Wang, Y., Wang, Q., Zhang, M., Zhang, S., Fan, F., Liu, K., Yang, B., He, Q., Zheng, Q., Shen, X., Wang, J., Hu, T., Zhu, C., Yang, F., He, Y., Hu, H., Wang, J., Li, Y., Zhang, Y. & Cao, Z. Integrative spatial analysis reveals tumor heterogeneity and immune colony niche related to clinical outcomes in small c...

    work page 2025

  14. [14]

    F., Liakopoulos, P ., Michielin, O., Wicki, A., Gut, G

    Wenckstern, J., Jain, E., Cheng, Y., von Querfurth, B., Vasilev, K., Pariset, M., Cheng, P . F., Liakopoulos, P ., Michielin, O., Wicki, A., Gut, G. & Bunne, C. AI-powered virtual tissues from spatial proteomics for clinical diagnostics and biomedical discovery.arXiv [q-bio.QM](Dec. 2025)

    work page 2025

  15. [15]

    Y., Song, A

    Shaban, M., Chang, Y., Qiu, H., Yeo, Y. Y., Song, A. H., Jaume, G., Wang, Y., Weishaupt, L. L., Ding, T., Vaidya, A.,et al.A foundation model for spatial proteomics.arXiv preprint arXiv:2506.03373(2025)

    work page arXiv 2025

  16. [16]

    Modeling patient tissues at molecular resolution with Eva.bioRxiv,2025–12 (2025)

    Liu, Y., Sharma, R., Bieniosek, M., Kang, A., Wu, E., Chou, P ., Li, I., Rahim, M., Bauer, E., Ji, R.,et al. Modeling patient tissues at molecular resolution with Eva.bioRxiv,2025–12 (2025)

    work page 2025

  17. [17]

    Huang, Z., Bianchi, F., Yuksekgonul, M., Montine, T. J. & Zou, J. A visual-language foundation model for pathology image analysis using medical Twitter. en.Nat. Med.29,2307–2316 (Sept. 2023)

    work page 2023

  18. [18]

    Y., Chen, B., Williamson, D

    Lu, M. Y., Chen, B., Williamson, D. F. K., Chen, R. J., Liang, I., Ding, T., Jaume, G., Odintsov, I., Le, L. P ., Gerber, G., Parwani, A. V ., Zhang, A. & Mahmood, F. A visual-language foundation model for computational pathology. en.Nat. Med.30,863–874 (Mar. 2024)

    work page 2024

  19. [19]

    M., Nirschl, J

    Xiang, J., Wang, X., Zhang, X., Xi, Y., Eweje, F., Chen, Y., Li, Y., Bergstrom, C., Gopaulchan, M., Kim, T., Yu, K.-H., Willens, S., Olguin, F. M., Nirschl, J. J., Neal, J., Diehn, M., Yang, S. & Li, R. A vision-language foundation model for precision oncology. en.Nature638,769–778 (Feb. 2025)

    work page 2025

  20. [20]

    W., Makky, A., SchÃijrch, C

    Wu, E., Bieniosek, M., Wu, Z., Thakkar, N., Charville, G. W., Makky, A., SchÃijrch, C. M., Huyghe, J. R., Peters, U., Li, C. I., Li, L., Giba, H., Behera, V ., Raman, A., Trevino, A. E., Mayer, A. T. & Zou, J. ROSIE: AI generation of multiplex immunofluorescence staining from histopathology images. en.Nat. Commun.16,7633 (Aug. 2025)

    work page 2025

  21. [21]

    Valanarasu, J. M. J., Xu, H., Usuyama, N., Kim, C., Wong, C., Argaw, P ., Ben Shimol, R., Crabtree, A., Matlock, K., Bartlett, A. Q., Bagga, J., Gu, Y., Zhang, S., Naumann, T., Fox, B. A., Wright, B., Robicsek, A., Piening, B., Bifulco, C., Wang, S. & Poon, H. Multimodal AI generates virtual population for tumor microenvironment modeling. en.Cell(Dec. 2025)

    work page 2025

  22. [22]

    M., Willens, S., Nirschl, J

    Li, Z., Li, Y., Xiang, J., Wang, X., Yang, S., Zhang, X., Eweje, F., Chen, Y., Luo, X., Li, Y., Mulholland, J., Bergstrom, C., Kim, T., Olguin, F. M., Willens, S., Nirschl, J. J., West, R., Neal, J., Diehn, M. & Li, R. AI-enabled virtual spatial proteomics from histopathology for interpretable biomarker discovery in lung cancer. en.Nat. Med.32,231–244 (Jan. 2026)

    work page 2026

  23. [23]

    N., Xiao, Y., Li, S., Shah, V

    Chen, W., Zhang, P ., Tran, T. N., Xiao, Y., Li, S., Shah, V . V ., Cheng, H., Brannan, K. W., Youker, K., Lai, L., Fang, L., Yang, Y., Le, N.-T., Abe, J.-I., Chen, S.-H., Ma, Q., Chen, K., Song, Q., Cooke, J. P . & Wang, G. A visual-omics foundation model to bridge histopathology with spatial transcriptomics. en. Nat. Methods22,1568–1582 (July 2025)

    work page 2025

  24. [24]

    R., Zhang, J., Dillon, D., Lin, N

    Wang, X., Zhao, J., Marostica, E., Yuan, W., Jin, J., Zhang, J., Li, R., Tang, H., Wang, K., Li, Y., Wang, F., Peng, Y., Zhu, J., Zhang, J., Jackson, C. R., Zhang, J., Dillon, D., Lin, N. U., Sholl, L., Denize, T., Meredith, D., Ligon, K. L., Signoretti, S., Ogino, S., Golden, J. A., Nasrallah, M. P ., Han, X., Yang, S. & Yu, K.-H. A pathology foundation ...

    work page 2024

  25. [25]

    T., Luo, R., Trevino, A

    Yiyao, L., Vakharia, N., Liang, W., Mayer, A. T., Luo, R., Trevino, A. E. & Wu, Z. OmicsNavigator: an LLM-driven multi-agent system for autonomous zero-shot biological analysis in spatial omics. en. bioRxiv,2025.07.21.665821 (July 2025)

    work page 2025

  26. [26]

    & Poon, H

    Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J. & Poon, H. Domain- specific language model pretraining for biomedical natural language processing.ACM Transactions on Computing for Healthcare3,1–23 (Oct. 2021)

    work page 2021

  27. [27]

    W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P ., Clark, J., Krueger, G

    Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P ., Clark, J., Krueger, G. & Sutskever, I. Learning transferable visual models from natural language supervision.arXiv [cs.CV](Feb. 2021)

    work page 2021

  28. [28]

    J., Robicsek, A., Piening, B., Bifulco, C., Wang, S

    Xu, H., Usuyama, N., Bagga, J., Zhang, S., Rao, R., Naumann, T., Wong, C., Gero, Z., Gonzà ˛ alez, J., Gu, Y., Xu, Y., Wei, M., Wang, W., Ma, S., Wei, F., Yang, J., Li, C., Gao, J., Rosemon, J., Bower, T., Lee, S., Weerasinghe, R., Wright, B. J., Robicsek, A., Piening, B., Bifulco, C., Wang, S. & Poon, H. A whole-slide foundation model for digital patholo...

    work page 2024

  29. [29]

    Ni, C., Yang, L., Xu, Q., Yuan, H., Wang, W., Xia, W., Gong, D., Zhang, W. & Yu, K. CD68- and CD163- positive tumor infiltrating macrophages in non-metastatic breast cancer: a retrospective study and meta-analysis.Journal of Cancer10,4463–4472 (2019)

    work page 2019

  30. [30]

    F., Berghoff, A., Dinhof, C., Jakesz, R., Gnant, M., Dubsky, P ., Jesch, B., Heinzl, H

    Schoppmann, S. F., Berghoff, A., Dinhof, C., Jakesz, R., Gnant, M., Dubsky, P ., Jesch, B., Heinzl, H. & Birner, P . Podoplanin-expressing cancer-associated fibroblasts are associated with poor prognosis in invasive breast cancer.Breast Cancer Research and Treatment134,237–244 (2012)

    work page 2012

  31. [31]

    K., Slorach, E

    Kouros-Mehr, H., Bechis, S. K., Slorach, E. M., Littlepage, L. E., Egeblad, M., Ewald, A. J., Pai, S.-Y., Ho, I.-C. & Werb, Z. GATA-3 Links Tumor Differentiation and Dissemination in a Luminal Breast Cancer Model.Cancer Cell13,141–152 (2008)

    work page 2008

  32. [32]

    & Cimino-Mathews, A

    Asch-Kendrick, R. & Cimino-Mathews, A. The role of GATA3 in breast carcinomas: a review.Human Pathology48,37–47 (2016)

    work page 2016

  33. [33]

    J., Arenas, R

    Yan, W., Cao, Q. J., Arenas, R. B., Bentley, B. & Shao, R. GATA3 inhibits breast cancer metastasis through the reversal of epithelial-mesenchymal transition.Journal of Biological Chemistry285,14042– 14051 (2010)

    work page 2010

  34. [34]

    Q., Danenberg, E., Huang, C.-S., Egle, D., Callari, M., Bermejo, B., Dugo, M., Zamagni, C., Thill, M., Anton, A., Zambelli, S., Russo, S., Ciruelos, E

    Wang, X. Q., Danenberg, E., Huang, C.-S., Egle, D., Callari, M., Bermejo, B., Dugo, M., Zamagni, C., Thill, M., Anton, A., Zambelli, S., Russo, S., Ciruelos, E. M., Greil, R., Gyorffy, B., Semiglazov, V ., Colleoni, M., Kelly, C. M., Mariani, G., Del Mastro, L., Maffeis, O., Valagussa, P ., Viale, G., Gianni, L., Bianchini, G., Magbanua, M. J. M. & Ali, H...

    work page 2023

  35. [35]

    A., Avalos, C., Tu, T

    Egelston, C. A., Avalos, C., Tu, T. Y., Simons, D. L., Jimenez, G., Jung, J. Y., Melstrom, L., Margolin, K., Yim, J. H., Kruper, L., Mortimer, J. & Lee, P . P . Human breast tumor-infiltrating CD8+ T cells retain polyfunctionality despite PD-1 expression.Nature Communications9,4297 (2018)

    work page 2018

  36. [36]

    L., Danforth, D

    Goff, S. L., Danforth, D. N., Schaer, D. A.,et al.CD4 + and CD8+ T cells have opposing roles in breast cancer progression and outcome.Oncotarget6,32797–32813 (2015)

    work page 2015

  37. [37]

    & Wang, S

    Hu, G. & Wang, S. Tumor-infiltrating CD45RO + memory T lymphocytes predict favorable clinical outcome in solid tumors.Scientific Reports7,10376 (2017)

    work page 2017

  38. [38]

    C., Harview, C

    Tumeh, P . C., Harview, C. L., Yearley, J. H., Shintaku, I. P ., Taylor, E. J. M., Robert, L., Chmielowski, B., Spasic, M., Henry, G., Ciobanu, V ., West, A. N., Carmona, M., Kivork, C., Seja, E., Cherry, G., Gutierrez, A. J., Grogan, T. R., Mateus, C., Tober, G., Pisegna, D., Buitrago, G., Connelly, A., Elashoff, R., Ribas, A. & Comin-Anduix, B. PD-1 blo...

    work page 2014

  39. [39]

    & Pagès, F

    Galon, J., Costes, A., Sanchez-Cabo, F., Kirilovsky, A., Mlecnik, B., Lagorce-Pagès, C., Tosolini, M., Ca- mus, M., Berger, A., Wind, P ., Zinzindohoué, F., Bruneval, P ., Cugnenc, P .-H., Trajanoski, Z., Fridman, W.-H. & Pagès, F. Type, density, and location of immune cells within human colorectal tumors predict clinical outcome.Science313,1960–1964 (2006)

    work page 1960

  40. [40]

    H., Pagès, F., Sautès-Fridman, C

    Fridman, W. H., Pagès, F., Sautès-Fridman, C. & Galon, J. The immune contexture in human tumours: impact on clinical outcome.Nature Reviews Cancer12,298–306 (2012)

    work page 2012

  41. [41]

    Pagès, F., Berger, A., Camus, M.,et al.Effector memory T cells, early metastasis, and survival in col- orectal cancer.New England Journal of Medicine353,2654–2666 (2005)

    work page 2005

  42. [42]

    Pardoll, D. M. The blockade of immune checkpoints in cancer immunotherapy.Nature Reviews Cancer 12,252–264 (2012)

    work page 2012

  43. [43]

    C., Joller, N

    Anderson, A. C., Joller, N. & Kuchroo, V . K. Lag-3, Tim-3, and TIGIT: Co-inhibitory receptors with specialized functions in immune regulation.Immunity44,989–1004 (2016)

    work page 2016

  44. [44]

    S., Giaccone, G., de Marinis, F., Reinmuth, N., Vergnenegre, A., Barrios, C

    Herbst, R. S., Giaccone, G., de Marinis, F., Reinmuth, N., Vergnenegre, A., Barrios, C. H., Morise, M., Felip, E., Andric, Z., Geater, S.,et al.Atezolizumab for first-line treatment of PD-L1–selected patients with NSCLC.N. Engl. J. Med.383,1328–1339 (2020)

    work page 2020

  45. [45]

    & Donnadieu, E

    Salmon, H., Franciszkiewicz, K., Damotte, D., Dieu-Nosjean, M.-C., Validire, P ., Trautmann, A., Mami- Chouaib, F. & Donnadieu, E. Matrix architecture defines the preferential localization and migration of T cells into the stroma of human lung tumors.J. Clin. Invest.122,899–910 (2012). 39 Linking H&E, mIF, and text in a unified framework

    work page 2012

  46. [46]

    & Allavena, P

    Mantovani, A., Marchesi, F., Malesci, A., Laghi, L. & Allavena, P . Tumour-associated macrophages as treatment targets in oncology.Nature Reviews Clinical Oncology14,399–416 (2017)

    work page 2017

  47. [47]

    Dieu-Nosjean, M.-C., Antoine, M., Danel, C., Heber, D., Wislez, M., Poulot, V ., Rabbe, N., Laurans, L., Tartour, E., de Châtelet, L.,et al.Long-term survival for patients with non-small-cell lung cancer with intratumoral lymphoid structures.J. Clin. Oncol.26,4410–4417 (2008)

    work page 2008

  48. [48]

    A., Svane, I

    Cabrita, R., Lauss, M., Sanna, A., Donia, M., Skaarup Larsen, M., Mitra, S., Johansson, I., Phung, B., Harbst, K., Vallon-Christersson, J., van Schoiack, A., Lövgren, K., Warren, S., Jirström, K., Olsson, H., Pietras, K., Ingvar, C., Isaksson, K., Schadendorf, D., Schmidt, H., Bastholt, L., Carneiro, A., Wargo, J. A., Svane, I. M. & Jönsson, G. Tertiary l...

    work page 2020

  49. [49]

    Joyce, J. A. & Fearon, D. T. T cell exclusion, immune privilege, and the tumor microenvironment. Science348,74–80 (2015)

    work page 2015

  50. [50]

    & Zeisberg, M

    Kalluri, R. & Zeisberg, M. Fibroblasts in cancer.Nat. Rev. Cancer6,392–401 (2006)

    work page 2006

  51. [51]

    X., Cui, H., Zhang, A

    Wang, C. X., Cui, H., Zhang, A. H., Xie, R., Goodarzi, H. & Wang, B. scGPT-spatial: Continual Pretrain- ing of Single-Cell Foundation Model for Spatial Transcriptomics.bioRxiv,2025.02.05.636714 (2025)

    work page 2025

  52. [52]

    C., Tejada-Lapuerta, A., Palla, G., Gutgesell, R., Halle, L., Minaeva, M., Vornholz, L., Dony, L., Drummer, F., Bahrami, M

    Schaar, A. C., Tejada-Lapuerta, A., Palla, G., Gutgesell, R., Halle, L., Minaeva, M., Vornholz, L., Dony, L., Drummer, F., Bahrami, M. & Theis, F. J. Nicheformer: a foundation model for single-cell and spatial omics.Nature Methods(2025)

    work page 2025

  53. [53]

    & Jin, W

    Huang, T., Liu, T., Babadi, M., Ying, R. & Jin, W. STPath: a generative foundation model for integrating spatial transcriptomics and whole-slide images. en.NPJ Digit. Med.8,659 (Nov. 2025)

    work page 2025

  54. [54]

    & Isola, P

    Huh, M., Cheung, B., Wang, T. & Isola, P . The platonic representation hypothesis.arXiv [cs.LG](May 2024)

    work page 2024

  55. [55]

    N., Cui, Y., Jin, D., Hua, Y., Xu, X

    Guo, P ., Chen, Y., Mao, L., Cardilla, A., Lee, C. N., Cui, Y., Jin, D., Hua, Y., Xu, X. & Deng, Y. Spatial profiling of chromatin accessibility in formalin-fixed paraffin-embedded tissues. en.Nat. Commun.16, 5945 (July 2025)

    work page 2025

  56. [56]

    & Sun, J.Deep residual learning for image recognitioninProceedings of the IEEE conference on computer vision and pattern recognition(2016), 770–778

    He, K., Zhang, X., Ren, S. & Sun, J.Deep residual learning for image recognitioninProceedings of the IEEE conference on computer vision and pattern recognition(2016), 770–778

    work page 2016

  57. [57]

    J., Oktay, D., Lin, Z., Verkuil, R., Tran, V

    Hayes, T., Rao, R., Akin, H., Sofroniew, N. J., Oktay, D., Lin, Z., Verkuil, R., Tran, V . Q., Deaton, J., Wiggert, M., Badkundri, R., Shafkat, I., Gong, J., Derry, A., Molina, R. S., Thomas, N., Khan, Y. A., Mishra, C., Kim, C., Bartie, L. J., Nemeth, M., Hsu, P . D., Sercu, T., Candido, S. & Rives, A. Simulating 500 million years of evolution with a lan...

    work page 2025