Pith. sign in

REVIEW 4 major objections 4 minor 3 cited by

A Foundation Model for Spatial Proteomics

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read KRONOS claims that one self-supervised vision transformer, pretrained on 47 million multiplexed tissue image patches, can serve as a universal backbone for spatial proteomics, beating task-specific baselines across 11 independent cohorts…

desk verdict A serious resource with strong cell phenotyping results, but the arbitrary-marker-panel claim rests on an unablated 3-to-50 channel shift and the clinical overclaims need trimming. read the letter →

arxiv 2506.03373 v1 pith:7AUP7YEF submitted 2025-06-03 cs.CV cs.AI

Muhammad Shaban , Yuzhou Chang , Huaying Qiu , Yao Yu Yeo , Andrew H. Song , Guillaume Jaume , Yuchen Wang , Luca L. Weishaupt
show 52 more authors
Tong Ding Anurag Vaidya Abdallah Lamane Daniel Shao Mohammed Zidane Yunhao Bai Paige McCallum Shuli Luo Wenrui Wu Yang Wang Precious Cramer Chi Ngai Chan Pierre Stephan Johanna Schaffenrath Jia Le Lee Hendrik A. Michel Caiwei Tian Cristina Almagro-Perez Sophia J. Wagner Sharifa Sahai Ming Y. Lu Richard J. Chen Andrew Zhang Mark Edward M. Gonzales Ahmad Makky Jia-Ying Joey Lee Hao Cheng Nourhan El Ahmar Sayed Matar Maximilian Haist Darci Phillips Yuqi Tan Garry P. Nolan W. Richard Burack Jacob D. Estes Jonathan T.C. Liu Toni K Choueiri Neeraj Agarwal Marc Barry Scott J. Rodig Long Phi Le Georg Gerber Christian M. Schürch Fabian J. Theis Youn H Kim Joe Yeong Sabina Signoretti Brooke E. Howitt Lit-Hsin Loo Qin Ma Sizun Jiang Faisal Mahmood
This is my paper
classification cs.CVcs.AI
keywords spatialproteomicsfoundationmodelself-supervisedlearningVisionTransformermultiplexedimagingcellphenotypingimageretrievalmarkerencoding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

KRONOS is a single self-supervised vision transformer that the authors train on 47 million image patches of multiplexed tissue, covering 175 protein markers, 16 tissue types, and 8 fluorescence imaging platforms. The paper's claim is that this one pretrained backbone, without task-specific retraining, produces representations that beat existing general-purpose and in-domain baselines for cell phenotyping, treatment-response prediction, and image retrieval across 11 independent cohorts, while needing far fewer labeled cells. If that holds, spatial proteomics—an imaging family with expensive, study-specific annotations—gains a shared starting point that lowers annotation cost, absorbs batch variation across institutions, and lets researchers search tissue images the way web images are searched today. The authors also argue that segmentation is not required: patch-level embeddings at the scale of a cell can replace the traditional segment-then-average pipeline.

What carries the argument

The load-bearing mechanism is the combination of a shared convolutional token-embedding layer with a non-learnable sinusoidal marker encoding, Eq. (2), plus learnable spatial positional encodings, inside a ViT-Small trained with DINO-v2. The shared conv weights mean every channel is treated by the same feature extractor; the marker encoding lets the transformer tell channels apart and, because it is sinusoidal in the marker index, gives new markers a free identity at inference by mapping them to an unused index. Pretraining batches are limited to three channels per image—one nuclear stain and two randomly drawn protein markers—to balance diversity and memory. Downstream, marker-specific embeddings (averaging tokens per marker) and token-specific embeddings (concatenating across markers per location) support both cell-level and pixel-dense tasks. This arrangement is what lets one fixed model switch from 3 to 43 markers without architectural change.

What would settle it

Zero out all but the nuclear channel plus two protein markers during inference on a full-panel cohort: if cell-phenotyping accuracy is unchanged, the extra channels are irrelevant, and if it collapses, the three-marker pretraining does not compose to full panels. A second check is to retrain the same model with six markers per batch and see whether downstream accuracy moves at all.

Watch

Extended reading notes

Core claim

The central discovery is that a marker-aware Vision Transformer pretrained with DINO-v2-style self-distillation and masked-image modeling on heterogeneous multiplexed tissue can learn general cell, region, and case-level representations. Each marker channel is embedded by a shared convolutional stem, tagged with a fixed sinusoidal marker index and a 2D positional index, and then processed jointly by the transformer. Because the marker encoding is not learned, an unseen marker can be introduced at inference simply by assigning it the next unused index, and the model accepts any number of channels in any order. Evaluated by linear probing, these frozen embeddings reach balanced accuracies of 0.74–0.80 on 9- to 16-class cell phenotyping, roughly 15–20 points above the best baselines; cross-dataset transfer, few-shot, zero-shot, segmentation-free phenotyping, and retrieval results follow the same pattern. The authors present KRONOS as evidence that a single generalist backbone can unify spatial proteomics analysis.

Load-bearing premise

The model sees only three markers at a time during pretraining but is trusted to work when a dozen or more markers are shown all at once, and the paper never directly tests whether what it learns in triplets survives that leap.

Editorial extensions

If this is right

  • Cell phenotyping with 100 labeled cells per class approaches the accuracy of 1,000 labels, so large annotation campaigns could be cut by an order of magnitude.
  • A classifier trained on one DLBCL cohort transfers to a second cohort collected elsewhere with only a small accuracy drop, supporting cross-institutional use without re-annotation.
  • Patch-level, segmentation-free phenotyping reaches a balanced accuracy of 0.72 on patches whose majority cell type covers the whole patch, suggesting cell segmentation may become optional for many analyses.
  • Embedding-space retrieval can act as a reverse image search engine that finds patches with similar cell composition and even matches EBV-positive and EBV-negative tumor microenvironments across datasets.
  • Batch-effect experiments on 24 protocol variations show KRONOS embeddings are less separated by technical condition than mean marker expression, pointing toward harmonization across staining protocols.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper never ablates the jump from 3-marker pretraining to full-panel inference; a natural test is to zero out all but three channels at inference and check whether accuracy is invariant. If it is, the marker encoding may be doing little beyond channel identification.
  • Because marker identity is an arbitrary index, two markers swapped between indices should produce identical embeddings for the same tissue; biological similarity between markers plays no role. This suggests a latent opportunity to replace indices with embeddings derived from protein sequence or antibody specificity.
  • The triplet design (nuclear stain plus two proteins) may under-represent co-expression patterns among protein pairs; pretraining with, say, six channels per batch would directly test whether joint protein context matters.
  • A similar architecture could be extended to ion-based imaging (IMC/MIBI) if its single-channel-per-acquisition output is reorganized into a channel stack, but the paper's own limitations section notes those modalities are not covered.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces KRONOS, a self-supervised Vision Transformer foundation model for spatial proteomics. The model is pretrained on SPM-47M, a corpus of 47 million single-marker patches spanning 175 protein markers, 16 tissue types, and 8 fluorescence imaging platforms. To handle variable marker counts, KRONOS uses a shared convolutional filter across markers and a sinusoidal marker-identity encoding. The authors evaluate KRONOS on cell phenotyping (three cohorts, up to 16 classes), few-shot and zero-shot phenotyping, patch-level segmentation-free phenotyping, region classification, artifact detection, unsupervised tissue clustering, patient stratification, and cell/patch/case retrieval, comparing against DINO-v2, UNI, CA-MAE, and mean marker expression. The central claim is that a single self-supervised backbone transfers to diverse downstream tasks, achieving state-of-the-art performance and high label efficiency across 11 cohorts. The model and code are publicly released.

Significance. If the central claims hold, KRONOS would be an important contribution to spatial proteomics: a generalist, segmentation-free backbone that reduces annotation cost, supports cross-institutional comparisons, and enables image search across studies. The benchmarking is unusually thorough in scope: multiple independently annotated cohorts, four-fold cross-validation with error bars, few-shot curves, cross-dataset transfer, ablations of architectural choices, and a batch-effect study. The consistent and large margins over strong baselines in cell phenotyping are notable. However, the significance is tempered by two load-bearing concerns: pretraining is performed with exactly three channels while all full-panel evaluation uses 12-50+ channels without an ablation of this mismatch, and the paper does not state that evaluation cohorts were excluded from the pretraining corpus. These issues directly affect the interpretation of the headline 'state-of-the-art' claim.

major comments (4)
  1. [Online Methods, 'Pretraining batch preparation'; Supplementary Table S4] Pretraining uses exactly three channels per image—one nuclear stain and two randomly selected protein markers—while the cell phenotyping evaluations use 12-18 markers (Supplementary Table S4) and patient stratification uses more than 50 markers. Because self-attention mixes tokens across all channels, the contextualized representation of any marker depends on every other marker in the input. At inference, the attention context jumps from 3 to 12-50 channels, a distribution shift the model has never seen. The claim that 'KRONOS can encode any multiplex image with an arbitrarily large number of markers' is asserted without an ablation varying the number of channels at pretraining or inference. The full-panel results should be accompanied by an experiment that systematically increases the number of channels at inference, or a fine-tuning/adaptation step that closes this gap.
  2. [Data and code availability; Figure 1A; 'Downstream datasets'] The manuscript nowhere states that the evaluation cohorts were held out from the SPM-47M pretraining corpus. The cHL benchmarking dataset is publicly available on Zenodo, and the DLBCL cohorts are from BIDMC, which is listed in Table S1 as one of the pretraining data sources. If any of the 11 'independent' evaluation cohorts overlap with pretraining, the reported benchmark gains could reflect memorization rather than generalization. The authors must report a clear dataset-level exclusion policy and ideally provide a list of cohort-to-pretraining-source overlaps.
  3. [Figure 5D; Table S20; Abstract] The patient stratification results on CTCL-P are near chance: KRONOS achieves AUC 0.5900 ± 0.0793 (PCA-64), with a 95% CI that overlaps that of DINO-v2 (0.4400 ± 0.0850) and UNI (0.3550 ± 0.0804) at the upper end. The abstract's claim of 'state-of-the-art performance across ... treatment response prediction' overstates these weak results. The authors should either temper the claim, report paired statistical significance tests across the 100 splits, or restrict the SOTA statement to the datasets where the margin is robust.
  4. [Eq. (2); 'KRONOS modeling'] Assigning a novel, unseen marker to an arbitrary unused sinusoidal index means that marker identity is represented only by an integer position in the encoding, with no biological grounding and no learned embedding for that marker. The model's ability to 'encode' new markers at inference is therefore untested: the sinusoidal encoding for a marker index never seen in pretraining will produce a vector that may be nonsensical to the initialized weights. An ablation that adds held-out markers with random indices and measures downstream task performance would be necessary to support the generalization claim.
minor comments (4)
  1. [Online Methods, 'Downstream datasets', DLBCL-2] The text says '31,098,443 cells' but Table S3 reports 1,098,443 total cells; this is likely a typo and should be corrected.
  2. [Supplementary Table S24] The hyperparameter table lists 'Patch size 224x224' under KRONOS, yet the main text says input patches are 256x256 and tokens are 16x16; please clarify the relationship between the pretraining input size, token size, and the downstream patch extraction.
  3. [Online Methods, 'Unsupervised clustering of tissue'] There are spelling errors in the methods text: 'startification' should be 'stratification' and 'strafied' should be 'stratified'.
  4. [Figure 4D] The scatter plot is labeled 'CLR' and 'DII' in the legend, but the axes or the panel could be clearer about which principal component corresponds to which axis; a legend entry for the two case types would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: KRONOS is pretrained self-supervised without downstream labels, and the reported evaluations are external benchmarks; the 3-channel-to-full-panel gap is an untested assumption, not a circular derivation.

full rationale

The central claim of KRONOS—that self-supervised pretraining on SPM-47M produces transferable spatial-proteomics representations—is not circular. Downstream labels (cell phenotypes, treatment response, EBV status) are never used during pretraining, and no fitted parameter defines the reported accuracy numbers. The representation-learning pipeline is self-contained: Eq. (1) defines a shared convolutional embedding, Eq. (2) defines sinusoidal marker encodings, and the pretraining objective is DINO-v2 self-distillation plus masked image modeling, not a regression toward the evaluation targets. Baselines (DINO-v2, UNI, CA-MAE, mean marker expression) are external comparators evaluated under the same linear-probing protocol. The main weaknesses are robustness and evaluation-validity concerns rather than circularity. First, pretraining batches contain exactly three channels ('Each batch is built to include three markers per image: one nuclear stain marker and two randomly selected protein markers'), while evaluation uses 12-18 or more than 50 markers, and the claim that 'KRONOS can encode any multiplex image with an arbitrarily large number of markers' is asserted without a channel-count ablation; this is an untested distribution-shift assumption, not a logical reduction to the model's inputs. Second, applying SPM-47M normalization statistics to the DINO-v2/UNI/CA-MAE baselines may disadvantage those baselines, but this is a fairness concern, not circularity. Third, several cited datasets and baseline models involve overlapping authors (e.g., UNI, MAPS/cHL), but those citations are used as data sources or comparator models, not to justify the correctness or uniqueness of KRONOS's architecture. No self-definitional step, fitted parameter renamed as a prediction, or load-bearing self-citation chain was found.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

KRONOS is an empirical model, so the ledger lists the design choices and domain assumptions the central claim rests on, rather than physical constants. The main burden is the unablated assumption that a model pretrained on triplets transfers to full 43-marker panels, plus the arbitrary marker-index encoding that has no biological grounding.

free parameters (4)
  • PCA dimension for patient stratification = 64 or 256 (per dataset)
    The 100-split evaluation is repeated for PCA-64 and PCA-256 and the better configuration per dataset is highlighted; this is a form of selection on test splits, though both are reported.
  • Number of markers per pretraining batch = 3 (DAPI + 2 markers)
    Chosen to balance diversity and memory ('Limiting the input to three channels'); not varied in ablations, yet the model's transfer to larger panels depends on it.
  • Cluster count K in CRC case classification = 4, 8, 16 (best at 8)
    K-means K is swept and the best value reported; K=8 also matches the prior CRC study's nine neighborhoods, so selection is partly motivated by prior knowledge.
  • Token size and overlap = 16x16 tokens with no overlap (pretraining)
    Ablation shows 4x4 tokens improve accuracy but can be matched by 16x16 with 50% overlap; final pretraining uses 16x16 as described in Methods, a hand-chosen trade-off.
assumptions (5)
  • domain assumption DINO-v2 self-supervised pretraining transfers to downstream tasks
    Taken from prior histology and natural image results (refs 17, 27); not re-derived here, but the entire paper is an empirical test of this assumption.
  • ad hoc to paper Shared convolutional filter across markers captures marker-specific features
    Eq. 1 forces all markers through the same weights; claimed to enable any marker count, but cross-marker interactions are only learned through attention.
  • ad hoc to paper Sinusoidal marker index encodes marker identity
    Eq. 2 assigns a marker a code based solely on its position in an arbitrary ordering; no marker name, ontology or protein embedding is used, and index permutation robustness is not tested.
  • ad hoc to paper Triplet pretraining generalizes to full marker panels
    Pretraining always uses 3 channels; downstream evaluation uses 12-43 channels; no experiment varies the number of channels at pretraining to validate this transfer.
  • domain assumption Manual marker quality control is trustworthy
    Staining quality was judged by manual review against vendor or Human Protein Atlas data; this is subjective and could affect which data enter SPM-47M.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Foundation Model for Spatial Proteomics." pith.science (2026). https://pith.science/paper/7AUP7YEF

@misc{pith2026250603373,
  author       = {Pith},
  title        = {Pith review of: A Foundation Model for Spatial Proteomics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7AUP7YEF}},
  note         = {Machine review of arXiv:2506.03373}
}
read the original abstract

Foundation models have begun to transform image analysis by acting as pretrained generalist backbones that can be adapted to many tasks even when post-training data are limited, yet their impact on spatial proteomics, imaging that maps proteins at single-cell resolution, remains limited. Here, we introduce KRONOS, a foundation model built for spatial proteomics. KRONOS was trained in a self-supervised manner on over 47 million image patches covering 175 protein markers, 16 tissue types, and 8 fluorescence-based imaging platforms. We introduce key architectural adaptations to address the high-dimensional, multi-channel, and heterogeneous nature of multiplex imaging. We demonstrate that KRONOS learns biologically meaningful representations across multiple scales, ranging from cellular and microenvironment to tissue levels, enabling it to address diverse downstream tasks, including cell phenotyping, region classification, and patient stratification. Evaluated across 11 independent cohorts, KRONOS achieves state-of-the-art performance across cell phenotyping, treatment response prediction, and retrieval tasks, and is highly data-efficient. KRONOS also introduces the paradigm of segmentation-free patch-level processing for efficient and scalable spatial proteomics analysis, allowing cross-institutional comparisons, and as an image reverse search engine for spatial patterns. Together, these results position KRONOS as a flexible and scalable tool for spatial proteomics. The model is publicly accessible at https://github.com/mahmoodlab/KRONOS.

Figures

Figures reproduced from arXiv: 2506.03373 by the authors.

Figure 1
Figure 1. Overview of KRONOS: A foundation model for multiplex spatial proteomics. A, KRONOS was pretrained on SPM-47M, a collection of multiplexed image sources from 30 cohorts and eight fluorescence￾based imaging platforms. B, SPM-47M comprises over 47 million single-marker image patches derived from 175 unique protein markers and 16 distinct tissue types. C, KRONOS is built on a Vision Transformer, DINO￾v2, adapted to mult… view at source ↗
Figure 2
Figure 2. Benchmarking KRONOS for cell phenotyping across three spatial proteomics datasets us￾ing linear probing evaluation. A, Overall performance comparison across three spatial proteomics datasets: 16-class classical Hodgkin lymphoma (cHL, n=134,552 cells) and two 9-class diffuse large B cell lymphoma datasets (DLBCL-1, n=361,341 cells and DLBCL-2, n=1,098,443 cells). B, Class-wise average precision com￾parison of KRONOS … view at source ↗
Figure 3
Figure 3. Comparison of KRONOS against marker expression embeddings for patch phenotyping, re￾gion classification, and artifact detection. A,B,C KRONOS for segmentation-free patch phenotyping. B: Patch phenotyping performance comparison on the DLBCL-1 dataset. Patch labels were assigned based on the majority cell phenotype within each patch. C, Visual comparison of KRONOS predictions against the ground truth phenotype map for… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Unsupervised tissue phenotyping and classification with KRONOS. A, Overview of KRONOS for unsupervised clustering of multiplex images followed by k-nearest neighbor (kNN) classification from cluster distributions. B, Examples of clusters from classical Hodgkin lymphoma…
Figure 5
Figure 5. Figure 5: Patient stratification with KRONOS. A, Overview of KRONOS for patient-level stratification us￾ing multiple instance learning (MIL) classification. B, ROC-AUC classification of KRONOS against UNI and DINO-v2 for predicting response to immune checkpoint–based combination…
Figure 6
Figure 6. Figure 6: Image retrieval using KRONOS embeddings. A, Cell-level retrieval measured with Accuracy@k (higher the better) on the cHL, DLBCL-1, and DLBCL-2 cohorts. Retrieval is deemed successful if at least one of the top-k retrieved images is of the same cell type as the query. A…

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 4 citations worldwide. Full citation record

  1. Evaluating Vision Foundation Models for Pixel and Object Classification in Microscopy

    cs.CV 2026-03 conditional novelty 6.0 of 10

    Vision foundation model features improve pixel and object classification in microscopy over hand-crafted features, and object-guided attentive probing (ObAP) can match or beat supervised baselines with very few labels.

  2. Towards Robust Foundation Models for Digital Pathology

    eess.IV 2025-07 conditional novelty 6.0 of 10

    PathoROB shows that all 20 evaluated pathology foundation models encode medical center information and that lower robustness correlates with larger downstream performance drops.

  3. Emerging AI Approaches for Cancer Spatial Omics

    q-bio.QM 2025-06 unverdicted novelty 2.0 of 10

    A review that groups AI methods for cancer spatial omics into data-driven, constraint-based, and mechanistic modeling paradigms, calling for more interpretable models and mouse-model-generated perturbational data.

Reference graph

Works this paper leans on

79 extracted references · 64 canonical work pages · cited by 3 Pith papers

  1. [1]

    Karimi, E. et al. Method of the year 2024: spatial proteomics. Nat Methods 21, 2195–2196 (2024)

    work page 2024

  2. [2]

    & Bodenmiller, B

    de Souza, N., Zhao, S. & Bodenmiller, B. Multiplex protein imaging in tumour biology. Nat. Rev. Cancer 24, 171–191 (2024)

    work page 2024

  3. [3]

    & Borner, G

    Lundberg, E. & Borner, G. H. Spatial proteomics: a powerful discovery tool for cell biology. Nature Reviews Molecular Cell Biology 20, 285–302 (2019)

    work page 2019

  4. [4]

    Lewis, S. M. et al. Spatial omics and multiplexed imaging to explore cancer biology. Nature methods 18, 997–1012 (2021)

    work page 2021

  5. [5]

    Wu, Z. et al. Graph deep learning for the characterization of tumour microenvironments from spatial protein profiles in tissue specimens. Nature Biomedical Engineering 6, 1435–1448 (2022)

    work page 2022

  6. [6]

    & Bodenmiller, B

    de Souza, N., Zhao, S. & Bodenmiller, B. Multiplex protein imaging in tumour biology. Nature Reviews Cancer 24, 171–191 (2024)

    work page 2024

  7. [7]

    Y .et al

    Yeo, Y . Y .et al. A hitchhiker’s guide to high-dimensional tissue imaging with multiplexed ion beam imaging. Methods in cell biology 186, 213 (2024)

    work page 2024

  8. [8]

    Jackson, H. W. et al. The single-cell pathology landscape of breast cancer. Nature 578, 615–620 (2020)

    work page 2020

Show all 79 references
  1. [9]

    Sch ¨urch, C. M. et al. Coordinated cellular neighborhoods orchestrate antitumoral immunity at the colorectal cancer invasive front. Cell 182, 1341–1359 (2020)

    2020

  2. [11]

    Zhu, B. et al. A multi-omics spatial framework for host-microbiome dissection within the intestinal tissue microenvironment. Nature Communications 16, 1230 (2025)

    2025

  3. [12]

    Y .et al

    Yeo, Y . Y .et al. Epstein-Barr Virus Orchestrates Spatial Reorganization and Immunomodulation within the Classic Hodgkin Lymphoma Tumor Microenvironment. bioRxiv [Preprint]. 2024.03.05.583586. (2024). 38496566

    2024

  4. [13]

    Lin, J.-R. et al. Multiplexed 3D atlas of state transitions and immune interaction in colorectal cancer. Cell 186, 363–381 (2023)

    2023

  5. [14]

    Keren, L. et al. A structured tumor-immune microenvironment in triple negative breast cancer revealed by multiplexed ion beam imaging. Cell 174, 1373–1387 (2018)

    2018

  6. [15]

    Jiang, S. et al. Combined protein and nucleic acid imaging reveals virus-dependent B cell and macrophage immunosuppression of tissue microenvironments. Immunity 55, 1118–1134 (2022)

    2022

  7. [16]

    Song, A. H. et al. Artificial intelligence for digital and computational pathology. Nature Reviews Bioengineering 1, 930–949 (2023)

    2023

  8. [17]

    Chen, R. J. et al. Towards a general-purpose foundation model for computational pathology. Nature Medicine 30, 850–862 (2024)

    2024

  9. [18]

    Xu, H. et al. A whole-slide foundation model for digital pathology from real-world data. Nature 1–8 (2024). 39

    2024

  10. [19]

    Virchow: A million-slide digital pathology foundation model

    V orontsov, E.et al. Virchow: A million-slide digital pathology foundation model. arXiv preprint arXiv:2309.07778 (2023)

    2023 arXiv

  11. [20]

    Shaikovski, G. et al. PRISM: A Multi-Modal Generative Foundation Model for Slide-Level Histopathology. arXiv preprint arXiv:2405.10254 (2024)

    2024 arXiv

  12. [21]

    Filiot, A. et al. Scaling self-supervised learning for histopathology with masked image modeling. medRxiv 2023–07 (2023)

    2023

  13. [22]

    & Saillard, C

    Filiot, A., Jacob, P., Mac Kain, A. & Saillard, C. Phikon-v2, a large and public feature extractor for biomarker prediction. arXiv preprint arXiv:2409.09173 (2024)

    2024 arXiv

  14. [23]

    Wang, X. et al. A pathology foundation model for cancer diagnosis and prognosis prediction. Nature 634, 970–978 (2024)

    2024

  15. [24]

    Ding, T. et al. Multimodal whole slide foundation model for pathology. arXiv preprint arXiv:2411.19666 (2024)

    2024 arXiv

  16. [25]

    Zhao, T. et al. A foundation model for joint segmentation, detection and recognition of biomedical objects across nine modalities. Nature methods 1–11 (2024)

    2024

  17. [26]

    & Wang, Y

    Xun, D., Wang, R., Zhang, X. & Wang, Y . Microsnoop: A generalist tool for microscopy image representation. The Innovation 5 (2024)

    2024

  18. [27]

    Oquab, M. et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 (2023)

    2023 arXiv

  19. [28]

    Radtke, A. J. et al. Ibex: an iterative immunolabeling and chemical bleaching method for high-content imaging of diverse tissues. Nature protocols 17, 378–401 (2022)

    2022

  20. [29]

    Radtke, A. J. et al. Multi-omic profiling of follicular lymphoma reveals changes in tissue architecture and enhanced stromal remodeling in high-risk patients. Cancer cell 42, 444–463 (2024)

    2024

  21. [30]

    The human body at cellular resolution: the NIH Human Biomolecular Atlas Program

    HuBMAPConsortium. The human body at cellular resolution: the NIH Human Biomolecular Atlas Program. Nature 574, 187–192 (2019)

    2019

  22. [31]

    Lee, J. Y . J., Yeong, J., Lee, L. W. J. N., Loo, L.-H. & Dong, J. 627 immunoatlas: an online public portal for sharing, visualizing, and referencing multiplex immunohistochemistry/immunofluorescence (mihc/if) images and results for immuno-oncology. Journal for Immunotherapy o...

    2021

  23. [32]

    Yeo, E. L. L. et al. Tumor immune microenvironment delineates progression trajectories of distinct nasopharyngeal carcinoma phenotypes. Cell Reports Medicine

  24. [33]

    Deep Profiling of Mouse Splenic Architecture with CODEX Multiplexed Imaging

    Goltsev, Y .et al. Deep Profiling of Mouse Splenic Architecture with CODEX Multiplexed Imaging. Cell 174, 968–981.e15 (2018)

    2018

  25. [34]

    Parra, E. R. et al. Validation of multiplex immunofluorescence panels using multispectral microscopy for immune-profiling of formalin-fixed and paraffin-embedded human tumor tissues. Sci. Rep. 7, 1–11 (2017)

    2017

  26. [35]

    Rivest, F. et al. Fully automated sequential immunofluorescence (seqIF) for hyperplex spatial proteomics. Sci. Rep. 13, 1–14 (2023)

    2023

  27. [36]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Dosovitskiy, A. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020). 40

    2020 arXiv

  28. [38]

    Caron, M. et al. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision , 9650–9660 (2021)

    2021

  29. [39]

    Zhou, J. et al. ibot: Image bert pre-training with online tokenizer. arXiv preprint arXiv:2111.07832 (2021)

    2021 arXiv

  30. [40]

    Assran, M. et al. Self-supervised learning from images with a joint-embedding predictive architecture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 15619–15629 (2023)

    2023

  31. [41]

    & Karaletsos, T

    Bao, Y ., Sivanandan, S. & Karaletsos, T. Channel vision transformers: An image is worth c x 16 x 16 words. arXiv preprint arXiv:2309.16108 (2023)

    2023 arXiv

  32. [42]

    Bourriez, N. et al. Chada-vit: Channel adaptive attention for joint representation learning of heterogeneous microscopy images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 11556–11565 (2024)

    2024

  33. [43]

    & Snijder, B

    Pfaendler, R., Hanimann, J., Lee, S. & Snijder, B. Self-supervised vision transformers accurately decode cellular state heterogeneity. Biorxiv 2023–01 (2023)

    2023

  34. [44]

    & Plummer, B

    Pham, C. & Plummer, B. Enhancing feature diversity boosts channel-adaptive vision transformers. Advances in Neural Information Processing Systems 37, 89782–89805 (2024)

    2024

  35. [45]

    Doron, M. et al. Unbiased single-cell morphology with self-supervised vision transformers. bioRxiv (2023)

    2023

  36. [46]

    X., Kraus, O

    Lu, A. X., Kraus, O. Z., Cooper, S. & Moses, A. M. Learning unsupervised feature representations for single cell microscopy images with paired cell inpainting. PLoS computational biology 15, e1007348 (2019)

    2019

  37. [47]

    Le, T. et al. Analysis of the human protein atlas weakly supervised single-cell classification competition. Nature methods 19, 1221–1229 (2022)

    2022

  38. [48]

    Viana, M. P. et al. Integrated intracellular organization and its variations in human iPS cells. Nature 613, 345–354 (2023)

    2023

  39. [49]

    Moshkov, N. et al. Learning representations for image-based profiling of perturbations. Nature communications 15, 1594 (2024)

    2024

  40. [50]

    C., Leonetti, M

    Kobayashi, H., Cheveralls, K. C., Leonetti, M. D. & Royer, L. A. Self-supervised deep learning encodes high-resolution features of protein subcellular localization. Nature methods 19, 995–1003 (2022)

    2022

  41. [51]

    Chen, Z. S. et al. Chammi: A benchmark for channel-adaptive models in microscopy imaging. Advances in Neural Information Processing Systems 36 (2024)

    2024

  42. [52]

    Kraus, O. et al. Masked autoencoders for microscopy are scalable learners of cellular biology. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 11757–11768 (2024)

    2024

  43. [53]

    Kenyon-Dean, K. et al. Vitally consistent: Scaling biological representation learning for cell microscopy. arXiv preprint arXiv:2411.02572 (2024). 41

    2024 arXiv

  44. [54]

    Wenckstern, J. et al. AI-powered virtual tissues from spatial proteomics for clinical diagnostics and biomedical discovery. arXiv preprint arXiv:2501.06039 (2025)

    2025

  45. [55]

    He, K. et al. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 16000–16009 (2022)

    2022

  46. [56]

    Giesen, C. et al. Highly multiplexed imaging of tumor tissues with subcellular resolution by mass cytometry. Nat. Methods 11, 417–422 (2014)

    2014

  47. [57]

    Magness, A. et al. Deep cell phenotyping and spatial analysis of multiplexed imaging with tracerx-phlex. Nature Communications 15, 5135 (2024)

    2024

  48. [58]

    Shaban, M. et al. MAPS: pathologist-level cell type annotation from tissue images through machine learning. Nature Communications 15, 28 (2024)

    2024

  49. [60]

    Fay, M. M. et al. Rxrx3: Phenomics map of biology. Biorxiv 2023–02 (2023)

    2023

  50. [61]

    Shaban, M. et al. Data for MAPS: Pathologist-level Cell Type Annotation from Tissue Images through Machine Learning. https://zenodo.org/ (2023). Accessed: 2025-05-22

    2023

  51. [62]

    Y .et al

    Yeo, Y . Y .et al. Same-slide spatial multi-omics integration reveals tumor virus-linked spatial reorganization of the tumor microenvironment. bioRxiv (2024)

    2024

  52. [63]

    CellSighter: a neural network to classify cells in highly multiplexed images

    Amitay, Y .et al. CellSighter: a neural network to classify cells in highly multiplexed images. Nat. Commun. 14, 1–13 (2023)

    2023

  53. [64]

    Zhang, W. et al. Comparison of QuPath and HALO platforms for analysis of the tumor microenvironment in prostate cancer. bioRxiv 2025.05.16.654590 (2025). URL https://doi.org/10.1101/2025.05.16.654590. 2025.05.16.654590

    2025 doi

  54. [65]

    & Pachitariu, M

    Stringer, C., Wang, T., Michaelos, M. & Pachitariu, M. Cellpose: a generalist algorithm for cellular segmentation. Nature methods 18, 100–106 (2021)

    2021

  55. [66]

    Greenwald, N. F. et al. Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning. Nature biotechnology 40, 555–565 (2022)

    2022

  56. [67]

    Bai, Y . et al. Adjacent cell marker lateral spillover compensation and reinforcement for multiplexed images. Frontiers in immunology 12, 652631 (2021)

    2021

  57. [68]

    Baker, G. J. et al. Quality control for single-cell analysis of high-plex tissue profiles using CyLinter. Nature Methods 21, 2248–2259 (2024)

    2024

  58. [69]

    Zhu, B. et al. Celllens enables cross-domain information fusion for enhanced cell population delineation in single-cell spatial omics data. Nature Immunology (2025)

    2025

  59. [70]

    & Welling, M

    Ilse, M., Tomczak, J. & Welling, M. Attention-based deep multiple instance learning. In Proceedings of the 35th International Conference on Machine Learning , 2132–2141 (2018)

    2018

  60. [71]

    Lu, M. Y . et al. Data-efficient and weakly supervised computational pathology on whole-slide images. Nature biomedical engineering 5, 555–570 (2021). 42

    2021

  61. [72]

    Phillips, D. et al. Immune cell topography predicts response to PD-1 blockade in cutaneous T cell lymphoma. Nat. Commun. 12, 1–18 (2021)

    2021

  62. [73]

    Y .et al

    Yeo, Y . Y .et al. Same-Slide Spatial Multi-Omics Integration Reveals Tumor Virus-Linked Spatial Reorganization of the Tumor Microenvironment. bioRxiv [Preprint]. 2024.12.20.629650. (2024). 39764057

    2024

  63. [74]

    & Dieu-Nosjean, M.-C

    Teillaud, J.-L., Houel, A., Panouillot, M., Riffard, C. & Dieu-Nosjean, M.-C. Tertiary lymphoid structures in anticancer immunity. Nat. Rev. Cancer 24, 629–646 (2024). 39117919

    2024

  64. [75]

    Pelka, K. et al. Spatially organized multicellular immune hubs in human colorectal cancer. Cell 184, 4734–4752.e20 (2021)

    2021

  65. [76]

    Lee, J. L. & Schaffenrath, J. Benchmarking tissue preparation pilotdata (2024). URL https://doi.org/10.5281/zenodo.11410274

    2024 doi

  66. [77]

    Angelo, M. et al. Multiplexed ion beam imaging of human breast tumors. Nat. Med. 20, 436–442 (2014)

    2014

  67. [78]

    Vaswani, A. et al. Attention is all you need. Advances in neural information processing systems 30 (2017)

    2017

  68. [79]

    & Ivanova, E

    Nechaev, D., Pchelnikov, A. & Ivanova, E. Hibou: A family of foundational vision transformers for pathology (2024). URL https://arxiv. org/abs/2406.05074

    2024 arXiv

  69. [80]

    Zimmermann, E. et al. Virchow2: Scaling self-supervised mixed magnification models in pathology. arXiv preprint arXiv:2408.00738 (2024)

    2024 arXiv

  70. [81]

    Chen, X., Xie, S. & He, K. An empirical study of training self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision , 9640–9649 (2021)

    2021

  71. [82]

    S. N., C. et al. JUMP Cell Painting dataset: morphological impact of 136,000 chemical and genetic perturbations (2023). [Online; accessed 29. May 2025]. 43 Multiplexed image View 1 View 2 Global crops Local crops Masked global crops Channel-wise token embedding Student modelTe...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.