Pith. sign in

REVIEW 3 major objections 6 minor 3 cited by

Segment Anything in Pathology Images with Natural Language

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read One model beats per-dataset specialists across 160 pathology categories.

desk verdict A serious, mostly solid engineering paper built around a valuable 275k-triple pathology segmentation benchmark, with real caveats about the label hierarchy's semantic limits and a missing pathology-specific baseline. read the letter →

arxiv 2506.20988 v2 pith:GYRKV46K submitted 2025-06-26 cs.CV cs.AI

Zhixuan Chen , Junlin Hou , Liqi Lin , Yihui Wang , Yequan Bie , Xi Wang , Yanning Zhou , Ronald Cheong Kin Chan
show 1 more author
Hao Chen
This is my paper
classification cs.CVcs.AI
keywords foundationmodeltextpromptpathologyimagesemanticsegmentationexplainablecancerdiagnosisPathSegdatasethierarchicallabels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that one compact, text-prompted segmentation model can cover the breadth of pathology image analysis that currently requires many specialized models. It introduces PathSeg, a dataset of 275,000 image-mask-label triples merged from 21 public sources into 160 hierarchical categories, and PathSegmentor, which takes a natural-language phrase such as "cell-level neutrophil in unspecified pathology" and outputs a semantic mask. On the internal benchmark the single PathSegmentor reaches an overall Dice of 0.671, above the 0.502 of a group of 16 dataset-tuned nnU-Nets, and it improves on the text-prompted baseline BiomedParse by 0.429 Dice. The authors also show the same segmentation masks can be used to explain a breast-cancer subtyping classifier by estimating which pathological objects drive its predictions. If these results hold, text-prompted segmentation becomes a practical route to flexible, explainable pathology AI without per-dataset retraining.

What carries the argument

The load-bearing design is the three-level semantic hierarchy [anatomical region]-[histological structure]-[object type], which does double duty: it is the taxonomy that merges 21 datasets into 160 consistent category names, and it is the prompt template the model is trained and queried with. The second component is the joint feature interaction module, a set of learnable queries that first cross-attend to image features, then self-attend while concatenated with the text features, so that textual category semantics directly guide mask generation. The final mask is chosen by matching class embeddings to the prompt's text embedding via cosine similarity, which is what lets a single forward pass convert a sentence into a pixel-level prediction.

What would settle it

Have pathologists independently re-annotate a stratified sample of the PathSeg test masks for the labels the paper admits are ambiguous — starting with 'tumor' across breast and colon, and all 'unspecified'-region categories — and recompute the overall Dice of PathSegmentor against nnU-Net on the images where the hierarchy's assignment survives the re-annotation. If the 0.169-point advantage shrinks or vanishes on the verified subset, the reported gain is an artifact of label merging rather than a genuine segmentation ability.

Watch

Extended reading notes

Core claim

PathSegmentor claims that pathology segmentation can be reframed as a single text-conditioned task: with a prompt assembled from the template "[histological structure]-level [object type] in [anatomical region] pathology," a single 0.45B-parameter model predicts semantic masks across 160 categories spanning 20 anatomical regions, 3 histological structures, and 61 object types. Built on the SEEM encoder-decoder design, it fuses image features from a FocalNet encoder and text features from PubMedBERT through a joint feature interaction module of learnable queries, and selects its output mask by cosine similarity between class embeddings and the prompt text. The paper reports that this one model beats 16 per-dataset specialists (0.671 vs 0.502 overall Dice for the nnU-Net group), surpasses spatial-prompted foundation models MedSAM (0.526) and SAM-Med2D (0.432), and exceeds the text-prompted BiomedParse (0.242) by a wide margin. The gain is largest on pathology's hardest objects — irregular, tiny, densely packed cells and nuclei — where the paper shows spatial-prompted models degrade sharply while PathSegmentor holds its score.

Load-bearing premise

The load-bearing premise is that the three-level hierarchy used to merge 21 datasets into 160 category names groups the same biological object under each name, even though the paper itself notes that a label like 'tumor' can mean tissue in one dataset and cells in another, and that many images carry only an 'unspecified' anatomical region.

Editorial extensions

If this is right

  • A single text-prompted pathology segmentation model can replace a group of per-dataset specialists, cutting deployed model count while covering roughly 40 times more categories.
  • Pathologists can obtain semantic masks for any of the 160 categories with one natural-language prompt; the paper estimates that spatial prompting requires about 15 times more prompts per mask.
  • Text-prompted segmentation stays accurate where spatial-prompted models collapse: irregular shapes, tiny instances, and dense fields of hundreds of nuclei or cells.
  • The same masks can be reused to explain cancer classifiers, through object-based feature importance and object-aware class activation maps that name the pathological biomarker driving a diagnosis.
  • External datasets from different laboratories and scanning conditions show the gains carry over to unseen images for the categories the model has learned.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the hierarchy is sound, the same region-structure-object decomposition could be pushed beyond 160 categories toward immunohistochemistry markers or subcellular structures by growing the dataset rather than retraining per task.
  • The paper's own CoNSeP result on sparse, clustered endothelial cells suggests a boundary of the text-prompted paradigm: spatial cues still win when instances are few and concentrated, pointing to hybrid prompt schemes as a natural next step.
  • The reported per-category gains imply that annotation scale, not model architecture, is the binding constraint in pathology segmentation, so semi-automated labeling with the model in the loop should compound the advantage.
  • Because prompts are text, the model can serve as the segmentation component of a language-driven pipeline in which clinical reports or free-form user questions are translated automatically into queries, making semantic maps a byproduct of normal diagnostic documentation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper presents two contributions: PathSeg, a curated collection of 275k image-mask-label triples from 21 public pathology datasets reorganized into 160 labels under a three-level [anatomical region]-[histological structure]-[object type] hierarchy, and PathSegmentor, a text-prompted segmentation foundation model built on SEEM with a FocalNet image encoder, PubMedBERT text encoder, and a joint feature interaction module. Experiments on 16 internal datasets report an overall Dice score of 0.671 for PathSegmentor, outperforming nnU-Net (0.502), DeepLabV3+ (0.462), SAM-Path (0.472), MedSAM (0.526), SAM-Med2D (0.432), and BiomedParse (0.242). External tests on 5 datasets and two explainability applications on TCGA-BRCA are also presented. The central claim is that a single text-prompted model can segment 160 pathology categories with higher accuracy and broader applicability than specialized and existing foundation models.

Significance. If the benchmark taxonomy is sound, this is a practically valuable contribution: PathSeg is a large aggregation of public annotations, the model is compact relative to a suite of specialized models, the experiments use bootstrap confidence intervals and paired tests, the external images are genuinely unseen, and the code is released. The interactive LLM-based use cases and the explainability pipelines are attractive extensions. However, the central claim depends on the semantic coherence of the 160-category hierarchy, and the paper's own Section 4.1 concedes that the hierarchy mixes tissue and cell levels for identically named entities and collapses anatomical regions into 'unspecified'. In addition, the claimed category advantage over specialized models is contradicted by the dataset statistics in Table A1. These issues make the headline numeric comparisons provisional until the benchmark semantics and the comparison protocol are clarified.

major comments (3)
  1. [Section 4.1, Table A5] The three-level label hierarchy is not a consistent semantic taxonomy by the paper's own admission. Section 4.1 states that identical labels may describe distinct biological entities across datasets (tumor can denote tumor tissue or tumor cells) and that cells annotated with region-wise masks are classified under the histological-structure level 'tissue'. Consequently, labels such as Breast-Tissue-Plasma cells, Breast-Tissue-Smooth muscle cells (as Unspecified-Tissue-Smooth muscle cells), and Breast-Tissue-Plasma cells in Table A5 are semantically inconsistent: they are cells placed at the tissue level. Similarly, the 'unspecified' anatomical region collapses distinct organs into one category. Since the 160 categories are the basis for training prompts, the benchmark itself, and the reported Dice scores, the headline improvement over specialized models may partly reflect the model learning dataset-specific label conventions rather than biologically coherent categories. Please provide a concrete validation of the hierarchy: for example, evaluate on a subset of labels where all contributing datasets agree on both the biological entity and the annotation granularity, exclude 'unspecified' regions and tissue-level cellular masks, and report whether the claimed margins over nnU-Net, MedSAM, and BiomedParse persist; alternatively, provide a pathologist consistency study of the label definitions.
  2. [Section 2.3.1, Fig. 3b, Table A1] The claim that each specialized model handles approximately 4 classes is contradicted by the paper's own dataset statistics. Section 2.3.1 states that specialized models each handle about 4 classes based on 61 object types across 16 models, but Table A1 reports per-dataset label counts such as PanNuke with 85 labels, BCSS with 18, NuCLS with 12, and SegPath with 8. A specialized model trained on PanNuke in a multi-class setting therefore has 85 output classes, not 4. The 40x category-scalability advantage of PathSegmentor is accordingly overstated and should be recomputed using the actual per-dataset class counts reported in Table A1, or the comparison should be restated in terms of the number of distinct benchmarks covered rather than classes per model.
  3. [Section 2.4, Tables A16-A17] The external evaluation is narrower than the text suggests. Section 2.4 explicitly states that the external categories were previously learned during model training, so the reported results demonstrate generalization to new images of known categories, not to novel categories. In addition, the external test sets are very small: CPM17 has 15 images, Lizard has 30, CPM15 has 64, and CoNSeP has 40. The bootstrap confidence intervals reflect this, for example CPM15 PathSegmentor Dice is 0.485 with 95% CI [0.365, 0.602]. Please temper the abstract and conclusion claims about 'generalizing to external datasets' to specify that only images, not categories, were unseen, and report sample sizes and confidence-interval widths in the main text when claiming state-of-the-art external performance.
minor comments (6)
  1. [Abstract and Section 2.3.3] The abstract's '0.145 and 0.429 improvements' do not identify the baselines; specify MedSAM and BiomedParse in the abstract or state them parenthetically, as is done in Section 2.3.3.
  2. [Section 4.4] Please disclose in Sections 2.3.3 and 4.4 that PathSegmentor is initialized from BiomedParse weights. This context is important for interpreting the large improvement over BiomedParse and for assessing the contribution of the new training data versus the initialization.
  3. [Section 2.3.2 and Figure 7] SegAnyPath is mentioned in the Introduction as a pathology-specific spatial-prompted foundation model but is not included in the baseline comparisons. Adding it to the internal or external evaluation would strengthen the claim that PathSegmentor advances the state of the art.
  4. [Section 2.4, Figure 7] The text references Fig. 7a-e before Fig. 7b and 7c, and the order of the panels in the description is confusing. Please reorder the panel references or the figure layout so that the narrative follows the visual order.
  5. [Section 4.5, Eq. (10)] There is a typo in the sentence after Eq. (10): 'epsilon is used for for numerical stability' should read 'epsilon is used for numerical stability.'
  6. [Section 4.4, Explainability] The Dice threshold of 0.4 used to select the 21 objects for imaging biomarker discovery is a free parameter; please provide an ablation or sensitivity analysis showing that the ranking of biomarker importance is stable with respect to this threshold.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PathSegmentor's central claims are empirical evaluations on held-out and external data, not derived from its own inputs.

full rationale

The paper's central results are benchmark measurements, not derivations. PathSegmentor is trained on PathSeg and evaluated on 20% held-out test splits of 16 internal datasets (Sec. 2.3 and Methods 4.1), and on five external datasets (CPM17, CPM15, Kumar, Lizard, CoNSeP) that are, in the paper's words, 'completely independent from the internal training datasets' (Sec. 2.4). Dice is computed by Eq. (18) as pixel overlap against ground truth, so the reported overall Dice score of 0.671 is not equivalent to any fitted parameter by construction. The model is initialized from BiomedParse weights (Sec. 4.4), but all parameters are subsequently trained on PathSeg, and the comparison with BiomedParse is performed after that training on held-out and external data; initialization is a transfer-learning choice, not a fitted input that forces the reported improvement. Self-citations to prior work by the same group (e.g., refs [4]-[7] and [52]) appear in related-work and architectural context and are not load-bearing for the empirical claims. The hierarchical label ambiguity conceded in Sec. 4.1 is a benchmark-validity concern, not a circularity: it does not make any reported Dice score equivalent to an input by definition. No derivation chain reduces to its own inputs, so no circular steps are identified.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central empirical claims rest on a newly assembled benchmark whose label taxonomy, preprocessing, and text templates are defined by the authors, and several choices (Dice threshold, instance filters, loss weights) shape the downstream analyses. No invented physical entities are introduced.

free parameters (3)
  • Dice threshold for explainability object selection = 0.4
    Section 4.4: of 47 breast objects, 21 are kept for biomarker discovery by requiring Dice >= 0.4; this post hoc filter shapes the reported interpretability results.
  • Instance size filter for density and dispersion metrics = 36 pixels
    Section 4.5: connected components smaller than 36 pixels are excluded before computing instance density and dispersion; different thresholds would change the intricate-object analysis.
  • BCE and Dice loss weights = 1, 1
    Section 4.4: lambda1 and lambda2 are both set to 1; standard but chosen by hand and not ablated.
assumptions (5)
  • ad hoc to paper The three-level label hierarchy [anatomical region]-[histological structure]-[object type] is a valid way to merge labels from 21 datasets into 160 consistent categories.
    Section 4.1: the paper itself notes that identical labels such as 'tumor' can denote tumor tissue or tumor cells; if the hierarchy does not fully disambiguate these, the benchmark and all text prompts inherit the ambiguity.
  • domain assumption Source datasets provide high-quality, precisely delineated masks and correct category labels.
    Section 2.1: inclusion criteria state 'high-quality segmentation masks with precisely delineated contours'; the central benchmark is only as good as these external annotations.
  • domain assumption Rescaling all images and masks to 40x magnification preserves the semantics of objects across datasets.
    Section 4.1: magnification normalization assumes the same object type maintains a relatively consistent size at 40x; if this fails, cross-dataset feature sharing is distorted.
  • ad hoc to paper The text template '[histological structure]-level [object type] in [anatomical region] pathology' is sufficient to specify the target category.
    Section 2.2: the model selects masks by matching class embeddings to this template; prompts outside the template are not evaluated.
  • domain assumption SEEM's Transformer encoder-decoder with joint feature interaction is an appropriate base architecture for pathology segmentation.
    Section 4.2: the method builds directly on SEEM; the paper does not ablate this architectural choice against other text-prompted segmentation backbones.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Segment Anything in Pathology Images with Natural Language." pith.science (2026). https://pith.science/paper/GYRKV46K

@misc{pith2026250620988,
  author       = {Pith},
  title        = {Pith review of: Segment Anything in Pathology Images with Natural Language},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GYRKV46K}},
  note         = {Machine review of arXiv:2506.20988}
}
read the original abstract

Pathology image segmentation is crucial in computational pathology for analyzing histological features relevant to cancer diagnosis and prognosis. However, current methods face major challenges in clinical applications due to limited annotated data and restricted category definitions. To address these limitations, we propose PathSegmentor, the first text-prompted segmentation foundation model designed specifically for pathology images. We also introduce PathSeg, the largest and most comprehensive dataset for pathology segmentation, built from 21 public sources and containing 275k image-mask-label triples across 160 diverse categories. With PathSegmentor, users can perform semantic segmentation using natural language prompts, eliminating the need for laborious spatial inputs such as points or boxes. Extensive experiments demonstrate that PathSegmentor outperforms specialized models with higher accuracy and broader applicability, while maintaining a compact architecture. It significantly surpasses existing spatial- and text-prompted models by 0.145 and 0.429 in overall Dice scores, respectively, showing strong robustness in segmenting complex structures and generalizing to external datasets. Moreover, PathSegmentor's outputs enhance the interpretability of diagnostic models through feature importance estimation and imaging biomarker discovery, offering pathologists evidence-based support for clinical decision-making. This work advances the development of explainable AI in precision oncology.

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Universal and Transferable Attacks on Pathology Foundation Models

    cs.CV 2025-10 accept novelty 6.0 of 10

    A single universal, transferable perturbation pattern, optimized against one pathology foundation model, degrades classification accuracy across seven unseen pathology foundation models and multiple datasets.

  2. A Versatile Pathology Co-pilot via Reasoning Enhanced Multimodal Large Language Model

    eess.IV 2025-07 conditional novelty 6.0 of 10

    A single pathology vision-language model trained with supervised and reinforcement fine-tuning beats four prior MLLMs across 72 ROI-level and slide-level tasks.

  3. Solutions for Mitotic Figure Detection and Atypical Classification in MIDOG 2025

    eess.IV 2025-08 conditional novelty 3.0 of 10

    A cascade of FCOS and a ResNet50/EfficientNet ensemble raises detection F1 to 0.8432 and a ConvNeXt ensemble reaches 0.86 balanced accuracy on MIDOG 2025.

Reference graph

Works this paper leans on

82 extracted references · 63 canonical work pages · cited by 3 Pith papers

  1. [1]

    IEEE Reviews in Biomedical Engineering 7, 97–114 (2013)

    Irshad, H., Veillard, A., Roux, L., Racoceanu, D.: Methods for nuclei detection, segmentation, and classification in digital histopathology: a review—current sta- tus and future potential. IEEE Reviews in Biomedical Engineering 7, 97–114 (2013)

    2013

  2. [2]

    : Recent advances of deep learning for computational histopathology: principles and applications

    Wu, Y., Cheng, M., Huang, S., Pei, Z., Zuo, Y., Liu, J., Yang, K., Zhu, Q., Zhang, J., Hong, H., et al. : Recent advances of deep learning for computational histopathology: principles and applications. Cancers 14(5), 1199 (2022)

    2022

  3. [3]

    IEEE Reviews in Biomedical Engineering 9, 234–263 (2016)

    Xing, F., Yang, L.: Robust nucleus/cell detection and segmentation in digital pathology and microscopy images: a comprehensive review. IEEE Reviews in Biomedical Engineering 9, 234–263 (2016)

    work page 2016

  4. [4]

    Medical Image Analysis 36, 135–146 (2017)

    Chen, H., Qi, X., Yu, L., Dou, Q., Qin, J., Heng, P.-A.: Dcan: Deep contour-aware networks for object instance segmentation from histology images. Medical Image Analysis 36, 135–146 (2017)

    work page 2017

  5. [5]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Jiang, H., Zhang, R., Zhou, Y., Wang, Y., Chen, H.: Donet: Deep de-overlapping network for cytology instance segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15641–15650 (2023)

    work page 2023

  6. [6]

    Medical Image Analysis 89, 102933 (2023)

    Lin, Y., Qu, Z., Chen, H., Gao, Z., Li, Y., Xia, L., Ma, K., Zheng, Y., Cheng, K.-T.: Nuclei segmentation with point annotations from pathology images via self- supervised learning and co-training. Medical Image Analysis 89, 102933 (2023)

    work page 2023

  7. [7]

    IEEE Transactions on Medical Imaging 43(6), 2137–2147 (2024)

    Lin, Y., Wang, Z., Zhang, D., Cheng, K.-T., Chen, H.: BoNuS: boundary mining for nuclei segmentation with partial point labels. IEEE Transactions on Medical Imaging 43(6), 2137–2147 (2024)

    work page 2024

  8. [8]

    Medical Image Analysis 58, 101563 (2019)

    Graham, S., Vu, Q.D., Raza, S.E.A., Azam, A., Tsang, Y.W., Kwak, J.T., Rajpoot, N.: Hover-net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images. Medical Image Analysis 58, 101563 (2019)

    work page 2019

Show all 82 references
  1. [9]

    Nature Methods 18(2), 203–211 (2021)

    Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.: nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods 18(2), 203–211 (2021)

    2021

  2. [10]

    In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp

    Zhang, J., Ma, K., Kapse, S., Saltz, J., Vakalopoulou, M., Prasanna, P., Samaras, D.: Sam-path: A segment anything model for semantic segmentation in digi- tal pathology. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 161–170 (2...

    2023

  3. [11]

    Medical Image Analysis 94, 103143 (2024)

    H¨ orst, F., Rempe, M., Heine, L., Seibold, C., Keyl, J., Baldini, G., Ugurel, S., Siveke, J., Gr¨ unwald, B., Egger, J.,et al.: Cellvit: Vision transformers for precise cell segmentation and classification. Medical Image Analysis 94, 103143 (2024)

    2024

  4. [12]

    arXiv preprint arXiv:2501.05269 (2025)

    H¨ orst, F., Rempe, M., Becker, H., Heine, L., Keyl, J., Kleesiek, J.: Cel- lvit++: Energy-efficient and adaptive cell segmentation and classification using foundation models. arXiv preprint arXiv:2501.05269 (2025)

    2025 arXiv

  5. [13]

    : Segment anything

    Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.-Y., et al. : Segment anything. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4015–4026 (2023)

    2023

  6. [14]

    Advances in Neural Information Processing Systems 36, 29914– 29934 (2023)

    Ke, L., Ye, M., Danelljan, M., Tai, Y.-W., Tang, C.-K., Yu, F.: Segment anything in high quality. Advances in Neural Information Processing Systems 36, 29914– 29934 (2023)

    2023

  7. [15]

    In: European Conference on Computer Vision, pp

    Li, F., Zhang, H., Sun, P., Zou, X., Liu, S., Li, C., Yang, J., Zhang, L., Gao, J.: Segment and recognize anything at any granularity. In: European Conference on Computer Vision, pp. 467–484 (2024). Springer

    2024

  8. [16]

    Advances in Neural Information Processing Systems 36 (2024)

    Zou, X., Yang, J., Zhang, H., Li, F., Li, L., Wang, J., Wang, L., Gao, J., Lee, Y.J.: Segment everything everywhere all at once. Advances in Neural Information Processing Systems 36 (2024)

    2024

  9. [17]

    Medical Image Analysis 89, 102918 (2023)

    Mazurowski, M.A., Dong, H., Gu, H., Yang, J., Konz, N., Zhang, Y.: Segment anything model for medical image analysis: an experimental study. Medical Image Analysis 89, 102918 (2023)

    2023

  10. [18]

    Computers in Biology and Medicine, 108238 (2024)

    Zhang, Y., Shen, Z., Jiao, R.: Segment anything model for medical image seg- mentation: Current applications and future directions. Computers in Biology and Medicine, 108238 (2024)

    2024

  11. [19]

    Nature Communications 15(1), 654 (2024)

    Ma, J., He, Y., Li, F., Han, L., You, C., Wang, B.: Segment anything in medical images. Nature Communications 15(1), 654 (2024)

    2024

  12. [20]

    arXiv preprint arXiv:2308.16184 (2023)

    Cheng, J., Ye, J., Deng, Z., Chen, J., Li, T., Wang, H., Su, Y., Huang, Z., Chen, J., Jiang, L., et al.: Sam-med2d. arXiv preprint arXiv:2308.16184 (2023)

    2023 arXiv

  13. [21]

    Nature Methods 22(1), 166–176 (2025)

    Zhao, T., Gu, Y., Yang, J., Usuyama, N., Lee, H.H., Kiblawi, S., Naumann, T., Gao, J., Crabtree, A., Abel, J., et al.: A foundation model for joint segmentation, detection and recognition of biomedical objects across nine modalities. Nature Methods 22(1), 166–176 (2025)

    2025

  14. [22]

    Nature Methods, 1–13 (2025) 37

    Archit, A., Freckmann, L., Nair, S., Khalid, N., Hilt, P., Rajashekar, V., Freitag, M., Teuber, C., Buckley, G., Haaren, S., et al.: Segment anything for microscopy. Nature Methods, 1–13 (2025) 37

    2025

  15. [23]

    Advances in Neural Information Processing Systems 37, 110746–110783 (2024)

    Du, Y., Bai, F., Huang, T., Zhao, B.: Segvol: Universal and interactive volumetric medical image segmentation. Advances in Neural Information Processing Systems 37, 110746–110783 (2024)

    2024

  16. [24]

    arXiv preprint arXiv:2312.17183 (2023)

    Zhao, Z., Zhang, Y., Wu, C., Zhang, X., Zhang, Y., Wang, Y., Xie, W.: One model to rule them all: Towards universal segmentation for medical images with text prompts. arXiv preprint arXiv:2312.17183 (2023)

    2023 arXiv

  17. [25]

    In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp

    Wang, A., Islam, M., Xu, M., Zhang, Y., Ren, H.: Sam meets robotic surgery: An empirical study on generalization, robustness and adaptation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–244 (2023). Springer

    2023

  18. [26]

    Nature Medicine 30(10), 2924–2935 (2024)

    Vorontsov, E., Bozkurt, A., Casson, A., Shaikovski, G., Zelechowski, M., Severson, K., Zimmermann, E., Hall, J., Tenenholtz, N., Fusi, N.,et al.: A foundation model for clinical-grade computational pathology and rare cancers detection. Nature Medicine 30(10), 2924–2935 (2024)

    2024

  19. [27]

    Nature Medicine 30(3), 850–862 (2024)

    Chen, R.J., Ding, T., Lu, M.Y., Williamson, D.F., Jaume, G., Song, A.H., Chen, B., Zhang, A., Shao, D., Shaban, M., et al.: Towards a general-purpose foundation model for computational pathology. Nature Medicine 30(3), 850–862 (2024)

    2024

  20. [28]

    Nature Medicine 30(3), 863–874 (2024)

    Lu, M.Y., Chen, B., Williamson, D.F., Chen, R.J., Liang, I., Ding, T., Jaume, G., Odintsov, I., Le, L.P., Gerber, G., et al.: A visual-language foundation model for computational pathology. Nature Medicine 30(3), 863–874 (2024)

    2024

  21. [29]

    arXiv preprint arXiv:2407.18449 (2024)

    Ma, J., Guo, Z., Zhou, F., Wang, Y., Xu, Y., Cai, Y., Zhu, Z., Jin, C., Lin, Y., Jiang, X., et al.: Towards a generalizable pathology foundation model via unified knowledge distillation. arXiv preprint arXiv:2407.18449 (2024)

    2024 arXiv

  22. [30]

    arXiv preprint arXiv:2407.15362 (2024)

    Xu, Y., Wang, Y., Zhou, F., Ma, J., Jin, C., Yang, S., Li, J., Zhang, Z., Zhao, C., Zhou, H., et al.: A multimodal knowledge-enhanced whole-slide pathology foundation model. arXiv preprint arXiv:2407.15362 (2024)

    2024 arXiv

  23. [31]

    IEEE Transactions on Medical Imaging (2024)

    Wang, C., Wan, Y., Li, S., Qu, K., Zhou, X., He, J., Ke, J., Yu, Y., Wang, T., Shen, Y.: Seganypath: a foundation model for multi-resolution stain-variant and multi-task pathology image segmentation. IEEE Transactions on Medical Imaging (2024)

    2024

  24. [32]

    In: Proceed- ings of the European Conference on Computer Vision, pp

    Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceed- ings of the European Conference on Computer Vision, pp. 801–818 (2018)

    2018

  25. [33]

    In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Zou, X., Dou, Z.-Y., Yang, J., Gan, Z., Li, L., Li, C., Dai, X., Behl, H., Wang, J., Yuan, L., et al.: Generalized decoding for pixel, image, and language. In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15116–15127 (2023) 38

    2023

  26. [34]

    In: International Conference on Machine Learning, pp

    Li, J., Li, D., Savarese, S., Hoi, S.: Blip-2: Bootstrapping language-image pre- training with frozen image encoders and large language models. In: International Conference on Machine Learning, pp. 19730–19742 (2023). PMLR

    2023

  27. [35]

    Journal of Paleontology 1(3), 179–183 (1927)

    Cox, E.: A method of assigning numerical and percentage values to the degree of roundness of sand grains. Journal of Paleontology 1(3), 179–183 (1927)

    1927

  28. [36]

    ACM Computing Surveys 56(4), 1–41 (2023)

    Patr ´ ıcio, C., Neves, J.C., Teixeira, L.F.: Explainable deep learning methods in medical image classification: A survey. ACM Computing Surveys 56(4), 1–41 (2023)

    2023

  29. [37]

    Medical Image Analysis 79, 102470 (2022)

    Velden, B.H., Kuijf, H.J., Gilhuijs, K.G., Viergever, M.A.: Explainable artificial intelligence (xai) in deep learning-based medical image analysis. Medical Image Analysis 79, 102470 (2022)

    2022

  30. [38]

    arXiv preprint arXiv:2410.02331 (2024)

    Hou, J., Liu, S., Bie, Y., Wang, H., Tan, A., Luo, L., Chen, H.: Self-explainable ai for medical image analysis: A survey and new outlooks. arXiv preprint arXiv:2410.02331 (2024)

    2024 arXiv

  31. [39]

    In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp

    Hou, J., Xu, J., Chen, H.: Concept-attention whitening for interpretable skin lesion diagnosis. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 113–123 (2024). Springer

    2024

  32. [40]

    Nature Communications 12(1), 1613 (2021)

    Diao, J.A., Wang, J.K., Chui, W.F., Mountain, V., Gullapally, S.C., Srinivasan, R., Mitchell, R.N., Glass, B., Hoffman, S., Rao, S.K., et al.: Human-interpretable image features derived from densely mapped cancer pathology slides predict diverse molecular phenotypes. Nature Co...

    2021

  33. [41]

    Cell Reports Medicine 5(9) (2024)

    Kludt, C., Wang, Y., Ahmad, W., Bychkov, A., Fukuoka, J., Gaisa, N., K¨ uhnel, M., Jonigk, D., Pryalukhin, A., Mairinger, F., et al.: Next-generation lung cancer pathology: Development and validation of diagnostic and prognostic algorithms. Cell Reports Medicine 5(9) (2024)

    2024

  34. [42]

    Nature 490, 61–70 (2012) https://doi.org/10.1038/nature11412

    Network, T.C.G.A.: Comprehensive molecular portraits of human breast tumors. Nature 490, 61–70 (2012) https://doi.org/10.1038/nature11412

    2012 doi

  35. [43]

    In: Pro- ceedings of the International Conference on Intelligent Information Technology, pp

    Zhou, Z.-H., Zhang, M.-L.: Neural networks for multi-instance learning. In: Pro- ceedings of the International Conference on Intelligent Information Technology, pp. 455–459 (2002). Citeseer

    2002

  36. [44]

    In: British Machine Vision Conference 2018, p

    Vitali, P., Abir, D., Kate, S.: RISE: randomized input sampling for explanation of black-box models. In: British Machine Vision Conference 2018, p. 151 (2018)

    2018

  37. [45]

    why should i trust you?

    Ribeiro, M.T., Singh, S., Guestrin, C.: “why should i trust you?” explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD Inter- national Conference on Knowledge Discovery and Data Mining, pp. 1135–1144 (2016) 39

    2016

  38. [46]

    Lippincott Williams & Wilkins, Philadel- phia (2001)

    Rosen, P.P.: Rosen’s Breast Pathology. Lippincott Williams & Wilkins, Philadel- phia (2001)

    2001

  39. [47]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep fea- tures for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2929 (2016)

    2016

  40. [48]

    : Foundation model for cancer imaging biomarkers

    Pai, S., Bontempi, D., Hadzic, I., Prudente, V., Sokaˇ c, M., Chaunzwa, T.L., Bernatz, S., Hosny, A., Mak, R.H., Birkbak, N.J., et al. : Foundation model for cancer imaging biomarkers. Nature Machine Intelligence 6(3), 354–367 (2024)

    2024

  41. [49]

    Nature Machine Intelligence 1(5), 206–215 (2019)

    Rudin, C.: Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence 1(5), 206–215 (2019)

    2019

  42. [50]

    Medical image analysis 71, 102062 (2021)

    Budd, S., Robinson, E.C., Kainz, B.: A survey on active learning and human- in-the-loop deep learning for medical image analysis. Medical image analysis 71, 102062 (2021)

    2021

  43. [51]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Li, M., Li, S., Zhang, X., Zhang, L.: Univs: Unified and universal video segmen- tation with prompts as queries. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3227–3238 (2024)

    2024

  44. [52]

    IEEE Transactions on Medical Imaging (2025)

    Hou, J., Xu, J., Feng, R., Chen, H.: Qmix: Quality-aware learning with mixed noise for robust retinal disease diagnosis. IEEE Transactions on Medical Imaging (2025)

    2025

  45. [53]

    Advances in Neural Information Processing Systems 35, 4203–4217 (2022)

    Yang, J., Li, C., Dai, X., Gao, J.: Focal modulation networks. Advances in Neural Information Processing Systems 35, 4203–4217 (2022)

    2022

  46. [54]

    ACM Transactions on Computing for Healthcare (HEALTH) 3(1), 1–23 (2021)

    Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J., Poon, H.: Domain-specific language model pretraining for biomedical natural language processing. ACM Transactions on Computing for Healthcare (HEALTH) 3(1), 1–23 (2021)

    2021

  47. [55]

    Computerized Medical Imaging and Graphics 112, 102337 (2024)

    Gadermayr, M., Tschuchnig, M.: Multiple instance learning for digital pathology: A review of the state-of-the-art, limitations and future potential. Computerized Medical Imaging and Graphics 112, 102337 (2024)

    2024

  48. [56]

    In: Interna- tional Conference on Learning Representations (2017)

    Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: Interna- tional Conference on Learning Representations (2017)

    2017

  49. [57]

    Advances in Neural Information Processing Systems, 8024–8035 40

    Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al.: Pytorch: An imperative style, high- performance deep learning library. Advances in Neural Information Processing Systems, 8024–8035 40

  50. [58]

    In: International Conference on Machine Learning, pp

    Ilse, M., Tomczak, J., Welling, M.: Attention-based deep multiple instance learn- ing. In: International Conference on Machine Learning, pp. 2127–2136 (2018). PMLR

    2018

  51. [59]

    Nature Biomedical Engineering 5(6), 555–570 (2021)

    Lu, M.Y., Williamson, D.F., Chen, T.Y., Chen, R.J., Barbieri, M., Mahmood, F.: Data-efficient and weakly supervised computational pathology on whole-slide images. Nature Biomedical Engineering 5(6), 555–570 (2021)

    2021

  52. [60]

    In: Interna- tional Conference on Learning Representations (2015)

    Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: Interna- tional Conference on Learning Representations (2015)

    2015

  53. [61]

    Pathology-Research and Practice 212(7), 583–597 (2016)

    Christgen, M., Steinemann, D., K¨ uhnle, E., L¨ anger, F., Gluz, O., Harbeck, N., Kreipe, H.: Lobular breast cancer: Clinical, molecular and morphological characteristics. Pathology-Research and Practice 212(7), 583–597 (2016)

    2016

  54. [62]

    In: 2016 Fourth International Conference on 3D Vision (3DV), pp

    Milletari, F., Navab, N., Ahmadi, S.-A.: V-net: Fully convolutional neural net- works for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV), pp. 565–571 (2016). IEEE

    2016

  55. [63]

    In: Proceedings of the IEEE International Conference on Computer Vision, pp

    He, K., Gkioxari, G., Doll´ ar, P., Girshick, R.: Mask r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969 (2017)

    2017

  56. [64]

    Bioinformatics 35(18), 3461–3467 (2019)

    Amgad, M., Elfandy, H., Hussein, H., Atteya, L.A., Elsebaie, M.A., Abo Elnasr, L.S., Sakr, R.A., Salem, H.S., Ismail, A.F., Saad, A.M., et al.: Structured crowd- sourcing enables convolutional segmentation of histology images. Bioinformatics 35(18), 3461–3467 (2019)

    2019

  57. [65]

    Biomedical Signal Processing and Control 66, 102402 (2021)

    Sitnik, D., Aralica, G., Hadˇ zija, M., Hadˇ zija, M.P., Paˇ ci´ c, A., Periˇ sa, M.M., Mano- jlovi´ c, L., Krstanac, K., Plaveti´ c, A., Kopriva, I.: A dataset and a methodology for intraoperative computer-aided diagnosis of a metastatic colon cancer in a liver. Biomedical Si...

    2021

  58. [66]

    : Conic challenge: Pushing the frontiers of nuclear detection, segmentation, classification and counting

    Graham, S., Vu, Q.D., Jahanifar, M., Weigert, M., Schmidt, U., Zhang, W., Zhang, J., Yang, S., Xiang, J., Wang, X., et al. : Conic challenge: Pushing the frontiers of nuclear detection, segmentation, classification and counting. Medical Image Analysis 92, 103047 (2024)

    2024

  59. [67]

    Medical Image Analysis 52, 199–211 (2019)

    Graham, S., Chen, H., Gamper, J., Dou, Q., Heng, P.-A., Snead, D., Tsang, Y.W., Rajpoot, N.: Mild-net: Minimal information loss dilated network for gland instance segmentation in colon histology images. Medical Image Analysis 52, 199–211 (2019)

    2019

  60. [68]

    : Digestpath: A benchmark dataset with challenge review for the pathological detection and segmentation of digestive-system

    Da, Q., Huang, X., Li, Z., Zuo, Y., Zhang, C., Liu, J., Chen, W., Li, J., Xu, D., Hu, Z., et al. : Digestpath: A benchmark dataset with challenge review for the pathological detection and segmentation of digestive-system. Medical Image Analysis 80, 102485 (2022) 41

    2022

  61. [69]

    Medical Image Analysis 35, 489–502 (2017)

    Sirinukunwattana, K., Pluim, J.P., Chen, H., Qi, X., Heng, P.-A., Guo, Y.B., Wang, L.Y., Matuszewski, B.J., Bruni, E., Sanchez, U.,et al.: Gland segmentation in colon histology images: The glas challenge contest. Medical Image Analysis 35, 489–502 (2017)

    2017

  62. [70]

    Journal of Pathology Informatics 7(1), 29 (2016)

    Janowczyk, A., Madabhushi, A.: Deep learning for digital pathology image anal- ysis: A comprehensive tutorial with selected use cases. Journal of Pathology Informatics 7(1), 29 (2016)

    2016

  63. [71]

    : Domain gener- alization across tumor types, laboratories, and species—insights from the 2022 edition of the mitosis domain generalization challenge

    Aubreville, M., Stathonikos, N., Donovan, T.A., Klopfleisch, R., Ammeling, J., Ganz, J., Wilm, F., Veta, M., Jabari, S., Eckstein, M., et al. : Domain gener- alization across tumor types, laboratories, and species—insights from the 2022 edition of the mitosis domain generaliza...

    2024

  64. [72]

    IEEE Transactions on Medical Imaging 40(12), 3413–3423 (2021)

    Verma, R., Kumar, N., Patil, A., Kurian, N.C., Rane, S., Graham, S., Vu, Q.D., Zwager, M., Raza, S.E.A., Rajpoot, N., et al.: Monusac2020: A multi-organ nuclei segmentation and classification challenge. IEEE Transactions on Medical Imaging 40(12), 3413–3423 (2021)

    2021

  65. [73]

    : Nucls: A scalable crowdsourcing approach and dataset for nucleus classification and segmentation in breast cancer

    Amgad, M., Atteya, L.A., Hussein, H., Mohammed, K.H., Hafiz, E., Elsebaie, M.A., Alhusseiny, A.M., AlMoslemany, M.A., Elmatboly, A.M., Pappalardo, P.A., et al. : Nucls: A scalable crowdsourcing approach and dataset for nucleus classification and segmentation in breast cancer. ...

    2022

  66. [74]

    In: Digital Pathology: 15th European Congress, ECDP 2019, Warwick, UK, April 10–13, 2019, Proceedings 15, pp

    Gamper, J., Alemi Koohbanani, N., Benet, K., Khuram, A., Rajpoot, N.: Pan- nuke: an open pan-cancer histology dataset for nuclei instance segmentation and classification. In: Digital Pathology: 15th European Congress, ECDP 2019, Warwick, UK, April 10–13, 2019, Proceedings 15, ...

    2019

  67. [75]

    Artificial Intelligence in Medicine 115, 102076 (2021)

    Salvi, M., Bosco, M., Molinaro, L., Gambella, A., Papotti, M., Acharya, U.R., Molinari, F.: A hybrid deep learning approach for gland segmentation in prostate histopathological images. Artificial Intelligence in Medicine 115, 102076 (2021)

    2021

  68. [76]

    Patterns 4(2) (2023)

    Komura, D., Onoyama, T., Shinbo, K., Odaka, H., Hayakawa, M., Ochi, M., Herdiantoputri, R.R., Endo, H., Katoh, H., Ikeda, T., et al.: Restaining-based annotation for cancer histology segmentation to overcome annotation-related limitations among pathologists. Patterns 4(2) (2023)

    2023

  69. [77]

    Mendeley Data (2020)

    Silva-Rodr ´ ıguez, J.: Sicapv2-prostate whole slide images with gleason grades annotations. Mendeley Data (2020)

    2020

  70. [78]

    IEEE Transactions on Medical Imaging 38(2), 448–459 (2018)

    Naylor, P., La´ e, M., Reyal, F., Walter, T.: Segmentation of nuclei in histopathol- ogy images by deep regression of the distance map. IEEE Transactions on Medical Imaging 38(2), 448–459 (2018)

    2018

  71. [79]

    arXiv preprint arXiv:2204.06455 (2022)

    Han, C., Pan, X., Yan, L., Lin, H., Li, B., Yao, S., Lv, S., Shi, Z., Mai, J., 42 Lin, J., et al.: Wsss4luad: Grand challenge on weakly-supervised tissue semantic segmentation for lung adenocarcinoma. arXiv preprint arXiv:2204.06455 (2022)

    2022 arXiv

  72. [80]

    : Methods for seg- mentation and classification of digital microscopy tissue images

    Vu, Q.D., Graham, S., Kurc, T., To, M.N.N., Shaban, M., Qaiser, T., Koohbanani, N.A., Khurram, S.A., Kalpathy-Cramer, J., Zhao, T., et al. : Methods for seg- mentation and classification of digital microscopy tissue images. Frontiers in bioengineering and biotechnology 7, 53 (2019)

    2019

  73. [81]

    IEEE Transactions on Medical Imaging 39(12), 4124–4136 (2020)

    Graham, S., Epstein, D., Rajpoot, N.: Dense steerable filter cnns for exploiting rotational symmetry in histology images. IEEE Transactions on Medical Imaging 39(12), 4124–4136 (2020)

    2020

  74. [82]

    [histological structure]-level [object type] in [anatomicalregion] pathology

    Graham, S., Jahanifar, M., Azam, A., Nimir, M., Tsang, Y.-W., Dodd, K., Hero, E., Sahota, H., Tank, A., Benes, K., et al. : Lizard: a large-scale dataset for colonic nuclear instance segmentation and classification. In: Proceedings of the IEEE/CVF International Conference on C...

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.