Pith. sign in

REVIEW 3 major objections 5 minor 64 references

Unleashing Diffusion and State Space Models for Medical Image Segmentation

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A two-stage query framework using k-Means Mask Mamba and diffusion-guided refinement claims state-of-the-art zero-shot segmentation of unseen tumor categories in CT volumes, beating open-vocabulary and OOD baselines on 8 of 9 metrics.

desk verdict Convincing integration of existing parts, but the MSD zero-shot evaluation is contaminated by model selection on the same data, so the 8-of-9 headline does not stand. read the letter →

arxiv 2506.12747 v2 pith:T7QTXPJH submitted 2025-06-15 cs.CV cs.AI

Rong Wu , Ziqi Chen , Liming Zhong , Heng Li , Hai Shu This is my paper · ORCID
classification cs.CVcs.AI
keywords diffusion-guidedsegmentationstatespacemodelsMambaopen-vocabularyzero-shottumorout-of-distributiondetectionmedicalimageCTimaging
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to show that one model can segment organs and also detect and segment tumor types it never saw during training, on CT volumes. It does this by first learning organ queries, then deriving anomaly prompts from the regions where those queries fail, and refining tumor queries with a diffusion-guided decoder plus CLIP text alignment. On three unseen tumor categories from the MSD dataset, the proposed DSM model claims the best score in 8 of 9 evaluation terms, including AUROC and Dice, against open-vocabulary and out-of-distribution baselines. If correct, this would allow clinicians to query a model for a new tumor class by name without retraining for each rare lesion type.

What carries the argument

The mechanism that carries the argument is the two-stage query system. Stage 1 trains organ queries in a k-Means Mask Mamba (kMMM) decoder, which combines query-wise argmax grouping, as in k-means clustering, with a state space model (SSM) layer to preserve long-range memory across 3D volumes. Stage 2 builds tumor queries: the negative maximal organ-query response is min-max normalized with a 0.5 threshold to form anomaly mask prompts, which are added to the attention logits so tumor queries focus on out-of-distribution regions. A Diffusion-guided Query Refinement (DQR) decoder then applies a nonlinear diffusion process (Perona–Malik style) to enhance boundary features, and CLIP text embeddings are used to compute a cosine similarity map for class alignment.

What would settle it

One could test on a set of low-contrast tumors known to resemble their host organ in CT attenuation (e.g., some pancreatic adenocarcinomas or small liver metastases): if the anomaly mask from the negative maximal organ-query response fails to cover these lesions, the zero-shot segmentation would degrade even though organ segmentation remains accurate.

Watch

Extended reading notes

Core claim

The central claim is that the negative maximal organ-query response — the regions where no learned organ matches the tissue — is a reliable visual prompt for locating novel tumors, and that tumor queries refined by this prompt can segment those tumors zero-shot. The paper further claims that adding a k-Means Mask Mamba layer for long-range query memory, a diffusion-guided query refinement decoder for sharper boundaries, and cosine alignment with CLIP text embeddings is what lets the model outperform existing open-vocabulary, prompt-based, and OOD-detection baselines on pancreas, lung, and colon tumors from the Medical Segmentation Decathlon.

Load-bearing premise

The approach depends on unseen tumors always producing lower organ-query affinity scores than healthy organ tissue, so that the negative maximal response marks tumor regions.

Editorial extensions

If this is right

  • A model can be asked, via a text class name, to segment a tumor type it has never seen, without any fine-tuning.
  • The organ queries double as a general out-of-distribution detector, so the approach could flag any unexpected anatomy, not just the three tested tumor families.
  • Diffusion-guided boundary enhancement transfers to seen organs and tumors, improving Dice on all tested structures, which suggests the boundary-sharpening idea is task-general.
  • The kMMM layer's SSM memory stabilizes training on long 3D sequences, which should carry over to other volumetric medical tasks with long-range dependencies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the anomaly prompt is a simple threshold on the negative maximal response, replacing it with a calibrated likelihood ratio could yield confidence scores for uncertain regions, a natural clinical extension the paper does not explore.
  • If the training organ taxonomy is incomplete, healthy structures that resemble tumors could be flagged as anomalies; testing on datasets with additional normal anatomy (e.g., vessels, bowel) would reveal this failure mode.
  • The diffusion-guided fusion is a learned anisotropic smoothing; an ablation that fixes the diffusivity to a hand-crafted edge-stopping function would show how much of the gain comes from learning rather than from the diffusion prior itself.
  • The paper evaluates only CT; the same query-anomaly mechanism should be testable on MRI or ultrasound, where boundary ambiguity is even more pronounced, and the zero-shot benefit might be larger.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes DSM, a two-stage framework for CT-based organ and tumor segmentation. Stage 1 trains organ queries through a k-Means Mask Mamba decoder that couples the kMaX-DeepLab query-grouping idea with an SSM layer. Stage 2 adds tumor queries, anomaly mask visual prompts derived from negative maximal organ-query responses, diffusion-guided boundary enhancement, and CLIP text-embedding alignment. The authors report state-of-the-art zero-shot segmentation of unseen MSD tumor categories (best in 8 of 9 metrics in Table 2), strong results on seen organs/tumors (Table 3), competitive fully supervised fine-tuning on MSD (Table 4), and a four-component ablation (Table 5).

Significance. If validated, the central claim is clinically valuable: a single model that segments known organs and detects and segments previously unseen tumor categories in CT volumes without per-task retraining. The framework is ambitious and integrates several currently popular mechanisms, and the authors release code, evaluate on a broad set of standard datasets against multiple baseline families (open-vocabulary, OOD detection, prompt-based medical segmentation), and report standard metrics with standard errors. The main barrier is not the method's plausibility but the evaluation protocol: the dataset presented as the unseen testbed is also used for component selection and fine-tuning, which would invalidate the zero-shot interpretation as stated.

major comments (3)
  1. [Sections 5.1, 6.3, 6.4; Tables 4 and 5] The central zero-shot claim is compromised by the use of MSD for model selection and fine-tuning. Section 5.1 defines MSD as the inference set for unseen tumor types, but Section 6.4 explicitly evaluates the ablation on 'Colon Tumor in MSD (Unseen)' and Section 6.3 fine-tunes DSM on 396 MSD images with a 396/74 split. Since the final architecture reported in Table 2 is the configuration selected in Table 5 using MSD labels, the Table 2 results cannot be read as independent evidence of zero-shot generalization; the 'best in 8 of 9' statistic may reflect model selection on the test distribution rather than a transferable capability. The authors should either freeze all architecture and hyperparameter choices before inspecting any MSD metric and state this explicitly, or re-run the zero-shot evaluation on a genuinely held-out set of tumor types that was never used for ablations, fine-tuning, or checkpoint selection.
  2. [Section 4.2.1, Eqs. (9) and (14)] The anomaly mask visual prompt is central to the method, but the notation for the query response is internally inconsistent. Equation (9) defines R_i as an argmax over organ queries, which is a hard assignment (e.g., one-hot) rather than an affinity score; Equation (14) then uses the 'negative maximal query response' as an anomaly score, which only makes sense if R_i contains continuous similarities or logits. As written, A_i in Eq. (14) would be binary and would not encode the degree to which a location is out-of-distribution. The authors need to clarify what R_i actually is, define the computation unambiguously, and provide evidence (e.g., score distributions or failure cases) for the underlying assumption that unseen tumors consistently produce lower organ-query affinity than healthy tissue.
  3. [Section 5.4] The baseline protocol for the key zero-shot comparison is underspecified. The text says that 'Swin UNETR and ZePT ... are fine-tuned on the inference dataset since they were not originally pretrained on this data', which, if taken literally, would mean that the main zero-shot baselines in Table 2 were fine-tuned on the MSD test distribution. It is also not explained how 2D open-vocabulary methods such as ZegFormer, OpenSeg, OVSeg, and Freeseg are applied to 3D CT volumes (e.g., slice-by-slice inference, volume aggregation, or some adaptation). The authors should state precisely which configuration of each baseline is used in each table and describe the adaptation of 2D methods to 3D data.
minor comments (5)
  1. [Section 5.1, Table 1] The training-set description contains a numeric inconsistency: the text says Stage 1 uses 2,668 images from 8 public datasets, while the sum of the training rows in Table 1 is 2,688. The sentence about '40% of BTCV, LiTS, and KiTS images (192 images)' is also confusing and should be rewritten to make clear that 20/52/120 images from those three datasets are used for training.
  2. [Table 5] The ablation table is difficult to read in the current formatting: the checkmark columns are not clearly aligned with the module names, so the reader cannot tell which component is active in each row. Please reformat the table so each row unambiguously shows the subset of kMMM, AMVP, DQR, and CLIP being evaluated.
  3. [Section 4.2.3] The sentence 'we calculate the cosine similarity between the text embeddings rather than relying solely on the vision embedding' is confusing because Eq. (20) computes cosine similarity between each text embedding and the projected query embedding. Please rephrase to match the actual computation.
  4. [Section 4.2.1, Eq. (8)] Equation (8) uses an unnormalized argmax over queries with no mention of how gradients flow through the hard assignment. If this is meant to be the kMaX-DeepLab-style masked attention, please state the exact operation (e.g., straight-through estimation or a separate soft assignment for training).
  5. [Section 4.2.3, class prompt list] There is a typo in the class prompt list: 'Celiac Truck' should be 'Celiac Trunk'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DSM's tumor segmentation is a supervised two-stage pipeline whose predictions are not definitionally forced by its inputs.

full rationale

DSM's central claim is that a two-stage supervised pipeline can segment tumor categories not present in the pixel-label training set. Stage 1 trains organ queries with ground-truth organ masks (Eqs. 9-13); Stage 2 trains tumor queries with ground-truth tumor masks from LiTS/KiTS and uses CLIP text alignment (Eqs. 19-20). The anomaly mask prompt (Eqs. 14-15) is generated from the model's own organ-query response, but it serves as an input feature to the tumor decoder, not as a training target that defines the tumor loss; tumor supervision is external. The diffusion-guided refinement (Eqs. 16-18) is a learnable local filter with a PDE-inspired form, not a fitted prediction. No load-bearing self-citation was found; architectural components are credited to external prior work such as kMaX-DeepLab, Mamba, and MaskFormer. The only notable protocol concern is that Section 6.4's ablation uses the MSD colon-tumor 'unseen' set for component selection, and Section 6.3 fine-tunes on MSD for a separate fully-supervised experiment; this is a benchmark-contamination or model-selection concern that may inflate the zero-shot margin, but it is not a circular reduction of the prediction to its inputs. The zero-shot results are not definitionally forced by the training losses.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

No new physical entities or fundamentally new structural assumptions are introduced. The method depends on standard SSM discretization, a heuristic diffusion-based smoothing operation, the transferability of CLIP text features to medical CT, and the partial-label training setup. The main free parameters are the number of queries, the OOD threshold, the temperature, and the fixed diffusion time step, none of which are ablated.

free parameters (5)
  • Temperature tau = 0.01
    Appears in Eq. 20 for the softmax over cosine similarities; chosen without sensitivity analysis.
  • Mask threshold = 0.5
    Threshold in Eq. 15 that converts anomaly scores to binary mask prompts; its value is not explored.
  • Number of organ queries No = 25
    Set to the number of organ classes in the training set; not tuned.
  • Number of tumor queries NT = 20
    Set to the number of tumor/disease categories; may limit generalization to the three MSD unseen classes.
  • Diffusion time steps t = 1
    The PDE in Eq. 16 is integrated for a single step; no ablation over t is provided.
assumptions (4)
  • standard math SSM discretization (zero-order hold) as in Mamba
    Used in Section 3.1 to define the state-space update; standard result accepted from Gu and Dao 2024.
  • domain assumption Perona-Malik diffusion as a boundary-enhancement heuristic
    Assumed in Section 3.2 that smoothing along boundaries improves segmentation; the discretization in Eq. 17 is a learned approximation.
  • domain assumption CLIP text embeddings transfer to 3D CT
    Assumed in Section 4.2.3 that pretrained CLIP text features for class names align with learned query embeddings for CT organs and tumors, despite CLIP being trained on natural images.
  • domain assumption Partial-label training strategy
    Assumed in Section 4.1.2 that training with masks only for specified organs on each image yields robust organ queries.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unleashing Diffusion and State Space Models for Medical Image Segmentation." pith.science (2026). https://pith.science/paper/T7QTXPJH

@misc{pith2026250612747,
  author       = {Pith},
  title        = {Pith review of: Unleashing Diffusion and State Space Models for Medical Image Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T7QTXPJH}},
  note         = {Machine review of arXiv:2506.12747}
}
read the original abstract

Existing segmentation models trained on a single medical imaging dataset often lack robustness when encountering unseen organs or tumors. Developing a robust model capable of identifying rare or novel tumor categories not present during training is crucial for advancing medical imaging applications. We propose DSM, a novel framework that leverages diffusion and state space models to segment unseen tumor categories beyond the training data. DSM utilizes two sets of object queries trained within modified attention decoders to enhance classification accuracy. Initially, the model learns organ queries using an object-aware feature grouping strategy to capture organ-level visual features. It then refines tumor queries by focusing on diffusion-based visual prompts, enabling precise segmentation of previously unseen tumors. Furthermore, we incorporate diffusion-guided feature fusion to improve semantic segmentation performance. By integrating CLIP text embeddings, DSM captures category-sensitive classes to improve linguistic transfer knowledge, thereby enhancing the model's robustness across diverse scenarios and multi-label tasks. Extensive experiments demonstrate the superior performance of DSM in various tumor segmentation tasks. Code is available at https://github.com/Rows21/k-Means_Mask_Mamba.

Figures

Figures reproduced from arXiv: 2506.12747 by the authors.

Figure 1
Figure 1. The k-Means Mask Mamba (kMMM) structure: To convert a Trans [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Pipeline overview: (a) Stage 1: Leveraging the kMaX-DeepLab (Yu et al., 2022), we introduce a feature grouping approach with SSM layers to train [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Diffusion-guided feature fusion mechanism. 4.2.2. Diffusion-guided Query Refinement Inspired by the diffusion process given in Equation (7), for multi-scale feature maps Fi , the semantic diffusion process can be described as: ∂Fi ∂t = Div(g(|Di | 2 )∇Fi), (16) where Di represents the guidance feature map with the same resolution as Fi , and t refers to the time steps to enhance the guidance matrix, which in our con… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Qualitative visualization of the proposed DSM compared with competing models in unseen tumor segmentation on MSD dataset. The segmentation results [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Qualitative visualization of the proposed DSM compared with competing models in fully supervised segmentation on MSD dataset. The segmentation [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

64 extracted references · 63 canonical work pages

  1. [1]

    , author Reinke, A

    author Antonelli, M. , author Reinke, A. , author Bakas, S. , author Farahani, K. , author Kopp-Schneider, A. , author Landman, B.A. , et al., year 2022 . title The medical segmentation decathlon . journal Nature communications volume 13 , pages 4128

    work page 2022

  2. [2]

    , author Christ, P

    author Bilic, P. , author Christ, P. , author Li, H.B. , author Vorontsov, E. , author Ben-Cohen, A. , author Kaissis, G. , et al., year 2023 . title The liver tumor segmentation benchmark (lits) . journal Med. Image Anal. volume 84 , pages 102680

    work page 2023

  3. [3]

    , author Mann, B

    author Brown, T. , author Mann, B. , author Ryder, N. , author Subbiah, M. , author Kaplan, J.D. , author Dhariwal, P. , et al., year 2020 . title Language models are few-shot learners . journal NeurIPS volume 33 , pages 1877--1901

    work page 2020

  4. [4]

    , author Ortiz, J.J.G

    author Butoi, V.I. , author Ortiz, J.J.G. , author Ma, T. , author Sabuncu, M.R. , author Guttag, J. , author Dalca, A.V. , year 2023 . title Universeg: Universal medical image segmentation , in: booktitle ICCV , pp. pages 21438--21451

    work page 2023

  5. [5]

    , author Miao, J

    author Chen, C. , author Miao, J. , author Wu, D. , author Zhong, A. , author Yan, Z. , author Kim, S. , et al., year 2024 . title MA-SAM : Modality-agnostic SAM adaptation for 3D medical image segmentation . journal Med. Image Anal. volume 98 , pages 103310

    work page 2024

  6. [6]

    , author Misra, I

    author Cheng, B. , author Misra, I. , author Schwing, A.G. , author Kirillov, A. , author Girdhar, R. , year 2022 . title Masked-attention mask transformer for universal image segmentation , in: booktitle CVPR , pp. pages 1290--1299

    work page 2022

  7. [7]

    , author Schwing, A

    author Cheng, B. , author Schwing, A. , author Kirillov, A. , year 2021 . title Per-pixel classification is not all you need for semantic segmentation . journal NeurIPS volume 34 , pages 17864--17875

    work page 2021

  8. [8]

    , author Gu, A

    author Dao, T. , author Gu, A. , year 2024 . title Transformers are SSMs : Generalized models and efficient algorithms through structured state space duality , in: booktitle ICML , pp. pages 10041--10071

    work page 2024

Show all 64 references
  1. [9]

    , author Chang, M.W

    author Devlin, J. , author Chang, M.W. , author Lee, K. , author Toutanova, K. , year 2019 . title Bert: Pre-training of deep bidirectional transformers for language understanding , in: booktitle NAACL-HLT , p. pages 4171–4186

    2019

  2. [10]

    , author Xue, N

    author Ding, J. , author Xue, N. , author Xia, G.S. , author Dai, D. , year 2022 . title Decoupling zero-shot semantic segmentation , in: booktitle CVPR , pp. pages 11583--11592

    2022

  3. [11]

    , author Uchibe, E

    author Elfwing, S. , author Uchibe, E. , author Doya, K. , year 2018 . title Sigmoid-weighted linear units for neural network function approximation in reinforcement learning . journal Neural Netw. volume 107 , pages 3--11

    2018

  4. [12]

    , author Yan, P

    author Fang, X. , author Yan, P. , year 2020 . title Multi-organ segmentation over partially labeled datasets with multi-scale feature abstraction . journal IEEE Trans. Med. Imaging volume 39 , pages 3619--3629

    2020

  5. [13]

    , author Gu, X

    author Ghiasi, G. , author Gu, X. , author Cui, Y. , author Lin, T.Y. , year 2022 . title Scaling open-vocabulary image segmentation with image-level labels , in: booktitle ECCV , pp. pages 540--557

    2022

  6. [14]

    , author Dao, T

    author Gu, A. , author Dao, T. , year 2024 . title Mamba: Linear-time sequence modeling with selective state spaces , in: booktitle COLM

    2024

  7. [15]

    , author Goel, K

    author Gu, A. , author Goel, K. , author R \'e , C. , year 2022 . title Efficiently modeling long sequences with structured state spaces , in: booktitle ICLR

    2022

  8. [16]

    , author Zhou, S

    author Gu, Z. , author Zhou, S. , author Niu, L. , author Zhao, Z. , author Zhang, L. , year 2020 . title Context-aware feature generation for zero-shot semantic segmentation , in: booktitle ACM MM , pp. pages 1921--1929

    2020

  9. [17]

    , year 1994

    author Hamilton, J.D. , year 1994 . title State-space models . journal Handbook of Econometrics volume 4 , pages 3039--3080

    1994

  10. [18]

    , author Nath, V

    author Hatamizadeh, A. , author Nath, V. , author Tang, Y. , author Yang, D. , author Roth, H.R. , author Xu, D. , year 2021 . title Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images , in: booktitle International MICCAI brainlesion workshop ...

    2021

  11. [19]

    , author Zhang, X

    author He, K. , author Zhang, X. , author Ren, S. , author Sun, J. , year 2016 . title Deep residual learning for image recognition , in: booktitle CVPR , pp. pages 770--778

    2016

  12. [20]

    , author McSweeney, S

    author Heller, N. , author McSweeney, S. , author Peterson, M.T. , author Peterson, S. , et al., year 2020 . title An international challenge to use artificial intelligence to define the state-of-the-art in kidney and kidney tumor segmentation in CT imaging . journal J. Clin. ...

    2020

  13. [21]

    , author Kuen, J

    author Huynh, D. , author Kuen, J. , author Lin, Z. , author Gu, J. , author Elhamifar, E. , year 2022 . title Open-vocabulary instance segmentation via robust cross-modal pseudo-labeling , in: booktitle CVPR , pp. pages 7020--7031

    2022

  14. [22]

    , author Jaeger, P.F

    author Isensee, F. , author Jaeger, P.F. , author Kohl, S.A. , author Petersen, J. , author Maier-Hein, K.H. , year 2021 . title nnu-net: a self-configuring method for deep learning-based biomedical image segmentation . journal Nature methods volume 18 , pages 203--211

    2021

  15. [23]

    , author Bai, H

    author Ji, Y. , author Bai, H. , author Ge, C. , author Yang, J. , author Zhu, Y. , author Zhang, R. , et al., year 2022 . title Amos: A large-scale abdominal multi-organ benchmark for versatile medical image segmentation . journal NeurIPS volume 35 , pages 36722--36732

    2022

  16. [24]

    , author Huang, Z

    author Jiang, Y. , author Huang, Z. , author Zhang, R. , author Zhang, X. , author Zhang, S. , year 2024 . title Zept: Zero-shot pan-tumor segmentation via query-disentangling and self-prompting , in: booktitle CVPR , pp. pages 11386--11397

    2024

  17. [25]

    , author Grave, E

    author Joulin, A. , author Grave, E. , author Bojanowski, P. , author Mikolov, T. , year 2017 . title Bag of tricks for efficient text classification , in: booktitle EACL , p. pages 427–431

    2017

  18. [26]

    , author Lee, J

    author Jung, S. , author Lee, J. , author Gwak, D. , author Choi, S. , author Choo, J. , year 2021 . title Standardized max logits: A simple yet effective approach for identifying unexpected road obstacles in urban-scene segmentation , in: booktitle ICCV , pp. pages 15425--15434

    2021

  19. [27]

    , author Gezer, N.S

    author Kavur, A.E. , author Gezer, N.S. , author Bar s , M. , author Aslan, S. , author Conze, P.H. , author Groza, V. , et al., year 2021 . title Chaos challenge-combined (ct-mr) healthy abdominal organ segmentation . journal Med. Image Anal. volume 69 , pages 101950

    2021

  20. [28]

    , author Mintun, E

    author Kirillov, A. , author Mintun, E. , author Ravi, N. , author Mao, H. , author Rolland, C. , author Gustafson, L. , et al., year 2023 . title Segment anything , in: booktitle ICCV , pp. pages 4015--4026

    2023

  21. [29]

    , author Xu, Z

    author Landman, B. , author Xu, Z. , author Igelsias, J. , author Styner, M. , author Langerak, T. , author Klein, A. , year 2015 . title Miccai multi-atlas labeling beyond the cranial vault--workshop and challenge , in: booktitle MICCAI Challenge , p. pages 12

    2015

  22. [30]

    , author Weinberger, K.Q

    author Li, B. , author Weinberger, K.Q. , author Belongie, S. , author Koltun, V. , author Ranftl, R. , year 2022 . title Language-driven semantic segmentation , in: booktitle ICLR

    2022

  23. [31]

    , author Wu, B

    author Liang, F. , author Wu, B. , author Dai, X. , author Li, K. , author Zhao, Y. , author Zhang, H. , et al., year 2023 . title Open-vocabulary semantic segmentation with mask-adapted clip , in: booktitle CVPR , pp. pages 7061--7070

    2023

  24. [32]

    , author Zhang, Y

    author Liu, J. , author Zhang, Y. , author Chen, J.N. , author Xiao, J. , author Lu, Y. , author A Landman, B. , et al., year 2023 . title Clip-driven universal model for organ segmentation and tumor detection , in: booktitle ICCV , pp. pages 21152--21164

    2023

  25. [33]

    , author Lin, Y

    author Liu, Z. , author Lin, Y. , author Cao, Y. , author Hu, H. , author Wei, Y. , author Zhang, Z. , et al., year 2021 . title Swin transformer: Hierarchical vision transformer using shifted windows , in: booktitle ICCV , pp. pages 10012--10022

    2021

  26. [34]

    , author Liao, W

    author Luo, X. , author Liao, W. , author Xiao, J. , author Chen, J. , author Song, T. , author Zhang, X. , et al., year 2022 . title WORD : A large scale dataset, benchmark and clinical applicable study for abdominal organ segmentation from ct image . journal Med. Image Anal....

    2022

  27. [35]

    , author Li, F

    author Ma, J. , author Li, F. , author Wang, B. , year 2024 . title U-mamba: Enhancing long-range dependency for biomedical image segmentation . journal arXiv:2401.04722

    2024 arXiv

  28. [36]

    , author Zhang, Y

    author Ma, J. , author Zhang, Y. , author Gu, S. , author Zhu, C. , author Ge, C. , author Zhang, Y. , et al., year 2021 . title Abdomenct-1k: Is abdominal organ segmentation a solved problem? journal TPAMI volume 44 , pages 6695--6714

    2021

  29. [37]

    , author Banerjee, O

    author Moor, M. , author Banerjee, O. , author Abad, Z.S.H. , author Krumholz, H.M. , author Leskovec, J. , author Topol, E.J. , et al., year 2023 . title Foundation models for generalist medical artificial intelligence . journal Nature volume 616 , pages 259--265

    2023

  30. [38]

    , author Woo, S

    author Park, K. , author Woo, S. , author Oh, S.W. , author Kweon, I.S. , author Lee, J.Y. , year 2022 . title Per-clip video object segmentation , in: booktitle CVPR , pp. pages 1352--1361

    2022

  31. [39]

    , author Xie, S

    author Peebles, W. , author Xie, S. , year 2023 . title Scalable diffusion models with transformers , in: booktitle ICCV , pp. pages 4195--4205

    2023

  32. [40]

    , author Malik, J

    author Perona, P. , author Malik, J. , year 1990 . title Scale-space and edge detection using anisotropic diffusion . journal TPAMI volume 12 , pages 629--639

    1990

  33. [41]

    , author Wu, J

    author Qin, J. , author Wu, J. , author Yan, P. , author Li, M. , author Yuxi, R. , author Xiao, X. , et al., year 2023 . title Freeseg: Unified, universal and open-vocabulary image segmentation , in: booktitle CVPR , pp. pages 19446--19455

    2023

  34. [42]

    , author Kim, J.W

    author Radford, A. , author Kim, J.W. , author Hallacy, C. , author Ramesh, A. , author Goh, G. , author Agarwal, S. , et al., year 2021 . title Learning transferable visual models from natural language supervision , in: booktitle ICML , pp. pages 8748--8763

    2021

  35. [43]

    , author Zhao, W

    author Rao, Y. , author Zhao, W. , author Chen, G. , author Tang, Y. , author Zhu, Z. , author Huang, G. , et al., year 2022 . title Denseclip: Language-guided dense prediction with context-aware prompting , in: booktitle CVPR , pp. pages 18082--18091

    2022

  36. [44]

    , author Yi, D

    author Rister, B. , author Yi, D. , author Shivakumar, K. , author Nobashi, T. , author Rubin, D.L. , year 2020 . title Ct-org, a new dataset for multiple organ segmentation in computed tomography . journal Scientific Data volume 7 , pages 381

    2020

  37. [45]

    , author Lu, L

    author Roth, H.R. , author Lu, L. , author Farag, A. , author Shin, H.C. , author Liu, J. , author Turkbey, E.B. , et al., year 2015 . title Deeporgan: Multi-level deep convolutional networks for automated pancreas segmentation , in: booktitle MICCAI , pp. pages 556--564

    2015

  38. [46]

    , year 2006

    author Sapiro, G. , year 2006 . title Geometric partial differential equations and image analysis . publisher Cambridge University Press

    2006

  39. [47]

    , author Azizi, S

    author Singhal, K. , author Azizi, S. , author Tu, T. , author Mahdavi, S.S. , author Wei, J. , author Chung, H.W. , et al., year 2023 . title Large language models encode clinical knowledge . journal Nature volume 620 , pages 172--180

    2023

  40. [48]

    , author Hanbury, A

    author Taha, A.A. , author Hanbury, A. , year 2015 . title Metrics for evaluating 3d medical image segmentation: analysis, selection, and tool . journal BMC medical imaging volume 15 , pages 1--28

    2015

  41. [49]

    , author Wu, S

    author Tan, H. , author Wu, S. , author Pi, J. , year 2022 . title Semantic diffusion network for semantic segmentation . journal NeurIPS volume 35 , pages 8702--8716

    2022

  42. [50]

    , author Yang, D

    author Tang, Y. , author Yang, D. , author Li, W. , author Roth, H.R. , author Landman, B. , author Xu, D. , et al., year 2022 . title Self-supervised pre-training of swin transformers for 3d medical image analysis , in: booktitle CVPR , pp. pages 20730--20740

    2022

  43. [51]

    , author Shazeer, N

    author Vaswani, A. , author Shazeer, N. , author Parmar, N. , author Uszkoreit, J. , author Jones, L. , author Gomez, A.N. , et al., year 2017 . title Attention is all you need . journal NeurIPS volume 30

    2017

  44. [52]

    , author Cao, J

    author Wang, H. , author Cao, J. , author Anwer, R.M. , author Xie, J. , author Khan, F.S. , author Pang, Y. , year 2023 . title Dformer: Diffusion-guided transformer for universal image segmentation . journal arXiv:2306.03437

    2023 arXiv

  45. [53]

    , author Guo, S

    author Wang, H. , author Guo, S. , author Ye, J. , author Deng, Z. , author Cheng, J. , author Li, T. , et al., year 2025 . title Sam-med3d: towards general-purpose segmentation models for volumetric medical images , in: booktitle ECCV , pp. pages 51--67

    2025

  46. [54]

    , author Lu, Y

    author Wang, Z. , author Lu, Y. , author Li, Q. , author Tao, X. , author Guo, Y. , author Gong, M. , author Liu, T. , year 2022 . title Cris: Clip-driven referring image segmentation , in: booktitle CVPR , pp. pages 11686--11695

    2022

  47. [55]

    , author Breit, H.C

    author Wasserthal, J. , author Breit, H.C. , author Meyer, M.T. , author Pradella, M. , author Hinck, D. , author Sauter, A.W. , et al., year 2023 . title Totalsegmentator: Robust segmentation of 104 anatomic structures in ct images . journal Radiology: Artificial Intelligence...

    2023

  48. [56]

    , author Fu, R

    author Wu, J. , author Fu, R. , author Fang, H. , author Zhang, Y. , author Yang, Y. , author Xiong, H. , author Liu, H. , author Xu, Y. , year 2024 . title Medsegdiff: Medical image segmentation with diffusion probabilistic model , in: booktitle Medical Imaging with Deep Lear...

    2024

  49. [57]

    , author Chen, Y

    author Wu, Q. , author Chen, Y. , author Yang, C. , author Yan, J. , year 2023 . title Energy-based out-of-distribution detection for graph neural networks , in: booktitle ICLR

    2023

  50. [58]

    , author Zhang, Y

    author Xia, Y. , author Zhang, Y. , author Liu, F. , author Shen, W. , author Yuille, A.L. , year 2020 . title Synthesize then compare: Detecting failures and anomalies for semantic segmentation , in: booktitle ECCV , pp. pages 145--161

    2020

  51. [59]

    , author Wang, H

    author Yu, Q. , author Wang, H. , author Qiao, S. , author Collins, M. , author Zhu, Y. , author Adam, H. , et al., year 2022 . title k-means mask transformer , in: booktitle ECCV , pp. pages 288--307

    2022

  52. [60]

    , author Xia, Y

    author Yuan, M. , author Xia, Y. , author Dong, H. , author Chen, Z. , author Yao, J. , author Qiu, M. , et al., year 2023 . title Devil is in the queries: advancing mask transformers for real-world medical image segmentation and out-of-distribution localization , in: booktitl...

    2023

  53. [61]

    , author Rosa, K.D

    author Zareian, A. , author Rosa, K.D. , author Hu, D.H. , author Chang, S.F. , year 2021 . title Open-vocabulary object detection using captions , in: booktitle CVPR , pp. pages 14393--14402

    2021

  54. [62]

    , author Herrmann, C

    author Zhang, J. , author Herrmann, C. , author Hur, J. , author Polania Cabrera, L. , author Jampani, V. , author Sun, D. , et al., year 2024 . title A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence . journal NeurIPS volume 36

    2024

  55. [63]

    , author Metaxas, D

    author Zhang, S. , author Metaxas, D. , year 2023 . title On the challenges and perspectives of foundation models for medical image analysis . journal Med. Image Anal. , pages 102996

    2023

  56. [64]

    , author Liao, B

    author Zhu, L. , author Liao, B. , author Zhang, Q. , author Wang, X. , author Liu, W. , author Wang, X. , year 2024 . title Vision mamba: Efficient visual representation learning with bidirectional state space model , in: booktitle ICML , pp. pages 62429--62442

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.