Pith. sign in

REVIEW 5 major objections 7 minor 55 references

RefSAM3D: Adapting SAM with Cross-modal Reference for 3D Medical Image Segmentation

T0 review · 5 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that adapting SAM to 3D medical volumes with cross-modal text prompts yields state-of-the-art segmentation and strong zero-shot transfer.

desk verdict The architecture is a legitimate extension of SAM to 3D medical segmentation, but the per-dataset text prompts encode acquisition statistics, so the reported SOTA and zero-shot gains are likely driven by dataset-identity conditioning, not segmentation ability. read the letter →

arxiv 2412.05605 v1 pith:7WFADZX6 submitted 2024-12-07 cs.CV

Xiang Gao , Kai Lu This is my paper · ORCID
classification cs.CV
keywords 3DmedicalimagesegmentationSegmentAnythingModelcross-modalprompttext-guidedvolumetricCTandMRIparameter-efficientadaptationhierarchicalcross-attentiontumor
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RefSAM3D attempts to make the Segment Anything Model (SAM) segment 3D medical volumes instead of 2D natural images. It adds a lightweight 3D adapter to SAM's visual encoder, converts the mask decoder to produce 3D masks, and conditions the decoder on a cross-modal prompt built by aligning CLIP text embeddings with hierarchical visual features. The paper reports state-of-the-art Dice scores on kidney, pancreas, liver, and colon tumor benchmarks and on BTCV multi-organ segmentation, plus zero-shot results on AMOS CT and MRI after training only on BTCV. The central assertion is that volumetric adaptation and text prompts together make SAM a competitive and generalizable 3D medical segmenter.

What carries the argument

The mechanism that carries the argument is a cross-modal reference prompt built by hierarchical cross-attention. The visual encoder is SAM's ViT with patch embedding decomposed into $1 \times k \times k$ and $k \times 1 \times 1$ 3D convolutions and positional embeddings extended by a learnable depth axis; lightweight adapters with depthwise 3D convolutions let the model adjust to volumes without fine-tuning all weights. Features from the four encoder stages are each projected by an adapter, then cross-attended with a sentence-level text embedding produced by the CLIP text encoder and aligned by an MLP; the concatenated outputs form the prompt for the mask decoder. The decoder replaces 2D convolutions with 3D ones, upsamples progressively to full resolution, and fuses the original image at the end.

What would settle it

Run RefSAM3D on the same benchmarks with text prompts reduced to organ names and modality only (no spacing, dimension, or slice-count statistics), keeping everything else fixed. If Dice on KiTS21, MSD, LiTS, and AMOS falls toward the no-prompt ablation level (72.3% on BTCV), the reported advantage is a prompt-leakage artifact; if it stays near the published numbers, the mechanism is semantic.

Watch

Extended reading notes

Core claim

The central claim is that a SAM-based model can be adapted to 3D medical imaging by (1) converting patch embedding and attention to accept 3D inputs while freezing most pre-trained weights, (2) inserting a 3D adapter with depthwise convolutions, and (3) conditioning the mask decoder on a cross-modal prompt generated by aligning CLIP text embeddings with hierarchical features from four encoder stages. In the paper's experiments, this design achieves 95.53% Dice on KiTS21 kidney tumor, 82.42% on MSD pancreas tumor, 80.10% on LiTS liver tumor, 70.14% on MSD colon cancer, and an average 88.3% Dice on BTCV 13-organ segmentation. After training only on BTCV CT, the same model reports zero-shot mean Dice of 85.7% on AMOS22 CT and 63.2% on AMOS22 MRI, rising to 84.1% with five fine-tuning scans. The ablation study attributes a 16-point Dice drop to removing the text prompt.

Load-bearing premise

The load-bearing premise is that the hand-written text prompts provide genuine anatomical semantics and not hidden dataset information; the prompts list spacing ranges, volume sizes, and slice counts that match the test sets, so if the model uses those to infer which dataset a volume comes from, the reported accuracy is inflated.

Editorial extensions

If this is right

  • The method reports state-of-the-art Dice on four tumor benchmarks, so a text-conditioned 3D SAM is a viable alternative to specialized segmentation networks on those tasks.
  • Training once on BTCV CT and evaluating on AMOS CT/MRI without retraining yields 85.7% and 63.2% Dice, indicating cross-dataset and cross-modality transfer from a single model.
  • With five AMOS MRI scans for fine-tuning, the reported 84.1% Dice suggests few-shot adaptation is enough to close much of the cross-modality gap.
  • Removing the text prompt drops BTCV Dice from 88.3% to 72.3%, making the text prompt a load-bearing component rather than a marginal addition.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The hand-designed prompts in Table II include dataset-specific numbers (spacing ranges, dimensions, slice counts). If the model exploits these statistics as a dataset-identity signal, the reported state-of-the-art and zero-shot gains may not reflect general segmentation ability; replacing the numbers with purely anatomical descriptions would test this.
  • The 16-point ablation gap between prompted and unprompted runs could come partly from the prompt acting as a distribution selector. A cross-dataset experiment that swaps prompts between tasks (e.g., using the colon-cancer prompt on liver volumes) would reveal whether the text contributes semantic guidance or dataset identification.
  • The same three-stage recipe (frozen 2D encoder, lightweight 3D adapter, cross-modal prompt) is general enough to apply to other interactive segmentation foundation models, letting the community check the claimed gains without rebuilding the pipeline.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes RefSAM3D, an adaptation of the Segment Anything Model (SAM) to 3D medical image segmentation. The method introduces a 3D image adapter for volumetric inputs, a cross-modal reference prompt generation module that fuses CLIP text embeddings with hierarchical visual features, and a lightweight 3D mask decoder. The authors report state-of-the-art Dice and Normalized Surface Dice scores on KiTS21, MSD Pancreas, LiTS, and MSD Colon tumor segmentation, average Dice of 88.3% on BTCV multi-organ segmentation, and zero-shot mean Dice of 85.7% on AMOS22 CT and 63.2% on AMOS22 MRI after training only on BTCV CT. Ablations show that removing the text prompt, the cross-modal projector, or the hierarchical fusion each causes a substantial performance drop.

Significance. If the results are valid, RefSAM3D would be a useful contribution: it is a parameter-efficient way to adapt a 2D foundation model to 3D medical volumes, and it reports large gains over strong baselines such as MA-SAM and nnU-Net. The paper also contains positive elements: a clearly described architecture with equations for the adapter and cross-attention, evaluations on multiple public datasets, and ablations of the main components. However, as written, the central performance claims are not verifiable or fairly interpretable. The text prompts encode dataset-specific acquisition statistics and can act as dataset-identity embeddings; the evaluation protocol is incomplete, with an unresolved reference and missing details; the dataset description for the cardiac experiment is self-contradictory; and the zero-shot experiment does not specify the test-time prompt. These issues bear directly on the paper's headline claims of consistent state-of-the-art performance and cross-modal generalization.

major comments (5)
  1. [Section IV.A, Table II, Table IV] The per-dataset text prompts in Table II contain acquisition statistics rather than purely semantic descriptions: KiTS21 includes spacing and dimensions, MSD Pancreas includes resolution and slice counts, BTCV includes slice thickness and in-plane resolution, and AMOS22 names 'CT and MRI images' with 'varying modalities and resolutions.' Since each prompt is a constant for an entire dataset and is passed through the trainable cross-modal MLP (Eq. 3) and hierarchical cross-attention (Eqs. 6-8), the text branch can act as a dataset-identity embedding rather than as anatomical guidance. The ablation in Table IV is consistent with this concern: removing the text prompt lowers BTCV Dice by 16.0 points (88.3 to 72.3), a swing much larger than the method's 1.1-point margin over MA-SAM. Please re-run the experiments with prompts that contain only organ or structure names, or otherwise demonstrate that the dataset statistics do not inflate the reported results, for example by swapping prompts across datasets.
  2. [Section IV.A (Implementation Details)] The evaluation protocol is underspecified and partly unreferenced. The phrase 'same as xxx []' is unresolved, and baseline descriptions such as 'SAM-B[]' and 'MA-SAM(1 relaxed 3D bbx/slice)' lack precise prompt construction details. The manuscript also does not specify the exact resampling, patch sampling, cropping, and inference-time prompt procedures. Without a complete, self-contained protocol and release of code and trained models, the numbers in Tables I and III cannot be independently reproduced or compared against baselines.
  3. [Section IV.A (MRI Cardiac Segmentation)] The dataset description contradicts itself: the subsection is titled 'MRI Cardiac Segmentation' and states that MM-WHS contains 20 CT and 20 MRI scans, but then says 'only the CT scans were used.' Table II gives the prompt as 'MRI images, cardiac structure segmentation,' and Section IV.C, Figure 5, and the accompanying text describe MRI cardiac segmentation. No quantitative cardiac results appear in any table. Please clarify which modality was used, report the corresponding quantitative results, and correct the text.
  4. [Section IV.C (Generalization Evaluation)] The zero-shot experiment is not well-defined because the test-time prompt is not specified. The model is trained on BTCV CT and then evaluated on AMOS22 CT and MRI; if the AMOS22 prompt from Table II (which names both modalities, 'varying modalities and resolutions', and 15 organs) is used at inference, then the model is informed of the target dataset and modality, so the reported 85.7% CT and 63.2% MRI Dice scores do not measure clean cross-dataset generalization. Please state the exact test-time prompts and, ideally, use prompts that do not encode modality or dataset-specific statistics.
  5. [Tables I, III, IV and Section IV.C] No error bars or statistical significance tests are reported for the main comparisons, and the only variance given anywhere is the single '±3.1%' in Section IV.C. The claim that RefSAM3D 'consistently outperforms' other methods (Section IV.B) is therefore not statistically supported. Please report mean and standard deviation over at least three runs or cross-validation folds, and state the number of runs used for each table.
minor comments (7)
  1. [Section IV.A] Unresolved markers 'same as xxx []', 'SAM-B[]', and 'Table ??' must be replaced with actual citations and table references.
  2. [Figure 5 caption and Section IV.B] The caption 'MRI cardical tumor segmentation' contains a typo, and the text refers to 'cardiac tumor segmentation,' but the MM-WHS task is cardiac structure segmentation, not tumor segmentation.
  3. [Section IV.B] The qualitative paragraph refers to 'our AutoSAM Adapter' when the proposed method is RefSAM3D; please correct the model name.
  4. [Section IV.D] The sentence 'HD increasing from 2.34% to 7.31%' uses percent signs for Hausdorff Distance, which is measured in mm; please fix the units.
  5. [Section IV.D] The sentence 'These results confirm When the Cross-Modal Projector is removed...' is grammatically incomplete and should be rewritten.
  6. [Section IV.A] The MSD Pancreas dataset is cited as reference [31], which is actually the Medical SAM Adapter paper; please cite the dataset source instead.
  7. [Table V] The checkmark layout is ambiguous; please label which stages are included in each row.

Circularity Check

1 steps flagged · score 5.0 of 10

Zero-shot and SOTA gains are partly conditioned by per-dataset text prompts that encode target acquisition statistics; no mathematical derivation is circular, but the generalization claim is not a clean prediction.

  1. fitted input called prediction [Section IV.C (Generalization Evaluation) with Table II and Eqs. (6)-(8)]
    "In the zero-shot generalization experiments, we evaluated the model's transferability by applying our Ref-SAM3D, trained exclusively on the BTCV CT dataset, to the AMOS 22 dataset without any additional fine-tuning. ... For each dataset, we designed specific text prompts to guide the segmentation process, as shown in Table ??. ... AMOS 22 Dataset: CT and MRI images, abdominal organ segmentation (15 organs), varying modalities and resolutions."

    The zero-shot experiment is the paper's evidence for generalization, but the test-time input is not the fixed BTCV prompt: it is the AMOS22-specific prompt that announces the target modality ('CT and MRI images') and target label count ('15 organs'). Because the text prompt is encoded by CLIP and injected as keys/values in the hierarchical cross-attention (Eqs. 6-8), the model's prediction is conditioned on target-dataset identity and statistics. The 'generalization' result is therefore not a prediction from an unchanged model input; it is a prediction from an input hand-designed using knowledge of the target dataset. This is a mild form of fitted-input-called-prediction: the prompt is manually fitted per dataset rather than learned, but it functions as a dataset-identity embedding.

full rationale

RefSAM3D is an empirical adaptation paper, not a derivation, so there is no equation-level circularity, no uniqueness theorem, and no load-bearing self-citation chain; the cited RefSAM is external work by Li et al. The only step approaching circularity is the zero-shot generalization claim in Section IV.C. Table II shows prompts are per-dataset constants containing target acquisition statistics, including 'CT and MRI images' and '15 organs' for AMOS22. Since those prompts flow through Eq. (3) and the cross-attention of Eqs. (6)-(8), the zero-shot evaluation is conditioned on the target distribution's identity, making the reported 85.7% / 63.2% numbers not a clean zero-shot test. I would not call the in-domain SOTA comparisons circular, because every baseline is also trained on the target dataset; the concern is specific to the generalization evidence. Additional verification issues, such as the missing citation 'same as xxx []' and the lack of released code or a precise test-time prompt protocol, are correctness risks rather than circularity. Overall, the central SOTA claim has independent content, but one key 'prediction' is partially circular by protocol construction.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The paper proposes no new physical entities. The main uncharged inputs are the hand-crafted text prompts, which embed dataset statistics, and the architectural assumption that frozen 2D SAM weights can be adapted to 3D via lightweight adapters.

free parameters (1)
  • Per-dataset text prompt (hand-crafted) = Table II prompt strings
    Prompts include dataset-specific statistics such as spacing ranges and volume dimensions, conditioning the model on the target distribution. This hand-chosen input may leak dataset identity and inflate reported performance.
assumptions (3)
  • domain assumption SAM's pre-trained 2D weights transfer to 3D volumetric inputs when combined with 3D adapters
    Section III.B assumes the frozen 2D ViT weights plus learned 3D adapters can represent volumetric features effectively.
  • ad hoc to paper A 1xkxk convolution followed by a kx1x1 3D convolution approximates a kxkxk convolution
    Section III.B claims this approximation, but it only holds for separable kernels; no justification is provided.
  • domain assumption Text prompts such as dataset statistics provide useful semantic guidance without leaking test-set identity
    Section IV.A and Table II use prompts with spacing and dimension ranges; the assumption that this is legitimate conditioning is unstated and questionable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RefSAM3D: Adapting SAM with Cross-modal Reference for 3D Medical Image Segmentation." pith.science (2026). https://pith.science/paper/7WFADZX6

@misc{pith2026241205605,
  author       = {Pith},
  title        = {Pith review of: RefSAM3D: Adapting SAM with Cross-modal Reference for 3D Medical Image Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7WFADZX6}},
  note         = {Machine review of arXiv:2412.05605}
}
read the original abstract

The Segment Anything Model (SAM), originally built on a 2D Vision Transformer (ViT), excels at capturing global patterns in 2D natural images but struggles with 3D medical imaging modalities like CT and MRI. These modalities require capturing spatial information in volumetric space for tasks such as organ segmentation and tumor quantification. To address this challenge, we introduce RefSAM3D, which adapts SAM for 3D medical imaging by incorporating a 3D image adapter and cross-modal reference prompt generation. Our approach modifies the visual encoder to handle 3D inputs and enhances the mask decoder for direct 3D mask generation. We also integrate textual prompts to improve segmentation accuracy and consistency in complex anatomical scenarios. By employing a hierarchical attention mechanism, our model effectively captures and integrates information across different scales. Extensive evaluations on multiple medical imaging datasets demonstrate the superior performance of RefSAM3D over state-of-the-art methods. Our contributions advance the application of SAM in accurately segmenting complex anatomical structures in medical imaging.

Figures

Figures reproduced from arXiv: 2412.05605 by the authors.

Figure 1
Figure 1. (A) The overview of our proposed RefSAM3D for 3D medical image segmentation, which integrates hierarchical cross-attention between image and [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The structure of the Cross-Modal Prompt Embedding module. 1) The left part illustrates the overall architecture, where hierarchical visual embeddings [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Qualitative visualizations of the proposed method and baseline approaches on liver tumor, kidney tumor, pancreas tumor and colon cancer segmentation [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Qualitative visualization of segmentation results generated from our Ref-SAM3D method and other state-of-the-art methods on BTCV dataset. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Qualitative visualization of segmentation results generated from different methods for MRI cardical tumor segmentation [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Comparison of zero-shot and five-shot generalization performance of [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 47 canonical work pages

  1. [1]

    Clinical applications of artificial intelligence in medical imaging and image processing—a review,

    R. Obuchowicz, M. Strzelecki, and A. Pi ´orkowski, “Clinical applications of artificial intelligence in medical imaging and image processing—a review,” Cancers, vol. 16, no. 10, pp. 1870, 2024

    work page 2024

  2. [2]

    Deep learning-enhanced image segmentation for medical diagnostics,

    S. Addimulam, M. A. Mohammed, R. K. Karanam, et al., “Deep learning-enhanced image segmentation for medical diagnostics,” Malaysian Journal of Medical and Biological Research , vol. 7, no. 2, pp. 145–152, 2020

    work page 2020

  3. [3]

    Ai in diagnostic imaging: Revolutionising accuracy and efficiency,

    Mohamed Khalifa and Mona Albadawy, “Ai in diagnostic imaging: Revolutionising accuracy and efficiency,” Computer Methods and Programs in Biomedicine Update , vol. 5, 2024

    work page 2024

  4. [4]

    Segment anything,

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al., “Segment anything,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 4015–4026

    work page 2023

  5. [5]

    Segment everything everywhere all at once,

    Xueyan Zou, Jianwei Yang, Hao Zhang, Feng Li, Linjie Li, Jianfeng Wang, Lijuan Wang, Jianfeng Gao, and Yong Jae Lee, “Segment everything everywhere all at once,” 2023

    work page 2023

  6. [6]

    Segment anything model for medical images?,

    Yuhao Huang, Xin Yang, Lian Liu, Han Zhou, Ao Chang, Xinrui Zhou, Rusi Chen, Junxuan Yu, Jiongquan Chen, et al., “Segment anything model for medical images?,” arXiv preprint, 2023

    work page 2023

  7. [7]

    Lora: Low-rank adaptation of large language models,

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen, “Lora: Low-rank adaptation of large language models,” CoRR, vol. abs/2106.09685, 2021

    arXiv 2021

  8. [8]

    Adapters: A unified library for parameter-efficient and modular transfer learning,

    Clifton Poth, Hannah Sterz, Indraneil Paul, Sukannya Purkayastha, Leon Engl ¨ander, Timo Imhof, Ivan Vulic, Sebas- tian Ruder, Iryna Gurevych, and Jonas Pfeiffer, “Adapters: A unified library for parameter-efficient and modular transfer learning,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023 - Sys- ...

    work page 2023

Show all 55 references
  1. [9]

    Med- tuning: A new parameter-efficient tuning framework for medical volumetric segmentation,

    Jiachen Shen, Wenxuan Wang, Chen Chen, Jianbo Jiao, Jing Liu, Yan Zhang, Shanshan Song, and Jiangyun Li, “Med- tuning: A new parameter-efficient tuning framework for medical volumetric segmentation,” 2024

    2024

  2. [10]

    Customized segment anything model for medical image segmentation,

    Kaidong Zhang and Dong Liu, “Customized segment anything model for medical image segmentation,” 2023

    2023

  3. [11]

    Sam-med3d: Towards general-purpose segmentation models for volumetric medical images,

    Haoyu Wang, Sizheng Guo, Jin Ye, Zhongying Deng, Junlong Cheng, Tianbin Li, Jianpin Chen, Yanzhou Su, Ziyan Huang, Yiqing Shen, Bin Fu, Shaoting Zhang, Junjun He, and Yu Qiao, “Sam-med3d: Towards general-purpose segmentation models for volumetric medical images,” 2024

    2024

  4. [13]

    3dsam- adapter: Holistic adaptation of sam from 2d to 3d for promptable tumor segmentation,

    Shizhan Gong, Yuan Zhong, Wenao Ma, Jinpeng Li, Zhao Wang, Jingyang Zhang, Pheng-Ann Heng, and Qi Dou, “3dsam- adapter: Holistic adaptation of sam from 2d to 3d for promptable tumor segmentation,” Medical Image Analysis , vol. 98, pp. 103324, Dec. 2024

    2024

  5. [14]

    Masksam: Towards auto-prompt sam with mask classification for medical image segmentation,

    Bin Xie, Hao Tang, Bin Duan, Dawen Cai, and Yan Yan, “Masksam: Towards auto-prompt sam with mask classification for medical image segmentation,” 2024

    2024

  6. [15]

    Autoprosam: Automated prompting sam for 3d multi-organ segmentation,

    Chengyin Li, Prashant Khanduri, Yao Qiang, Rafi Ibn Sultan, Indrin Chetty, and Dongxiao Zhu, “Autoprosam: Automated prompting sam for 3d multi-organ segmentation,” 2024

    2024

  7. [16]

    Towards segment anything model (sam) for medical image segmentation: A survey,

    Yichi Zhang and Rushi Jiao, “Towards segment anything model (sam) for medical image segmentation: A survey,” 2023

    2023

  8. [17]

    Segment anything in medical images,

    Jun Ma, Yuting He, Feifei Li, Lin Han, Chenyu You, and Bo Wang, “Segment anything in medical images,” Nature Communications, vol. 15, no. 1, Jan. 2024

    2024

  9. [18]

    Autosam: Adapting sam to medical images by overloading the prompt encoder,

    Tal Shaharabany, Aviad Dahan, Raja Giryes, and Lior Wolf, “Autosam: Adapting sam to medical images by overloading the prompt encoder,” 2023

    2023

  10. [19]

    Segment any cell: A sam-based auto-prompting fine- tuning framework for nuclei segmentation,

    Saiyang Na, Yuzhi Guo, Feng Jiang, Hehuan Ma, and Junzhou Huang, “Segment any cell: A sam-based auto-prompting fine- tuning framework for nuclei segmentation,” 2024

    2024

  11. [20]

    Recent advances in natural language processing via large pre-trained language models: A survey,

    Bonan Min, Hayley Ross, Elior Sulem, Amir Pouran Ben Veyseh, Thien Huu Nguyen, Oscar Sainz, Eneko Agirre, Ilana Heintz, and Dan Roth, “Recent advances in natural language processing via large pre-trained language models: A survey,” CoRR, vol. abs/2111.01243, 2021

    2021 arXiv

  12. [21]

    Learning transferable visual models from natural language supervision,

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever, “Learning transferable visual models from natural language supervision,” CoRR, vol. abs/210...

    2021 arXiv

  13. [22]

    Scaling up visual and vision-language repre- sentation learning with noisy text supervision,

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V . Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig, “Scaling up visual and vision-language repre- sentation learning with noisy text supervision,” CoRR, vol. abs/2102.05918, 2021

    2021 arXiv

  14. [23]

    Segment everything everywhere all at once,

    Xueyan Zou, Jianwei Yang, Hao Zhang, Feng Li, Linjie Li, Jianfeng Wang, Lijuan Wang, Jianfeng Gao, and Yong Jae Lee, “Segment everything everywhere all at once,” in Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems...

    2023

  15. [24]

    Seggpt: Segmenting everything in context,

    Xinlong Wang, Xiaosong Zhang, Yue Cao, Wen Wang, Chunhua Shen, and Tiejun Huang, “Seggpt: Segmenting everything in context,” 2023

    2023

  16. [25]

    Dinov2: Learning robust visual features without supervision,

    Maxime Oquab, Timoth ´ee Darcet, and etc. Th ´eo Moutakanni, “Dinov2: Learning robust visual features without supervision,” 2024

    2024

  17. [26]

    Detect any shadow: Segment anything for video shadow detection,

    Yonghui Wang, Wengang Zhou, Yunyao Mao, and Houqiang Li, “Detect any shadow: Segment anything for video shadow detection,” 2023

    2023

  18. [27]

    Segment anything model (sam) for digital pathology: Assess zero-shot segmentation on whole slide imaging,

    R. Deng, C. Cui, Q. Liu, T. Yao, L. W. Remedios, S. Bao, B. A. Landman, L. E. Wheless, L. A. Coburn, K. T. Wilson, et al., “Segment anything model (sam) for digital pathology: Assess zero-shot segmentation on whole slide imaging,” arXiv preprint, 2023

    2023

  19. [28]

    Accuracy of segment- anything model (sam) in medical image segmentation tasks,

    Sheng He, Rina Bao, Jingpeng Li, Jeffrey Stout, Atle Bjørnerud, P. Ellen Grant, and Yangming Ou, “Accuracy of segment- anything model (sam) in medical image segmentation tasks,” arXiv preprint, 2023

    2023

  20. [29]

    When sam meets medical images: An investigation of segment anything model (sam) on multi-phase liver tumor segmentation,

    C. Hu and X. Li, “When sam meets medical images: An investigation of segment anything model (sam) on multi-phase liver tumor segmentation,” 2023

    2023

  21. [30]

    Can sam segment polyps?,

    Tao Zhou, Yizhe Zhang, Yi Zhou, Ye Wu, and Chen Gong, “Can sam segment polyps?,” arXiv preprint, 2023

    2023

  22. [31]

    Medical sam adapter: Adapting segment anything model for medical image segmentation,

    Junde Wu, Wei Ji, Yuanpei Liu, Huazhu Fu, Min Xu, Yanwu Xu, and Yueming Jin, “Medical sam adapter: Adapting segment anything model for medical image segmentation,” 2023

    2023

  23. [32]

    Sam-med2d,

    Junlong Cheng, Jin Ye, Zhongying Deng, Jianpin Chen, Tianbin Li, Haoyu Wang, Yanzhou Su, Ziyan Huang, Jilong Chen, Lei Jiang, Hui Sun, Junjun He, Shaoting Zhang, Min Zhu, and Yu Qiao, “Sam-med2d,” 2023

    2023

  24. [33]

    Medlsam: Localize and segment anything model for 3d ct images,

    Wenhui Lei, Xu Wei, Xiaofan Zhang, Kang Li, and Shaoting Zhang, “Medlsam: Localize and segment anything model for 3d ct images,” 2024

    2024

  25. [34]

    Sam3d: Segment anything in 3d scenes,

    Yunhan Yang, Xiaoyang Wu, Tong He, Hengshuang Zhao, and Xihui Liu, “Sam3d: Segment anything in 3d scenes,” 2023

    2023

  26. [35]

    Ma-sam: Modality-agnostic sam adaptation for 3d medical image segmentation,

    Cheng Chen, Juzheng Miao, Dufan Wu, Zhiling Yan, Sekeun Kim, Jiang Hu, Aoxiao Zhong, Zhengliang Liu, Lichao Sun, Xiang Li, Tianming Liu, Pheng-Ann Heng, and Quanzheng Li, “Ma-sam: Modality-agnostic sam adaptation for 3d medical image segmentation,” 2023

    2023

  27. [36]

    St-adapter: Parameter-efficient image-to-video transfer learning,

    Junting Pan, Ziyi Lin, Xiatian Zhu, Jing Shao, and Hongsheng Li, “St-adapter: Parameter-efficient image-to-video transfer learning,” 2022

    2022

  28. [37]

    Conditional prompt learning for vision-language models,

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu, “Conditional prompt learning for vision-language models,” 2022

    2022

  29. [38]

    Visual prompt tuning,

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim, “Visual prompt tuning,” 2022

    2022

  30. [39]

    Learning transferable visual models from natural language supervision,

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al., “Learning transferable visual models from natural language supervision,” in International conference on machine learning ...

    2021

  31. [40]

    Scaling up visual and vision-language representation learning with noisy text supervision,

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig, “Scaling up visual and vision-language representation learning with noisy text supervision,” in International confer- ence on machine learning . PMLR, 2021,...

    2021

  32. [41]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    Alexey Dosovitskiy, “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929, 2020

    2010 arXiv

  33. [42]

    Vision-language transformer and query generation for referring segmentation,

    Henghui Ding, Chang Liu, Suchen Wang, and Xudong Jiang, “Vision-language transformer and query generation for referring segmentation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 16321–16330

    2021

  34. [43]

    Refsam: Efficiently adapting segmenting anything model for referring video object segmentation,

    Yonglin Li, Jing Zhang, Xiao Teng, Long Lan, and Xinwang Liu, “Refsam: Efficiently adapting segmenting anything model for referring video object segmentation,” 2024

    2024

  35. [44]

    The kits21 challenge: Automatic segmentation of kidneys, renal tumors, and renal cysts in corticomedullary-phase ct,

    Nicholas Heller, Fabian Isensee, and etc. Dasha Trofimova, “The kits21 challenge: Automatic segmentation of kidneys, renal tumors, and renal cysts in corticomedullary-phase ct,” 2023

    2023

  36. [45]

    The liver tumor segmentation benchmark (lits),

    Patrick Bilic, Patrick Christ, Hongwei Bran Li, Eugene V orontsov, Avi Ben-Cohen, and etc Kaissis, “The liver tumor segmentation benchmark (lits),” Medical Image Analysis , vol. 84, pp. 102680, Feb. 2023

    2023

  37. [46]

    The medical segmentation decathlon,

    Michela Antonelli, Annika Reinke, Spyridon Bakas, Keyvan Farahani, Annette Kopp-Schneider, Bennett A. Landman, Geert Litjens, Bjoern Menze, Olaf Ronneberger, Ronald M. Summers, et al., “The medical segmentation decathlon,” Nature Commu- nications, vol. 13, no. 1, pp. 4128, 2022

    2022

  38. [47]

    Evaluation of algorithms for multi-modality whole heart segmentation: An open-access grand challenge,

    Xiahai Zhuang, Lei Li, and etc Christian Payer, “Evaluation of algorithms for multi-modality whole heart segmentation: An open-access grand challenge,” 2019

    2019

  39. [48]

    Miccai multi-atlas labeling beyond the cranial vault–workshop and challenge,

    Bennett Landman, Zhoubing Xu, J. Iglesias, Martin Styner, T. Langerak, and Arno Klein, “Miccai multi-atlas labeling beyond the cranial vault–workshop and challenge,” in Proc. MICCAI Multi-Atlas Labeling Beyond Cranial Vault—Workshop Challenge, 2015, vol. 5, p. 12

    2015

  40. [49]

    Self-supervised pre-training of swin transformers for 3d med- ical image analysis,

    Yucheng Tang, Dong Yang, Wenqi Li, Holger Roth, Bennett Landman, Daguang Xu, Vishwesh Nath, and Ali Hatamizadeh, “Self-supervised pre-training of swin transformers for 3d med- ical image analysis,” 2022

    2022

  41. [50]

    Amos: A large-scale abdominal multi- organ benchmark for versatile medical image segmentation,

    Yuanfeng Ji, Haotian Bai, Jie Yang, Chongjian Ge, Ye Zhu, Ruimao Zhang, Zhen Li, Lingyan Zhang, Wanling Ma, Xiang Wan, and Ping Luo, “Amos: A large-scale abdominal multi- organ benchmark for versatile medical image segmentation,” 2022

    2022

  42. [51]

    nnu-net: Self-adapting framework for u-net-based medical im- age segmentation,

    Fabian Isensee, Jens Petersen, Andre Klein, David Zimmerer, Paul F. Jaeger, Simon Kohl, Jakob Wasserthal, Gregor Koehler, Tobias Norajitra, Sebastian Wirkert, and Klaus H. Maier-Hein, “nnu-net: Self-adapting framework for u-net-based medical im- age segmentation,” 2018

    2018

  43. [52]

    U-net: Convolutional networks for biomedical image segmentation,

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention - MICCAI, Nassir Navab, Joachim Hornegger, William M. Wells III, and Alejandro F. Frangi, Eds., 2015

    2015

  44. [53]

    Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images,

    Ali Hatamizadeh, Vishwesh Nath, Yucheng Tang, Dong Yang, Holger Roth, and Daguang Xu, “Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images,” 2022

    2022

  45. [54]

    nnformer: V olumetric medical image segmentation via a 3d transformer,

    Hong-Yu Zhou, Jiansen Guo, Yinghao Zhang, Xiaoguang Han, Lequan Yu, Liansheng Wang, and Yizhou Yu, “nnformer: V olumetric medical image segmentation via a 3d transformer,” IEEE Trans. Image Process. , vol. 32, pp. 4036–4045, 2023

    2023

  46. [55]

    Unetr++: Delving into efficient and accurate 3d medical image segmentation,

    Abdelrahman Shaker, Muhammad Maaz, Hanoona Rasheed, Salman Khan, Ming-Hsuan Yang, and Fahad Shahbaz Khan, “Unetr++: Delving into efficient and accurate 3d medical image segmentation,” 2024

    2024

  47. [56]

    3d ux-net: A large kernel volumetric convnet modernizing hierarchical transformer for medical image seg- mentation,

    Ho Hin Lee, Shunxing Bao, Yuankai Huo, and Bennett A. Landman, “3d ux-net: A large kernel volumetric convnet modernizing hierarchical transformer for medical image seg- mentation,” 2023

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.