Pith. sign in

REVIEW 2 major objections 5 minor 14 references

MCP-MedSAM: A Powerful Lightweight Medical Segment Anything Model Trained with a Single GPU in Just One Day

T0 review · 2 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A lightweight medical SAM outsegments heavier rivals after one day of single-GPU training, using modality and content prompts plus modality-balanced sampling.

desk verdict A solid, reproducible lightweight MedSAM paper whose headline 87.50 DSC is honest but selected: the same validation set was used both to tune and to report results. read the letter →

arxiv 2412.05888 v3 pith:JLH722MH submitted 2024-12-08 cs.CV

Donghang Lyu , Ruochen Gao , Marius Staring This is my paper · ORCID
classification cs.CV
keywords medicalimagesegmentationSegmentAnythingModellightweightmodalitypromptcontentmodality-balancedsamplingsingle-GPUtrainingDicesimilaritycoefficient
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MCP-MedSAM sets out to show that a medical Segment Anything Model does not need large GPU clusters or weeks of training to be competitive: the authors train a lightweight SAM variant on a single 40GB A100 GPU in 23.8 hours and report the best Dice (87.50) and Normalized Surface Dice (89.40) among the lightweight models compared on the CVPR 2024 laptop challenge data. The improvement comes from two added prompt types that inject information the standard box prompt omits, plus a sampling scheme that balances the eleven imaging modalities during training. A sympathetic reading is that prior knowledge about the imaging modality and about the target inside the bounding box can substitute, in large part, for a bigger image encoder. The result matters because it lowers the compute barrier for building general-purpose medical segmentation models.

What carries the argument

The load-bearing additions are two prompt embeddings and a sampler. The modality prompt fuses a frozen CLIP text embedding of a '{modality} image' description with a learnable embedding drawn from a per-modality pool, adding modality-specific bias through FiLM-style weight and bias modulation of the mask decoder's dense features; an auxiliary classifier forces this embedding to carry modality identity. The content prompt crops the region inside the box, resizes it, and produces both a sparse embedding via a frozen CLIP image encoder and a dense embedding via a small ResNet-style CNN, with a contrastive loss aligning the two. The modality-based data sampling strategy (Algorithm 1) selects, for each batch slot, a uniform-random modality, then a case and a slice from that modality, giving underrepresented modalities like PET equal representation in every batch. Together these carry the argument: the prompts supply information the box alone lacks, and the sampler replaces the heavily skewed raw slice distribution, in which CT slices account for roughly 76% of the dataset.

What would settle it

Run the exact MCP-MedSAM training recipe on a held-out split of the challenge data, or on the eventual hidden test set, and compare DSC and NSD against DAFT, Swin-LiteMedSAM, and MedficientSAM; if the margin over DAFT (87.50 vs 87.18 DSC, 89.40 vs 88.32 NSD) does not survive outside the validation set, the claimed superiority is a selection artifact. A cheaper check is to keep the architecture fixed, switch modality sampling back to slice sampling, and measure whether the PET DSC gain (66.21 to 73.38) and the overall 87.50 figure reproduce on held-out PET data.

Watch

Extended reading notes

Core claim

The authors claim that, building on LiteMedSAM's tiny ViT image encoder, adding a modality prompt (a frozen medical-domain CLIP text embedding combined with a learnable per-modality embedding) and a content prompt (a cropped-and-resized box image encoded by a frozen CLIP image encoder for a sparse embedding and by a small ResNet-style CNN for a dense embedding) into the SAM prompt encoder, with both prompt representations fused into the mask decoder, gives a lightweight model that outperforms the compared lightweight medical SAMs on the challenge validation set used as test. They further claim that a modality-based data sampling strategy, which samples each modality roughly equally within every batch, is responsible for both the overall gain and the more balanced per-modality results, particularly lifting PET performance. With these choices, MCP-MedSAM reaches 87.50 +/- 6.91 DSC and 89.40 +/- 10.37 NSD, statistically significantly above the baselines, while training in 23.8 hours, an order of magnitude less GPU time than the compared methods.

Load-bearing premise

The reported accuracy is measured on the challenge's validation set used as the test set, and the data-sampling strategy and loss weights were chosen after comparing options on that same set, so the headline advantage is partly selected rather than fully out-of-sample.

Editorial extensions

If this is right

  • MCP-MedSAM reproduces the reported 87.50 DSC and 89.40 NSD with only 23.8 hours on one A100, so state-of-the-art medical SAM training no longer requires a large GPU cluster.
  • The ablations show each added prompt contributes independently, so the design should transfer to other box-prompt SAM variants without changing the image encoder.
  • The modality-based sampling strategy trades a small drop on CT and MR for a large gain on underrepresented modalities, for example PET DSC rising from 66.21 to 73.38, making per-modality performance more balanced.
  • CPU inference time is 4.6 seconds per case, slower than DAFT's 0.4 seconds, a cost the authors accept in exchange for accuracy and fast training.
  • Training on a 24GB RTX 6000 with batch size 8 still reaches 86.87 DSC and 88.34 NSD in 54.6 hours, so the approach degrades gracefully when only a mid-range GPU is available.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors do not test this, but if the content prompt's cropped-box image is the main source of the gain, the same idea could transfer to any box-conditioned segmentation model, including SAM2-based medical models, without requiring CLIP.
  • The modality-sampling result suggests that for any multi-modal medical segmentation dataset with skewed slice counts, per-modality batch balancing may matter as much as architecture choice; this is testable by swapping only the sampler in a standard training pipeline.
  • Because the competition validation set was used as the test set and the sampler and loss weights were chosen after comparing options on this set, the reported margins over DAFT and other methods may shrink on a truly held-out set; the practical claim to watch is out-of-sample DSC, not the leaderboard number.
  • The modality classification auxiliary loss means the model needs a modality label at inference time; in clinical use that label may be uncertain, and the paper does not quantify how sensitive the final mask is to a wrong modality label.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes MCP-MedSAM, a lightweight adaptation of the MedSAM/LiteMedSAM framework that adds two auxiliary prompt types (a modality prompt combining a CLIP text embedding with a learnable modality embedding, and a content prompt derived from the cropped bounding-box image via a frozen CLIP image encoder and a small CNN) and a modality-stratified data sampling strategy. The model is trained on the CVPR 2024 "Segment Anything in Medical Images on Laptop" challenge training set and evaluated on the competition validation set, which is used as the testing set because the original test set was never released. The authors report a DSC of 87.50 ± 6.91 and an NSD of 89.40 ± 10.37 on this set, which is the best among the six compared lightweight MedSAM variants, and a single-A100 GPU training time of 23.8 hours. The paper also provides ablations for the prompt components, pretraining choices, and data sampling strategies.

Significance. If the reported numbers were true out-of-sample estimates, this would be a practically valuable contribution: it demonstrates that a lightweight medical SAM can be trained in one GPU-day on a large multi-modal dataset while matching or exceeding the accuracy of heavier baselines. The authors should be credited for releasing code and Docker-based reproductions of benchmark predictions, which allowed independent verification of the main accuracy numbers. The central weakness is that the evaluation set was also used for model selection, so the headline 'best among compared models' claim is not an unbiased generalization estimate. That issue is load-bearing, but it is addressable by re-analyzing with a proper holdout split or by transparently framing the results as leaderboard-style selected estimates.

major comments (2)
  1. [Section 3.1, Section 4.3.2, Tables 3–5 and Eq. (1)] The competition validation set is used as the testing set, and the same 3,278 samples were used for model selection: the modality sampling strategy was chosen because it gave the best overall DSC/NSD on this set (Table 5), the prompt components were selected by comparison on this set (Table 3), the pretrained weights were selected on this set (Table 4), and the auxiliary loss weights (lambda3 = lambda4 = 0.01 in Eq. (1)) and learning rate schedule were fixed using this set. Therefore the reported 87.50 DSC and 89.40 NSD are selected-model estimates, not out-of-sample estimates, and the Wilcoxon p-values in Table 1 do not account for the selection process. This weakens the central claim of 'best among compared lightweight models.' The authors should either carve out a truly held-out split for the final evaluation after all ablations, use repeated or nested evaluation, or explicitly re-frame the numbers as leaderboard-style selected results and discuss the likely upward bias.
  2. [Section 4.3.1, Table 2] The claim that MCP-MedSAM requires the shortest GPU training time rests on heterogeneous sources and estimates: GPU training times are 'sourced from the respective papers,' the LiteMedSAM-Rep and Rep-MedSAM times are converted to A100-equivalents using a generic Lambda benchmark ratio, and the DAFT time is reported only as '> 42.9' because one training stage is unspecified. Since the one-day training is a headline contribution, the efficiency comparison should be measured under a standardized protocol (same GPU, same data-loading pipeline, same optimization schedule), or the estimates should be clearly labeled with their assumptions and uncertainty. The 23.8-hour figure for MCP-MedSAM itself appears measured and reproducible, but the relative efficiency claim is considerably weaker than Table 2 suggests.
minor comments (5)
  1. [Section 1 and Section 4.3.1] There are several typos, including 'zero-short capabilities' (should be 'zero-shot'), 'adaption' (should be 'adaptation'), and inconsistent spelling of 'MedficientSAM' versus 'MedificientSAM'.
  2. [Section 3.3.2 and Table 5] The statement that the modality sampling strategy was 'determined through comparisons with other commonly used data sampling strategies' should explicitly reference Table 5 and state that the comparison was performed on the same validation set later used as the test set.
  3. [Section 3.2.4, Eq. (6)–(8)] Because sim2 is the transpose of sim1, the two cross-entropy terms in Eq. (8) are not independent; the authors should clarify whether this redundancy is intentional and how the labels y are aligned for the transposed similarity matrix.
  4. [Table 2] The baseline row is empty; the text should state explicitly that training and inference times for the challenge baseline were not disclosed and therefore are omitted, rather than leaving the dashes unexplained.
  5. [Algorithm 1] The notation in step 15 should clarify that K is the number of masks associated with the selected slice, and whether the same slice is reused or sampled with replacement when a slice contains multiple independent masks.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical architecture-and-training contribution whose reported comparisons rest on independently reproduced baselines and external pretrained components.

full rationale

MCP-MedSAM does not claim a first-principles derivation; its contributions are an architecture, two auxiliary prompt mechanisms, and a sampling strategy trained end-to-end on a public challenge dataset. The benchmark comparisons in Table 1 use Docker images provided by the respective authors, and the pretrained components are external (LiteMedSAM tiny ViT and PubMedCLIP), so no load-bearing step reduces to a self-citation or to a fitted parameter renamed as a prediction. The auxiliary losses (modality classification and contrastive alignment) support training but are not fitted to the reported DSC/NSD values. The only methodological caveat is Section 3.1: 'As the testing set was not released after the challenge, we used the competition's validation set as our testing set,' and design choices such as the modality sampling strategy in Table 5 were selected on that same set. This is a real generalization-risk and selection-bias concern, but it is not circularity under the specified definition: the reported scores are measured on the chosen configuration, not derived from the selection criterion by construction, and no equation or fitted parameter equates the prediction with its input. Accordingly, no enumerated circular step is present.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

All model weights are trained from data in the usual way; the free parameters above are the hand-set hyperparameters and learned lookup tables that the central performance claim depends on. No new physical or scientific entities are postulated; the modality and content prompts are architectural components, not entities with independent falsifiable handles.

free parameters (5)
  • Auxiliary loss weights lambda1..lambda4 = lambda1=lambda2=1, lambda3=lambda4=0.01
    Set by hand in Eq. (1); the balance between segmentation, IoU, modality classification, and contrastive losses affects final DSC.
  • Modality embedding pool size = N=11, F chosen such that MLP output is 256
    The per-modality learnable embeddings in Section 3.2.1 are a hand-sized, data-fit lookup table; their content is learned during training.
  • Learning rate schedule = 2e-4 initial, decay 0.9 per 5 epochs, final 5e-5
    Chosen in Section 4.1; no principled derivation, and it affects convergence within the one-day budget.
  • Batch size and number of epochs = 16 for A100, 8 for RTX 6000; 25 epochs
    Picked to fit the one-day training target; the RTX 6000 experiment shows batch size affects final performance.
  • Prompt MLP embedding dimension = 256
    Prompt MLPs output 256-dimensional embeddings; this capacity choice is manual and not independently justified.
assumptions (4)
  • domain assumption The competition validation set is an acceptable substitute for the unavailable test set.
    Section 3.1 states the official test set was not released, so the validation set is used as the testing set for all accuracy comparisons.
  • domain assumption Pretrained medical weights from LiteMedSAM and PubMedCLIP transfer useful priors to MCP-MedSAM.
    Section 3.3.1 relies on these pretrained components to accelerate convergence; Table 4 shows ablations supporting this, but the benefit is assumed across all nine evaluated modalities.
  • domain assumption A modality label and a bounding box are always available at inference time.
    Both new prompts depend on knowing the modality (index and text) and having a box to crop, as described in Section 3.2; this limits application to box-prompted, modality-known settings.
  • domain assumption Balanced within-batch sampling of modalities improves overall performance without harming clinically relevant subpopulations.
    Algorithm 1 and Table 5 validate this on the validation set, but the choice of the best strategy is made on the same set used to report final performance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MCP-MedSAM: A Powerful Lightweight Medical Segment Anything Model Trained with a Single GPU in Just One Day." pith.science (2026). https://pith.science/paper/JLH722MH

@misc{pith2026241205888,
  author       = {Pith},
  title        = {Pith review of: MCP-MedSAM: A Powerful Lightweight Medical Segment Anything Model Trained with a Single GPU in Just One Day},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JLH722MH}},
  note         = {Machine review of arXiv:2412.05888}
}
read the original abstract

Medical image segmentation involves partitioning medical images into meaningful regions, with a focus on identifying anatomical structures and lesions. It has broad applications in healthcare, and deep learning methods have enabled significant advancements in automating this process. Recently, the introduction of the Segmentation Anything Model (SAM), the first foundation model for segmentation task, has prompted researchers to adapt it for the medical domain to improve performance across various tasks. However, SAM's large model size and high GPU requirements hinder its scalability and development in the medical domain. In this work, we propose MCP-MedSAM, a powerful and lightweight medical SAM model designed to be trainable on a single A100 GPU with 40GB of memory within one day while delivering superior segmentation performance. Recognizing the significant internal differences between modalities and the need for direct segmentation target information within bounding boxes, we introduce two kinds of prompts: the modality prompt and the content prompt. After passing through the prompt encoder, their embedding representations can further improve the segmentation performance by incorporating more relevant information without adding significant training overhead. Additionally, we adopt an effective modality-based data sampling strategy to address data imbalance between modalities, ensuring more balanced performance across all modalities. Our method was trained and evaluated using a large-scale challenge dataset, compared to top-ranking methods on the challenge leaderboard, MCP-MedSAM achieved superior performance while requiring only one day of training on a single GPU. The code is publicly available at \textcolor{blue}{https://github.com/dong845/MCP-MedSAM}.}

Figures

Figures reproduced from arXiv: 2412.05888 by the authors.

Figure 1
Figure 1. Examples of various medical imaging modalities. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Data distribution across imaging modalities in [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. We introduce two additional prompts into the [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 3
Figure 3. Figure 3: The top section provides an overview of MCP-MedSAM, highlighting our newly introduced content prompt and [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]
Figure 4
Figure 4. Figure 4: Visualization of multiple modalities yielded by our proposed method and the other benchmark models. The [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Visualizations generated from multiple imaging modalities using three prompting strategies: modality-only, [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 3 canonical work pages

  1. [2]

    arXiv preprint arXiv:2408.00874,

    arXiv

  2. [3]

    Does clip benefit visual question answering in the medical domain as much as it does in the general domain? arXiv preprint arXiv:2112.13906,

    Sedigheh Eslami, Gerard de Melo, and Christoph Meinel. Does clip benefit visual question answering in the medical domain as much as it does in the general domain? arXiv preprint arXiv:2112.13906,

    arXiv

  3. [6]

    Segment anything in medical images

    Jun Ma, Yuting He, Feifei Li, Lin Han, Chenyu You, and Bo Wang. Segment anything in medical images. Nature Communications, 15(1):654, 2024a. Jun Ma, Feifei Li, Sumin Kim, Reza Asakereh, Bao-Hiep Le, Dang-Khoa Nguyen-Vu, Alexander Pfefferle, Muxin Wei, Ruochen Gao, Donghang Lyu, et al. Efficient medsams: Segment anything in medical images on laptop. arXiv ...

    arXiv

  4. [7]

    Radiology objects in context (roco): a multimodal image dataset

    Obioma Pelka, Sven Koitka, Johannes R ¨uckert, Felix Nensa, and Christoph M Friedrich. Radiology objects in context (roco): a multimodal image dataset. In Intravascular Imaging and Computer Assisted Stenting and Large-Scale Annotation of Biomedical Data and Expert Label Synthe- sis: 7th Joint International Workshop, CVII-STENT 2018 and Third International...

    work page 2018

  5. [10]

    Repvit-sam: Towards real-time segmenting any- thing

    Ao Wang, Hui Chen, Zijia Lin, Jungong Han, and Guiguang Ding. Repvit-sam: Towards real-time segmenting any- thing. arXiv preprint arXiv:2312.05760,

    arXiv

  6. [11]

    Repvit: Revisiting mobile cnn from vit perspective

    Ao Wang, Hui Chen, Zijia Lin, Jungong Han, and Guiguang Ding. Repvit: Revisiting mobile cnn from vit perspective. 150 MCP-MedSAM In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 15909–15920, 2024a. Ziyang Wang, Jian-Qing Zheng, Yichi Zhang, Ge Cui, and Lei Li. Mamba-UNet: UNet-like pure visual mamba for medical ...

    arXiv

  7. [12]

    Faster segment anything: Towards lightweight sam for mobile applications

    Chaoning Zhang, Dongshen Han, Yu Qiao, Jung Uk Kim, Sung-Ho Bae, Seungkyu Lee, and Choong Seon Hong. Faster segment anything: Towards lightweight sam for mobile applications. arXiv preprint arXiv:2306.14289 , 2023a. Jianpeng Zhang, Yutong Xie, Yong Xia, and Chunhua Shen. Dodnet: Learning to segment multi-organ and tumors from multiple partially labeled da...

    arXiv

  8. [13]

    One model to rule them all: Towards universal segmentation for medical images with text prompts

    Ziheng Zhao, Yao Zhang, Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. One model to rule them all: Towards universal segmentation for medical images with text prompts. arXiv preprint arXiv:2312.17183,

    arXiv

Show all 14 references
  1. [2015]

    Vm-UNet: Vision mamba UNet for medical image segmentation

    Jiacheng Ruan, Jincheng Li, and Suncheng Xiang. Vm-UNet: Vision mamba UNet for medical image segmentation. arXiv preprint arXiv:2402.02491,

    arXiv

  2. [2021]

    TransUNet: Rethinking the U-Net archi- tecture design for medical image segmentation through the lens of transformers

    Jieneng Chen, Jieru Mei, Xianhang Li, Yongyi Lu, Qihang Yu, Qingyue Wei, Xiangde Luo, Yutong Xie, Ehsan Adeli, Yan Wang, et al. TransUNet: Rethinking the U-Net archi- tecture design for medical image segmentation through the lens of transformers. Medical Image Analysis , 97: 1...

    arXiv

  3. [2022]

    TransUNet: Transformers make strong encoders for medi- cal image segmentation

    Jieneng Chen, Yongyi Lu, Qihang Yu, Xiangde Luo, Ehsan Adeli, Yan Wang, Le Lu, Alan L Yuille, and Yuyin Zhou. TransUNet: Transformers make strong encoders for medi- cal image segmentation. arXiv preprint arXiv:2102.04306,

    arXiv

  4. [2023]

    Medfi- cientsam: a robust medical segmentation model with 149 Lyu, Gao and Staring, 2025 optimized inference pipeline for limited clinical settings

    Bao-Hiep Le, Dang-Khoa Nguyen-Vu, Trong-Hieu Nguyen- Mau, Hai-Dang Nguyen, and Minh-Triet Tran. Medfi- cientsam: a robust medical segmentation model with 149 Lyu, Gao and Staring, 2025 optimized inference pipeline for limited clinical settings. In Medical Image Segmentation Ch...

    2025

  5. [2024]

    Lightm-UNet: Mamba as- sists in lightweight UNet for medical image segmentation

    Weibin Liao, Yinghao Zhu, Xinyuan Wang, Chengwei Pan, Yasha Wang, and Liantao Ma. Lightm-UNet: Mamba as- sists in lightweight UNet for medical image segmentation. arXiv preprint arXiv:2403.05246,

    arXiv

  6. [2025]

    U- Net: Convolutional networks for biomedical image seg- mentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- Net: Convolutional networks for biomedical image seg- mentation. In Medical image computing and computer- assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, pro- ceedings,...

    2015

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.