Medical SAM 2: Segment medical images as video via Segment Anything Model 2
read the original abstract
Medical image segmentation plays a pivotal role in clinical diagnostics and treatment planning, yet existing models often face challenges in generalization and in handling both 2D and 3D data uniformly. In this paper, we introduce Medical SAM 2 (MedSAM-2), a generalized auto-tracking model for universal 2D and 3D medical image segmentation. The core concept is to leverage the Segment Anything Model 2 (SAM2) pipeline to treat all 2D and 3D medical segmentation tasks as a video object tracking problem. To put it into practice, we propose a novel \emph{self-sorting memory bank} mechanism that dynamically selects informative embeddings based on confidence and dissimilarity, regardless of temporal order. This mechanism not only significantly improves performance in 3D medical image segmentation but also unlocks a \emph{One-Prompt Segmentation} capability for 2D images, allowing segmentation across multiple images from a single prompt without temporal relationships. We evaluated MedSAM-2 on five 2D tasks and nine 3D tasks, including white blood cells, optic cups, retinal vessels, mandibles, coronary arteries, kidney tumors, liver tumors, breast cancer, nasopharynx cancer, vestibular schwannoma, mediastinal lymph nodules, cerebral artery, inferior alveolar nerve, and abdominal organs, comparing it against state-of-the-art (SOTA) models in task-tailored, general and interactive segmentation settings. Our findings demonstrate that MedSAM-2 surpasses a wide range of existing models and updates new SOTA on several benchmarks. The code is released on the project page: https://supermedintel.github.io/Medical-SAM2/.
This paper has not been read by Pith yet.
Forward citations
Cited by 19 Pith papers
-
Human and AI collaboration for pulmonary nodule segmentation
Hi-Seg achieves a mean Dice score of nearly 85% for pulmonary nodule segmentation by having humans iteratively refine prompts for the Segment Anything Model, outperforming standalone deep learning and SAM models on a ...
-
VERITAS: A Multi-Agent Co-Scientist for Verifiable Image-Derived Hypothesis Testing
VERITAS is a multi-agent system for verifiable hypothesis testing on multimodal clinical MRI datasets that achieves 81.4% verdict accuracy with frontier models and introduces an epistemic evidence labeling framework.
-
Seeing Through the Tool: A Controlled Benchmark for Occlusion Robustness in Foundation Segmentation Models
SAM-family models split into occluder-aware types that avoid predicting into occluded regions and occluder-agnostic types that confidently segment hidden areas, shown via a new benchmark on polyp datasets.
-
Backdoor Attacks on Prompt-Driven Video Segmentation Foundation Models
BadVSFM is the first effective backdoor attack on prompt-driven video segmentation foundation models, using a two-stage encoder-decoder strategy to achieve high attack success rates with limited clean performance loss.
-
Towards Voxel Spacing Consistency for Medical Image Segmentation
Consispace is a semantic-aware resampling method that uses an implicit neural network with ODE constraints and feature reweighting to achieve consistent axial voxel spacing while preserving anatomy and semantics, impr...
-
Tractography-Driven Synthetic Data Generation for Fiber Bundle Segmentation in Tracer Histology
Tractography from ex vivo dMRI is used as a generative prior to create synthetic patches that, when mixed with real data and domain randomization, train a U-Net to segment fiber bundles in tracer histology with 3x les...
-
Speech-Guided Multimodal Learning for Vocal Tract Segmentation in Real-Time MRI
A multimodal training pipeline with phonological bounding-box priors and cross-modal contrastive alignment transfers speech supervision to single-modality rtMRI vocal tract segmentation and outperforms prior methods o...
-
Memory-SAM: Human-Prompt-Free Tongue Segmentation via Retrieval-to-Prompt
Memory-SAM retrieves similar prior cases via DINOv3 features and FAISS to generate point prompts for SAM2, achieving mIoU 0.9863 on 600 tongue images without training or human prompts.
-
Segment Any-Quality Images with Generative Latent Space Enhancement
GleSAM integrates latent diffusion into SAM and SAM2 to boost segmentation robustness on low-quality images using minimal extra parameters and a new LQSeg dataset.
-
Geometry-Consistent Endoscopic Representations for Image-Guided Navigation via Structured Foundation Model Adaptation
A synthetic-data-driven, hierarchy-aware adaptation of foundation models produces geometry-consistent representations that improve pose estimation and monocular depth in endoscopy.
-
CAR-SAM: Cross-Attention Reconstruction for Post-Training Quantization of the Segment Anything Model
CAR-SAM introduces MatMul-Aware Compensation and Joint Cross-Attention Reconstruction to enable stable 4-bit post-training quantization of SAM, outperforming prior PTQ methods by 14.6% mAP on SAM-B and 6.6% on SAM-L.
-
A Real-time Scale-robust Network for Glottis Segmentation in Nasal Transnasal Intubation
A scale-robust lightweight CNN for glottis segmentation achieves 92.9% mDice at over 170 FPS with a 19 MB model size on three datasets.
-
SGP-SAM: Self-Gated Prompting for Transferring 3D Segment Anything Models to Lesion Segmentation
SGP-SAM transfers 3D SAM to lesion segmentation using a self-gated module for conditional multi-scale enhancement and a Zoom Loss, achieving 7.3% mDice gain over fine-tuning on MSD Liver Tumor data.
-
Towards Any-Quality Image Segmentation via Generative and Adaptive Latent Space Enhancement
GleSAM++ improves SAM robustness on degraded images by using generative enhancement, feature alignment, and adaptive degradation prediction while adding few parameters.
-
RASALoRE: Region Aware Spatial Attention with Location-based Random Embeddings for Weakly Supervised Anomaly Detection in Brain MRI Scans
A novel weakly supervised anomaly detection method for brain MRI that uses discriminative dual prompt tuning for pseudo masks and region-aware spatial attention with location-based random embeddings to achieve SOTA re...
-
On Efficient Variants of Segment Anything Model: A Survey
A survey that reviews efficient variants of the Segment Anything Model, categorizes acceleration strategies, and provides a unified hardware evaluation on benchmarks.
-
Parameter-Efficient Adaptation of SAM 3 for Automated ITV Generation from 4DCT Images
LoRA-adapted SAM 3 with hard-negative mining and phase-coherent filtering achieves median Dice 0.968 on pulmonary structures from 4DCT using seven annotated volumes.
-
Enhancing MedSAM with a Lightweight Box Predictor for Medical Image Segmentation
Enhances MedSAM with a 1.6M-parameter Box Predictor trained in two stages to convert single clicks to bounding boxes, reporting Dice scores of 0.89-0.98 on four medical datasets across CT, MRI, and ultrasound.
-
Label-Efficient Cross-Modality Generalization for Liver Segmentation in Multi-Phase MRI
A fine-tuned 3D foundation segmentation model combined with cross pseudo supervision achieves robust liver segmentation across labeled and unlabeled multi-phase, multi-vendor MRI without spatial registration.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.