MedSAM2: Segment Anything in 3D Medical Images and Videos
read the original abstract
Medical image and video segmentation is a critical task for precision medicine, which has witnessed considerable progress in developing task or modality-specific and generalist models for 2D images. However, there have been limited studies on building general-purpose models for 3D images and videos with comprehensive user studies. Here, we present MedSAM2, a promptable segmentation foundation model for 3D image and video segmentation. The model is developed by fine-tuning the Segment Anything Model 2 on a large medical dataset with over 455,000 3D image-mask pairs and 76,000 frames, outperforming previous models across a wide range of organs, lesions, and imaging modalities. Furthermore, we implement a human-in-the-loop pipeline to facilitate the creation of large-scale datasets resulting in, to the best of our knowledge, the most extensive user study to date, involving the annotation of 5,000 CT lesions, 3,984 liver MRI lesions, and 251,550 echocardiogram video frames, demonstrating that MedSAM2 can reduce manual costs by more than 85%. MedSAM2 is also integrated into widely used platforms with user-friendly interfaces for local and cloud deployment, making it a practical tool for supporting efficient, scalable, and high-quality segmentation in both research and healthcare environments.
This paper has not been read by Pith yet.
Forward citations
Cited by 29 Pith papers
-
MRI2Rep: Autoregressive Structured Report Generation for 3D Liver MRI
MRI2Rep generates LI-RADS structured reports from 3D liver MRI via autoregressive modeling on 3929 real-world pairs, reporting 76% case-level sensitivity and 70-75% clinical acceptability in reader study.
-
BenSyc: Benchmarking Conversational Sycophancy and Human Alignment in LLMs for Bengali Contexts
BenSyc is the first benchmark for conversational sycophancy in Bengali, with top LLMs achieving only 61.8 Macro-F1 on binary detection and 61.7 on five-class classification while often generating overly validating responses.
-
EchoPilot: Training-Free Ultrasound Video Segmentation via Scale-Space Semantic Prompting and Reliability-Gated Memory
EchoPilot delivers state-of-the-art training-free ultrasound video segmentation from a single point prompt by introducing scale-space semantic prompting via S.E.E.D. and reliability-gated memory updates.
-
MedCore: Boundary-Preserving Medical Core Pruning for MedSAM
MedCore achieves 60% parameter and 58.4% FLOP reduction on MedSAM with Dice 0.9549 and preserved boundary metrics via dual-intervention pruning and a new boundary leverage principle.
-
Beyond a Single Frame: Multi-Frame Spatially Grounded Reasoning Across Volumetric MRI
A new multi-frame VQA benchmark on volumetric MRI demonstrates that bounding-box supervised fine-tuning improves spatial grounding in VLMs over zero-shot baselines.
-
VERITAS: A Multi-Agent Co-Scientist for Verifiable Image-Derived Hypothesis Testing
VERITAS is a multi-agent system for verifiable hypothesis testing on multimodal clinical MRI datasets that achieves 81.4% verdict accuracy with frontier models and introduces an epistemic evidence labeling framework.
-
IBISAgent: Reinforcing Pixel-Level Visual Reasoning in MLLMs for Universal Biomedical Object Referring and Segmentation
IBISAgent enables MLLMs to perform iterative pixel-level visual reasoning for biomedical object referring and segmentation via text-based clicks and agentic RL, outperforming prior SOTA methods without model modifications.
-
SAM 3: Segment Anything with Concepts
SAM 3 introduces promptable concept segmentation that doubles accuracy of prior systems on images and videos while improving standard SAM segmentation performance.
-
Closed-loop coupling of personalised and foundation models for real-time treatment guidance with MRI
Closed-loop coupling of personalised temporal prediction and foundation model segmentation improves anatomical prediction and reduces dosimetric error for 400 ms horizons in MRI-guided radiotherapy.
-
Towards Voxel Spacing Consistency for Medical Image Segmentation
Consispace is a semantic-aware resampling method that uses an implicit neural network with ODE constraints and feature reweighting to achieve consistent axial voxel spacing while preserving anatomy and semantics, impr...
-
MedSIGHT: Towards Grounded Visual Comprehension in Medical Large Vision-Language Models
MedSIGHT unifies medical image comprehension and segmentation in Med-LVLMs via a Region Perceiver module and region codebook, trained progressively on 72K pairs to reach SOTA on both tasks across modalities.
-
MS-DKC: A Dataset Knowledge Card Framework for Designing and Adapting Medical Image Segmentation Models
MS-DKC is a dataset knowledge card framework that maps image, morphology, supervision, context, and risk descriptors to design priors and failure modes, shown to produce dataset-specific model adaptations with improve...
-
MedVol-R1: Reward-Driven Evidence Grounding for Volumetric Reasoning Segmentation
MedVol-R1 is an RL framework that decouples 2D evidence grounding from 3D mask generation for volumetric reasoning segmentation and reports SOTA results on M3D-Seg benchmarks.
-
Rethinking Visual Attribution for Chest X-ray Reasoning in Large Vision Language Models
Existing visual attribution methods often fail to identify the visual evidence used by LVLMs in chest X-ray reasoning, while MedFocus using unbalanced optimal transport and targeted interventions substantially outperf...
-
VoxCor: Training-Free Volumetric Features for Multimodal Voxel Correspondence
VoxCor creates reusable volumetric features from frozen 2D ViT models by combining triplanar inference with a closed-form weighted partial least squares projection, enabling direct voxel correspondence across modaliti...
-
ScribbleDose: Scribble-Guided Dose Prediction in Radiotherapy
ScribbleDose achieves competitive radiotherapy dose prediction from sparse scribble annotations by completing them into dense anatomical masks with supervoxel regularization and conditioning dose generation on the res...
-
ScribbleDose: Scribble-Guided Dose Prediction in Radiotherapy
Scribble-guided dose prediction uses a completion module with supervoxel regularization to generate dense masks from sparse labels and conditions a dose network on them, achieving strong performance on the GDP-HMM dat...
-
ESICA: A Scalable Framework for Text-Guided 3D Medical Image Segmentation
ESICA delivers state-of-the-art accuracy on a five-modality 3D medical segmentation benchmark while offering a compact variant with far fewer parameters.
-
Learning from Noisy Prompts: Saliency-Guided Prompt Distillation for Robust Segmentation with SAM
SPD improves SAM segmentation robustness to noisy prompts by learning anatomical saliency priors, distilling consensus prompts from adjacent slices, and enforcing pairwise slice consistency.
-
CrossPan: A Comprehensive Benchmark for Cross-Sequence Pancreas MRI Segmentation and Generalization
CrossPan benchmark shows cross-sequence MRI domain shifts cause pancreas segmentation models to fail catastrophically, establishing sequence generalization as the primary barrier to clinical deployment over center var...
-
PGE-SAM: Prompt-Guided Feature Enhancement for Interactive Segmentation under Degradation
PGE-SAM adds a Prompt Guidance Generator, multi-scale feature interaction, and foreground reconstruction loss to SAM for better interactive segmentation on degraded images, plus a new DM-Seg benchmark.
-
SWARD: Stochastic Window-Attention-Based Relational Distillation for Cross-Architectural Semantic Segmentation
SWARD introduces stochastic windowed attention distillation and prototype discriminative regularization to improve cross-architecture knowledge transfer from transformers to CNNs for semantic segmentation.
-
Align then Refine: Text-Guided 3D Prostate Lesion Segmentation
A text-guided multi-encoder U-Net with alignment loss, heatmap calibration, and confidence-gated cross-attention refiner sets new state-of-the-art 3D prostate lesion segmentation performance on the PI-CAI dataset.
-
SGP-SAM: Self-Gated Prompting for Transferring 3D Segment Anything Models to Lesion Segmentation
SGP-SAM transfers 3D SAM to lesion segmentation using a self-gated module for conditional multi-scale enhancement and a Zoom Loss, achieving 7.3% mDice gain over fine-tuning on MSD Liver Tumor data.
-
BAAI Cardiac Agent: An intelligent multimodal agent for automated reasoning and diagnosis of cardiovascular diseases from cardiac magnetic resonance imaging
BAAI Cardiac Agent automates end-to-end cardiac MRI analysis for seven cardiovascular diseases, achieving AUC >0.93 internally and >0.81 externally with high correlation to expert measurements.
-
UniReason-Med: A Shared Grounded Reasoning Interface for 2D-to-3D Transfer in Medical VQA
UniReason-Med introduces a unified framework for 2D and 3D medical VQA with shared grounded reasoning, trained on a 220K dataset, claiming that joint 2D+3D supervision improves 3D performance over 3D-only training.
-
AGA3DNet: Anatomy-Guided Gaussian Priors with Multi-view xLSTM for 3D Brain MRI Subtype Classification
AGA3DNet improves 3D brain MRI subtype classification by feeding anatomy-guided Gaussian priors derived from radiology reports into a 3D CNN and multi-view xLSTM architecture.
-
SegTTA: Training-Free Test-Time Augmentation for Zero-Shot Medical Imaging Segmentation
SegTTA improves MedSAM2 zero-shot segmentation on uterus and liver datasets by test-time augmentations plus weighted voting, delivering +1.6 mIoU and -2.0 HD95 on multiclass hepatic vessels.
-
Delving Aleatoric Uncertainty in Medical Image Segmentation via Vision Foundation Models
Vision foundation models quantify aleatoric uncertainty via feature diversity and singular value energy to enable uncertainty-aware data filtering and dynamic training optimization for improved medical image segmentation.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.