AutoSAM: Adapting SAM to Medical Images by Overloading the Prompt Encoder
read the original abstract
The recently introduced Segment Anything Model (SAM) combines a clever architecture and large quantities of training data to obtain remarkable image segmentation capabilities. However, it fails to reproduce such results for Out-Of-Distribution (OOD) domains such as medical images. Moreover, while SAM is conditioned on either a mask or a set of points, it may be desirable to have a fully automatic solution. In this work, we replace SAM's conditioning with an encoder that operates on the same input image. By adding this encoder and without further fine-tuning SAM, we obtain state-of-the-art results on multiple medical images and video benchmarks. This new encoder is trained via gradients provided by a frozen SAM. For inspecting the knowledge within it, and providing a lightweight segmentation solution, we also learn to decode it into a mask by a shallow deconvolution network.
This paper has not been read by Pith yet.
Forward citations
Cited by 7 Pith papers
-
SemiSAM-O1: How far can we push the boundary of annotation-efficient medical image segmentation?
SemiSAM-O1 narrows the gap to fully supervised medical image segmentation performance while using only a single annotated template image through foundation-model feature propagation and uncertainty-guided iterative re...
-
APRIL-MedSeg: A Modular Medical Image Segmentation Toolbox Embracing Modern Paradigms
APRIL-MedSeg is a new open-source modular toolbox that uses YAML configuration and component registries to unify multiple advanced paradigms for medical image segmentation.
-
Efficient Remote Sensing Instance Segmentation with Linear-Time State Space Distilled Visual Foundation Models
RS4D distills ViT knowledge into SSM backbones for remote sensing instance segmentation, delivering 8x fewer parameters and 9x fewer FLOPs than ViT methods while matching or exceeding accuracy on SSDD, WHU, and NWPU datasets.
-
SARIF: Segment Anything for Robust Image Forensics
SARIF combines SAM with a feedback-guided decoder and block-wise prompting on residual features to improve cross-dataset forgery localization and robustness to image corruptions.
-
RobustMedSAM: Degradation-Resilient Medical Image Segmentation via Robust Foundation Model Adaptation
RobustMedSAM fuses MedSAM's image encoder with RobustSAM's mask decoder and fine-tunes only the decoder on 35 medical datasets with corruptions to raise degraded-image Dice from 0.613 to 0.719.
-
On Efficient Variants of Segment Anything Model: A Survey
A survey that reviews efficient variants of the Segment Anything Model, categorizes acceleration strategies, and provides a unified hardware evaluation on benchmarks.
-
APRIL-MedSeg: A Modular Medical Image Segmentation Toolbox Embracing Modern Paradigms
Presents APRIL-MedSeg, a modular YAML-configurable toolbox for 2D medical image segmentation integrating semi-supervised, domain adaptation, distillation, weakly supervised, text-guided, and foundation model paradigms...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.