REVIEW 5 cited by
How to build the best medical image segmentation algorithm using foundation models: a comprehensive empirical study with Segment Anything Model
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Automated segmentation is a fundamental medical image analysis task, which enjoys significant advances due to the advent of deep learning. While foundation models have been useful in natural language processing and some vision tasks for some time, the foundation model developed with image segmentation in mind - Segment Anything Model (SAM) - has been developed only recently and has shown similar promise. However, there are still no systematic analyses or "best-practice" guidelines for optimal fine-tuning of SAM for medical image segmentation. This work summarizes existing fine-tuning strategies with various backbone architectures, model components, and fine-tuning algorithms across 18 combinations, and evaluates them on 17 datasets covering all common radiology modalities. Our study reveals that (1) fine-tuning SAM leads to slightly better performance than previous segmentation methods, (2) fine-tuning strategies that use parameter-efficient learning in both the encoder and decoder are superior to other strategies, (3) network architecture has a small impact on final performance, (4) further training SAM with self-supervised learning can improve final model performance. We also demonstrate the ineffectiveness of some methods popular in the literature and further expand our experiments into few-shot and prompt-based settings. Lastly, we released our code and MRI-specific fine-tuned weights, which consistently obtained superior performance over the original SAM, at https://github.com/mazurowski-lab/finetune-SAM.
Forward citations
Cited by 5 Pith papers
-
Are Vision Foundation Models Ready for Out-of-the-Box Medical Image Registration?
On a benchmark of four breast MRI registration tasks, the SAM encoder outperforms classical optimization methods for gross breast alignment but not for fine fibroglandular tissue, and medical pre-training did not help.
-
SegmentAnyMuscle: A universal muscle segmentation model across different locations in MRI
A publicly released MRI muscle segmentation model, trained on a new 316-exam single-center dataset, reaches DSC 88.45% on common sequences and 86.21% on challenging cases across 11 body locations.
-
Brain Imaging Foundation Models, Are We There Yet? A Systematic Review of Foundation Models for Brain Imaging and Biomedical Research
A systematic review of brain imaging foundation models covering 86 models and 161 datasets, with a performance tournament, dataset atlas, and duplicated-data warnings.
-
Customize Segment Anything Model for Multi-Modal Semantic Segmentation with Mixture of LoRA Experts
MLE-SAM adapts SAM2 with modality-specific LoRA experts and a routing mechanism, reporting state-of-the-art multi-modal segmentation results on DELIVER, MUSES, and MCubeS, although the comparisons are confounded by di...
-
Optimized Vessel Segmentation: A Structure-Agnostic Approach with Small Vessel Enhancement and Morphological Correction
OVS-Net, a SAM-based vessel segmentation framework with a micro-vessel enhancement branch and morphology-correction post-processing, reports higher Dice and better connectivity than six SAM baselines and 17 expert mod...
Discussion (0). Continue with ORCID to comment.