Pith. sign in

REVIEW 5 cited by

Learning to Prompt Segment Anything Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2401.04651 v1 pith:BLDETY3L submitted 2024-01-09 cs.CV

Learning to Prompt Segment Anything Models

Jiaxing Huang , Kai Jiang , Jingyi Zhang , Han Qiu , Lewei Lu , Shijian Lu , Eric Xing This is my paper
classification cs.CV
keywords promptssamspromptlearninganythingsegmentsemanticspatial
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

Segment Anything Models (SAMs) like SEEM and SAM have demonstrated great potential in learning to segment anything. The core design of SAMs lies with Promptable Segmentation, which takes a handcrafted prompt as input and returns the expected segmentation mask. SAMs work with two types of prompts including spatial prompts (e.g., points) and semantic prompts (e.g., texts), which work together to prompt SAMs to segment anything on downstream datasets. Despite the important role of prompts, how to acquire suitable prompts for SAMs is largely under-explored. In this work, we examine the architecture of SAMs and identify two challenges for learning effective prompts for SAMs. To this end, we propose spatial-semantic prompt learning (SSPrompt) that learns effective semantic and spatial prompts for better SAMs. Specifically, SSPrompt introduces spatial prompt learning and semantic prompt learning, which optimize spatial prompts and semantic prompts directly over the embedding space and selectively leverage the knowledge encoded in pre-trained prompt encoders. Extensive experiments show that SSPrompt achieves superior image segmentation performance consistently across multiple widely adopted datasets.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Few-Shot Semantic Segmentation Meets SAM3

    cs.CV 2026-04 unverdicted novelty 7.0

    Spatial concatenation of support and query images lets a frozen SAM3 achieve state-of-the-art few-shot semantic segmentation on PASCAL-5^i and COCO-20^i.

  2. Learning from Noisy Prompts: Saliency-Guided Prompt Distillation for Robust Segmentation with SAM

    cs.CV 2026-04 unverdicted novelty 6.0

    SPD improves SAM segmentation robustness to noisy prompts by learning anatomical saliency priors, distilling consensus prompts from adjacent slices, and enforcing pairwise slice consistency.

  3. Enhancing MedSAM with a Lightweight Box Predictor for Medical Image Segmentation

    cs.CV 2026-06 conditional novelty 5.0

    A lightweight MLP reads a 5x5 patch of MedSAM's image embedding around one click, predicts a bounding box, and uses it as a spatial prompt, improving Dice by about 0.5-1.5 points on CT/MRI/ultrasound benchmarks.

  4. Enhancing MedSAM with a Lightweight Box Predictor for Medical Image Segmentation

    cs.CV 2026-06 unverdicted novelty 4.0

    Enhances MedSAM with a 1.6M-parameter Box Predictor trained in two stages to convert single clicks to bounding boxes, reporting Dice scores of 0.89-0.98 on four medical datasets across CT, MRI, and ultrasound.

  5. SAM-MI: A Mask-Injected Framework for Enhancing Open-Vocabulary Semantic Segmentation with SAM

    cs.CV 2025-11 conditional novelty 4.0

    SAM-MI improves open-vocabulary segmentation by injecting aggregated SAM masks as low- and high-frequency guidance into CLIP cost maps, with sparse text-guided point prompts for speed.