REVIEW 22 cited by
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Personalize Segment Anything Model with One Shot
read the original abstract
Driven by large-data pre-training, Segment Anything Model (SAM) has been demonstrated as a powerful and promptable framework, revolutionizing the segmentation models. Despite the generality, customizing SAM for specific visual concepts without man-powered prompting is under explored, e.g., automatically segmenting your pet dog in different images. In this paper, we propose a training-free Personalization approach for SAM, termed as PerSAM. Given only a single image with a reference mask, PerSAM first localizes the target concept by a location prior, and segments it within other images or videos via three techniques: target-guided attention, target-semantic prompting, and cascaded post-refinement. In this way, we effectively adapt SAM for private use without any training. To further alleviate the mask ambiguity, we present an efficient one-shot fine-tuning variant, PerSAM-F. Freezing the entire SAM, we introduce two learnable weights for multi-scale masks, only training 2 parameters within 10 seconds for improved performance. To demonstrate our efficacy, we construct a new segmentation dataset, PerSeg, for personalized evaluation, and test our methods on video object segmentation with competitive performance. Besides, our approach can also enhance DreamBooth to personalize Stable Diffusion for text-to-image generation, which discards the background disturbance for better target appearance learning. Code is released at https://github.com/ZrrSkywalker/Personalize-SAM
Forward citations
Cited by 22 Pith papers
-
Reason Twice: Segmentation via Candidate Discovery and Comparative Reasoning
Rea2Seg turns image segmentation into candidate mask discovery from MLLM attention followed by MLLM-based comparative scoring and selection, plus a new multi-dimensional reasoning benchmark ReasonSeg-SGDR.
-
FindIt: A Format-Informed Visual Detection Benchmark for Generalist Multimodal LLMs
FindIt is the first comprehensive benchmark for evaluating generalist MLLMs on promptable object detection, referring expression detection, instance-level detection, and video detection with standardized parsable outputs.
-
Functionalization via Structure Completion and Motion Rectification
Object functionalization is cast as neural graph completion over a functional graph of parts, contacts, and motions, followed by geometry realization that also rectifies erroneous motions, demonstrated on furniture wi...
-
Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V
Set-of-Mark prompting marks segmented image regions with alphanumerics and masks to let GPT-4V achieve state-of-the-art zero-shot results on referring expression comprehension and segmentation benchmarks like RefCOCOg.
-
Repurposing CLIP to Localize at Pixel Level
CLIPix extracts class-specific activation maps from CLIP's classification backpropagation, denoises them via a correction strategy, and embeds them into image features for zero-shot binary semantic segmentation, achie...
-
ExACT: Exemplar-Driven Calibrated Refinement for Training-Free Visual Grounding in Remote Sensing Images
ExACT combines a Vision Exemplar-based Calibrator and Structure-Aware Refiner to improve training-free visual grounding of language descriptions in remote sensing images using frozen MLLMs and SAM.
-
Mask to Concept: Auto-Promptable SAM3 via Efficient Test-Time Concept Embedding Search for Few-Shot Annotation
M2C turns SAM3 into an auto-promptable annotator for medical few-shot segmentation via test-time concept embedding optimization and uncertainty-driven active refinement.
-
Training-free Cross-domain Few-shot Segmentation via Robust Semantic Representation and Matching
A training-free CD-FSS framework built on DINOv3 with SAFR, ASE, and HPM modules reports state-of-the-art results on four target-domain datasets without any parameter updates.
-
Hierarchical Spatial and Channel Aggregation for Cross-domain Few-shot Segmentation
DHANet uses multi-scale spatial and channel aggregation with a probabilistic bank to mitigate over-alignment and reports state-of-the-art results on four target datasets for cross-domain few-shot segmentation.
-
Zero-Shot Learning in Industrial Scenarios: New Large-Scale Benchmark, Challenges and Baseline
Presents MMIO benchmark and RTVP method achieving state-of-the-art 42.2% AP in zero-shot industrial defect detection.
-
Unification of Closed-Open Industrial Detection Scenarios: New Large-Scale Benchmarks,Challenges and Baselines
Presents MMIOC-1M benchmark with 1M+ samples across 14 super-categories and RTVPNet with domain projection, sparse sampling, and bidirectional interaction, claiming SOTA on MMIOC-1M, LVIS, and COCO.
-
DeCoDrift: Stabilizing Decoder Coupling in Closed-Loop Foundation Segmentation
DeCoDrift stabilizes decoder coupling in closed-loop foundation segmentation by constraining prompt updates without retraining or ground truth.
-
Lighting-aware Unified Model for Instance Segmentation
Introduces LCA dual-branch adapter and pairwise loss for lighting-robust SAM instance segmentation, validated on existing benchmarks plus a new Unity synthetic dataset.
-
Lighting-aware Unified Model for Instance Segmentation
A dual-branch adapter module called LCA with contrast maps and pairwise training on a Unity synthetic dataset improves SAM's instance segmentation performance across lighting variations.
-
SegRAG: Training-Free Retrieval-Augmented Semantic Segmentation
SegRAG augments SAM3 with class-specific point prompts retrieved via DINOv3 features and filtered by ICCD, using TSG at inference to improve open-vocabulary segmentation.
-
SegRAG: Training-Free Retrieval-Augmented Semantic Segmentation
SegRAG is a training-free retrieval-augmented framework that extracts class-specific point prompts from a filtered DINOv3 feature bank to boost SAM3 semantic segmentation performance on standard and agricultural benchmarks.
-
PanoSAM2: Lightweight Distortion- and Memory-aware Adaptions of SAM2 for 360 Video Object Segmentation
PanoSAM2 adapts SAM2 with a Pano-Aware Decoder, Distortion-Guided Mask Loss, and Long-Short Memory Module to improve 360 video object segmentation, reporting +5.6 and +6.7 gains over base SAM2 on two benchmarks.
-
RPG-SAM: Reliability-Weighted Prototypes and Geometric Adaptive Threshold Selection for Training-Free One-Shot Polyp Segmentation
RPG-SAM improves one-shot polyp segmentation by weighting high-fidelity support features and dynamically adjusting thresholds via morphological consensus, yielding 5.56% mIoU gain on Kvasir.
-
AHCQ-SAM: Toward Accurate and Hardware-Compatible Post-Training Segment Anything Model Quantization
AHCQ-SAM introduces ACNR, HLUQ, CAG, and LNQ quantization techniques that deliver 15.2% mAP gain on 4-bit SAM-B and 14.01% J&F gain on 4-bit SAM2-Tiny versus prior PTQ methods.
-
Mask to Concept: Auto-Promptable SAM3 via Efficient Test-Time Concept Embedding Search for Few-Shot Annotation
M2C performs test-time concept embedding search in frozen SAM3 plus hybrid uncertainty estimation to enable few-shot medical segmentation with active human correction.
-
Learning to Focus and Precise Cropping: A Reinforcement Learning Framework with Information Gaps and Grounding Loss for MLLMs
A two-stage RL method with information gaps and grounding loss trains MLLMs to focus on and precisely crop relevant image regions, yielding SOTA results on high-resolution VQA benchmarks.
-
OmniCD: A Foundational Framework for Remote Sensing Image Change Detection Guided by Multimodal Semantics
OmniCD proposes a multimodal semantic-guided framework for remote sensing change detection supporting binary to zero-shot tasks, plus the RSITCD dataset, with claimed SOTA performance.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.