REVIEW 11 cited by
Segment Everything Everywhere All at Once
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
In this work, we present SEEM, a promptable and interactive model for segmenting everything everywhere all at once in an image, as shown in Fig.1. In SEEM, we propose a novel decoding mechanism that enables diverse prompting for all types of segmentation tasks, aiming at a universal segmentation interface that behaves like large language models (LLMs). More specifically, SEEM is designed with four desiderata: i) Versatility. We introduce a new visual prompt to unify different spatial queries including points, boxes, scribbles and masks, which can further generalize to a different referring image; ii) Compositionality. We learn a joint visual-semantic space between text and visual prompts, which facilitates the dynamic composition of two prompt types required for various segmentation tasks; iii) Interactivity. We further incorporate learnable memory prompts into the decoder to retain segmentation history through mask-guided cross-attention from decoder to image features; and iv) Semantic-awareness. We use a text encoder to encode text queries and mask labels into the same semantic space for open-vocabulary segmentation. We conduct a comprehensive empirical study to validate the effectiveness of SEEM across diverse segmentation tasks. Notably, our single SEEM model achieves competitive performance across interactive segmentation, generic segmentation, referring segmentation, and video object segmentation on 9 datasets with minimum 1/100 supervision. Furthermore, SEEM showcases a remarkable capacity for generalization to novel prompts or their combinations, rendering it a readily universal image segmentation interface.
Forward citations
Cited by 11 Pith papers
-
Discovering and using Spelke segments
SpelkeNet, a self-supervised video world model, discovers Spelke segments in static images by aggregating motion correlations across imagined pokes.
-
Densely Connected Parameter-Efficient Tuning for Referring Image Segmentation
DETRIS uses dense mixtures of convolutions and cross-attention adapters to tune a frozen DINOv2/CLIP pair, achieving top reported IoU on three referring image segmentation benchmarks while updating only a small fracti...
-
On Moving Object Segmentation from Monocular Video with Transformers
M3Former fuses appearance and motion streams in a Mask2Former-style transformer and, when trained on a diverse mix that includes KITTI and DAVIS train splits, reaches state-of-the-art motion segmentation scores on tho...
-
Segment Any Class (SAC): Multi-Class Few-Shot Semantic Segmentation via Class Region Proposals
A training-free prompting method that adapts SAM to multi-class few-shot segmentation and reports higher mIoU than trained baselines on COCO-20i as the number of classes grows.
-
From Vision To Language through Graph of Events in Space and Time: An Explainable Self-supervised Approach
The paper presents GEST, an event-graph representation of videos that is converted automatically into natural language and is also used as a teacher to pre-train end-to-end video captioning models.
-
INT: Instance-Specific Negative Mining for Task-Generic Promptable Segmentation
INT improves task-generic promptable segmentation by progressively mining negative candidates, using VLM output differences after masking to select and refine instance-specific prompts.
-
Efficient MedSAMs: Segment Anything in Medical Images on Laptop
A medical imaging competition produced lightweight segmentation models that match a large foundation model's accuracy on a new hidden test set while running on laptop CPUs with substantially less compute.
-
SPT: Sequence Prompt Transformer for Interactive Image Segmentation
A sequence-aware transformer for interactive image segmentation that uses previous images and clicks as prompts, claiming state-of-the-art results without reporting them.
-
DrivingGaussian++: Towards Realistic Reconstruction and Editable Simulation for Surrounding Dynamic Driving Scenes
DrivingGaussian++ reconstructs dynamic surround-view driving scenes and performs training-free multi-task editing (weather, texture, object manipulation) using Gaussians, diffusion models, and LLM-generated trajectories.
-
Deformable Attentive Visual Enhancement for Referring Segmentation Using Vision-Language Model
SegVLM reports 53.87 IoU on PhraseCut referring segmentation by adding SE blocks, deformable convolutions, residual shortcuts, and a fused BCE-Focal-Dice loss to CRIS.
-
Prompt Engineering in Segment Anything Model: Methodologies, Applications, and Emerging Challenges
A structured survey of prompt engineering methods for the Segment Anything Model, covering geometric, textual, and multimodal prompts and their applications.
Discussion (0). Continue with ORCID to comment.