Pith. sign in

REVIEW 17 cited by

Semantic-SAM: Segment and Recognize Anything at Any Granularity

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2307.04767 v1 pith:DGULUZIM submitted 2023-07-10 cs.CV

Feng Li , Hao Zhang , Peize Sun , Xueyan Zou , Shilong Liu , Jianwei Yang , Chunyuan Li , Lei Zhang
show 1 more author
Jianfeng Gao
This is my paper
classification cs.CV
keywords modelsegmentationmultiplesemantic-awarenessanythingdatasetsgranularitygranularity-abundance
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In this paper, we introduce Semantic-SAM, a universal image segmentation model to enable segment and recognize anything at any desired granularity. Our model offers two key advantages: semantic-awareness and granularity-abundance. To achieve semantic-awareness, we consolidate multiple datasets across three granularities and introduce decoupled classification for objects and parts. This allows our model to capture rich semantic information. For the multi-granularity capability, we propose a multi-choice learning scheme during training, enabling each click to generate masks at multiple levels that correspond to multiple ground-truth masks. Notably, this work represents the first attempt to jointly train a model on SA-1B, generic, and part segmentation datasets. Experimental results and visualizations demonstrate that our model successfully achieves semantic-awareness and granularity-abundance. Furthermore, combining SA-1B training with other segmentation tasks, such as panoptic and part segmentation, leads to performance improvements. We will provide code and a demo for further exploration and evaluation.

Discussion (0). Sign in to comment.

Forward citations

Cited by 17 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. COCOTree: A Dataset and Benchmark for Open Tree-Structured Visual Decomposition

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    COCOTree is a 21K-image benchmark with 1.8M nodes and an OTQ metric for the new task of open tree-structured visual decomposition.

  2. Vision Harnessing Agent for Open Ad-hoc Segmentation

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    VASA is a vision-guided agent for open ad-hoc segmentation that creates and validates masks through planning, tool use, and error recovery, outperforming baselines on the new PARS benchmark and RefCOCOm.

  3. Self-Correcting Text-to-Video Generation with Misalignment Detection and Localized Refinement

    cs.CV 2024-11 unverdicted novelty 7.0 of 10

    VideoRepair detects text-video misalignments via MLLM-generated questions and performs localized, region-preserving refinement to improve alignment in existing T2V diffusion models.

  4. Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V

    cs.CV 2023-10 accept novelty 7.0 of 10

    Set-of-Mark prompting marks segmented image regions with alphanumerics and masks to let GPT-4V achieve state-of-the-art zero-shot results on referring expression comprehension and segmentation benchmarks like RefCOCOg.

  5. G$^2$TAM: Geometry Grounded Track Anything Model

    cs.CV 2026-07 accept novelty 6.5 of 10

    Spatially aligned geometric features serve as implicit memory so one model reconstructs scenes and produces promptable, cross-view consistent instance masks from unordered RGB only.

  6. In-context Region-based Drag: Drag Any Region to Any Shape

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    ICRDrag performs region-based drag editing in diffusion models by feeding source image, source mask, and target mask into an in-context framework with image-mask attention consistency and source-target attention corre...

  7. EPS3D: End-to-End Feed-Forward 3D Panoptic Segmentation

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    EPS3D is an end-to-end architecture for 3D panoptic segmentation from multi-view images that uses distillation and semantic-instance mutual enhancement to achieve higher benchmark performance and speed than prior methods.

  8. Amodal SAM: A Unified Amodal Segmentation Framework with Generalization

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    Amodal SAM extends SAM with a Spatial Completion Adapter, Target-Aware Occlusion Synthesis for data, and consistency losses to reach SOTA amodal segmentation with strong generalization to new objects and scenes.

  9. PASG: A Closed-Loop Framework for Automated Geometric Primitive Extraction and Semantic Anchoring in Robotic Manipulation

    cs.CV 2025-08 unverdicted novelty 6.0 of 10

    PASG automatically extracts object keypoints and axes and couples them through a fine-tuned vision-language model to task semantics, claiming manipulation performance comparable to manual annotations.

  10. Training-free Geometric Image Editing on Diffusion Models

    cs.CV 2025-07 conditional novelty 6.0 of 10

    FreeFine splits geometric image editing into object transformation, source-region inpainting, and target refinement, using temporal attention, local noise, and text guidance in a training-free way.

  11. One Polyp Identifies All: One-Shot Polyp Segmentation with SAM via Cascaded Priors and Iterative Prompt Evolution

    cs.CV 2025-07 conditional novelty 6.0 of 10

    OP-SAM turns one labeled polyp image into iterative SAM prompts, reaching 76.93% IoU on Kvasir with no retraining.

  12. MV3DIS: Multi-View Mask Matching via 3D Guides for Zero-Shot 3D Instance Segmentation

    cs.CV 2026-04 unverdicted novelty 5.0 of 10

    MV3DIS uses 3D-guided mask matching and depth consistency to produce more consistent multi-view 2D masks that refine into accurate zero-shot 3D instances.

  13. Integrating SAM Supervision for 3D Weakly Supervised Point Cloud Segmentation

    cs.CV 2025-08 conditional novelty 5.0 of 10

    A weakly supervised 3D point cloud segmentation method that back-projects Semantic-SAM 2D masks into 3D, propagates sparse labels inside masks, and uses reliability-filtered pseudo labels, reporting state-of-the-art m...

  14. Personalization Toolkit: Training Free Personalization of Large Vision Language Models

    cs.CV 2025-02 unverdicted novelty 5.0 of 10

    Presents a training-free personalization toolkit for LVLMs that extracts features via vision foundation models, applies RAG for instance retrieval, and uses visual prompting for multi-concept adaptation on images and ...

  15. Ultralytics YOLO26: Unified Real-Time End-to-End Vision Models

    cs.CV 2026-06 unverdicted novelty 4.0 of 10

    YOLO26 presents a unified real-time vision model family with dual-head end-to-end design, new training components, and task-specific heads that reports improved mAP-latency tradeoffs on COCO and LVIS benchmarks across...

  16. UnAC: Adaptive Visual Prompting with Abstraction and Stepwise Checking for Complex Multimodal Reasoning

    cs.CV 2026-05 unverdicted novelty 4.0 of 10

    UnAC improves LMM performance on visual reasoning benchmarks by combining adaptive visual prompting, image abstraction, and gradual self-checking.

  17. SAM-MI: A Mask-Injected Framework for Enhancing Open-Vocabulary Semantic Segmentation with SAM

    cs.CV 2025-11 conditional novelty 4.0 of 10

    SAM-MI improves open-vocabulary segmentation by injecting aggregated SAM masks as low- and high-frequency guidance into CLIP cost maps, with sparse text-guided point prompts for speed.

Pith tools