Pith. sign in

REVIEW 10 cited by

Open-Vocabulary Universal Image Segmentation with MaskCLIP

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2208.08984 v2 pith:ZT4G4LAH submitted 2022-08-18 cs.CV

Zheng Ding , Jieke Wang , Zhuowen Tu This is my paper · ORCID
classification cs.CV
keywords maskclipsegmentationinstancesemanticclippre-trainedcategoriesencoder
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

In this paper, we tackle an emerging computer vision task, open-vocabulary universal image segmentation, that aims to perform semantic/instance/panoptic segmentation (background semantic labeling + foreground instance segmentation) for arbitrary categories of text-based descriptions in inference time. We first build a baseline method by directly adopting pre-trained CLIP models without finetuning or distillation. We then develop MaskCLIP, a Transformer-based approach with a MaskCLIP Visual Encoder, which is an encoder-only module that seamlessly integrates mask tokens with a pre-trained ViT CLIP model for semantic/instance segmentation and class prediction. MaskCLIP learns to efficiently and effectively utilize pre-trained partial/dense CLIP features within the MaskCLIP Visual Encoder that avoids the time-consuming student-teacher training process. MaskCLIP outperforms previous methods for semantic/instance/panoptic segmentation on ADE20K and PASCAL datasets. We show qualitative illustrations for MaskCLIP with online custom categories. Project website: https://maskclip.github.io.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. OpenSeg-R: Improving Open-Vocabulary Segmentation via Step-by-Step Visual Reasoning

    cs.CV 2025-05 conditional novelty 6.0 of 10

    OpenSeg-R uses an LMM's step-by-step visual explanations as extra text prompts to improve open-vocabulary segmentation masks.

  2. DPSeg: Dual-Prompt Cost Volume Learning for Open-Vocabulary Semantic Segmentation

    cs.CV 2025-05 conditional novelty 6.0 of 10

    DPSeg adds Stable Diffusion generated visual prompts to CLIP text prompts to build cost volumes for open-vocabulary semantic segmentation, reporting state-of-the-art mIoU on five benchmarks.

  3. DynAlign: Unsupervised Dynamic Taxonomy Alignment for Cross-Domain Segmentation

    cs.CV 2025-01 conditional novelty 6.0 of 10

    DynAlign aligns source and target taxonomies by using GPT-4 to map labels, SAM for mask proposals, and CLIP to reassign fine-grained target labels, improving unsupervised cross-domain segmentation under label-space shift.

  4. Fine-Grained Image-Text Correspondence with Cost Aggregation for Open-Vocabulary Part Segmentation

    cs.CV 2025-01 conditional novelty 6.0 of 10

    PartCATSeg improves open-vocabulary part segmentation by separating object- and part-level cost volumes, adding a compositional loss, and injecting DINO structural guidance, achieving over 10% h-IoU gains on three benchmarks.

  5. LarvSeg: Exploring Image Classification Data For Large Vocabulary Semantic Segmentation via Category-wise Attentive Classifier

    cs.CV 2025-01 conditional novelty 6.0 of 10

    LarvSeg uses image classification data and a category-wise attentive classifier to extend semantic segmentation to 21,000 categories without mask labels for new categories.

  6. Effective SAM Combination for Open-Vocabulary Semantic Segmentation

    cs.CV 2024-11 conditional novelty 6.0 of 10

    ESC-Net combines CLIP with pre-trained SAM decoder blocks fed by pseudo prompts from image-text correlations to reach state-of-the-art open-vocabulary segmentation on three major benchmarks.

  7. CLIPer: Hierarchically Improving Spatial Representation of CLIP for Open-Vocabulary Semantic Segmentation

    cs.CV 2024-11 conditional novelty 6.0 of 10

    CLIPer reaches state-of-the-art training-free open-vocabulary segmentation by combining early-layer CLIP attention averaging with Stable Diffusion attention refinement.

  8. Low-Frequency Stochastic Gravitational-Wave Background in Gaia DR3 catalog

    astro-ph.CO 2026-03 unverdicted novelty 5.0 of 10

    Gaia DR3 quasar proper-motion noise and sky coverage imply a detectable stochastic GW strain floor of order 10^{-11} below ~5.6 nHz, with VSH more robust than Hellings-Downs to uneven sampling.

  9. What You Perceive Is What You Conceive: A Cognition-Inspired Framework for Open Vocabulary Image Segmentation

    cs.CV 2025-05 conditional novelty 5.0 of 10

    A framework that generates image-level object concepts with a vision-language model before region segmentation improves open-vocabulary segmentation on multiple benchmarks.

  10. ObjectRelator: Enabling Cross-View Object Relation Understanding Across Ego-Centric and Exo-Centric Perspectives

    cs.CV 2024-11 conditional novelty 5.0 of 10

    ObjectRelator adds multimodal condition fusion and cross-view embedding alignment to PSALM, improving ego-exo object correspondence IoU by about 4 points on Ego-Exo4D and setting SOTA on HANDAL-X.

Pith tools