Pith. sign in

REVIEW 4 major objections 4 minor 67 references

Training-Free Class Purification for Open-Vocabulary Semantic Segmentation

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Open-vocabulary segmentation is degraded by classes that are not in the image, and FreeCP shows these can be detected and removed by comparing each class activation map before and after affinity-based refinement.

desk verdict A useful, honest empirical paper with a genuinely new purification mechanism, but the undisclosed per-dataset thresholds put the generalization claim on shaky ground. read the letter →

arxiv 2508.00557 v1 pith:AP4IOYPR submitted 2025-08-01 cs.CV

Qi Chen , Lingxiao Yang , Yun Chen , Nailong Zhao , Jianhuang Lai , Jie Shao , Xiaohua Xie This is my paper
classification cs.CV
keywords open-vocabularysemanticsegmentationtraining-freeclasspurificationactivationmapspatialconsistencyvisual-languageambiguityCLIPaffinityrefinement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Open-vocabulary semantic segmentation is usually run with a fixed, overcomplete vocabulary, so most candidate classes are absent from the image and several present classes are semantically close. The paper claims that these two failure modes—class redundancy and visual-language ambiguity—actively degrade CLIP-based predictions, and proposes FreeCP, a training-free module that removes them. FreeCP computes a class activation map for every vocabulary entry, refines each map with the CLIP image encoder's own self-attention as an affinity matrix, and measures the spatial consistency between original and refined maps. Classes whose maps change too much are treated as redundant and deleted; classes whose refined maps overlap heavily are grouped as ambiguous and resolved locally by comparing crops against fine-grained LLM-generated descriptions. The reported payoff is that attaching FreeCP to MaskCLIP, GEM, ClearCLIP, and SCLIP raises average mIoU by 10.9, 3.7, 4.1, and 3.9 points respectively across eight benchmarks, with no retraining.

What carries the argument

The load-bearing object is the Spatial Consistency score $SC(X,Y)$ defined as the intersection-over-union of two activation maps. The paper uses it in two places: $S_i=SC(M_i,\tilde{M}_i)$ compares a class's original activation map $M_i$ with its affinity-refined version $\tilde{M}_i=M_i\times SA$, where $SA$ is the average of resized self-attention matrices from CLIP layers; a low score removes the class as redundant. Then $P_{i,j}=SC(\tilde{M}_i,\tilde{M}_j)$ compares refined maps across classes, and pairs above a threshold are connected by depth-first search into ambiguity groups. Each group's high-response region is cropped, re-encoded by CLIP, and assigned to the class whose averaged fine-grained LLM description matches best, so purification is carried by attention statistics plus a text-based local competition.

What would settle it

On a held-out validation split, compute the spatial consistency $SC(M_i, \tilde{M}_i)$ for every class in every image and separate the values by whether the class is present in the ground truth. If the two distributions overlap substantially, no choice of $T_{rp}$ can purify safely; the paper's own refinement-only setting (VOC21 mIoU dropping from 59.8 to 27.5) shows the heuristic must be doing all the work, so this distributional check settles it.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the difference between a class activation map and its affinity-refined counterpart is a usable, training-free signal in open-vocabulary settings. Classes genuinely present in the image tend to keep a consistent spatial layout under refinement, while redundant classes pick up scattered spurious activations; the same refined maps reveal visual-language ambiguity when two or more classes share nearly identical activation patterns. FreeCP operationalizes the first signal as an IoU-based spatial consistency score with a per-dataset threshold, and the second as a graph of high-overlap classes resolved by region-level text matching. The paper claims this purification is general across CLIP backbones and can be plugged into existing training-free segmentors without modifying their weights.

Load-bearing premise

The load-bearing premise is that a class whose activation pattern changes a lot after attention-based refinement is probably absent from the image, and the method's main gain collapses if that signal stops separating present from absent classes.

Editorial extensions

If this is right

  • Any training-free CLIP-based segmentor can be upgraded by attaching FreeCP as post-processing, with no retraining and only precomputed text descriptions as extra input.
  • The benefit is concentrated in overcomplete-vocabulary settings: the larger the gap between full vocabulary and the classes actually present, the more redundancy purification has to remove.
  • Ambiguity purification turns a per-pixel segmentation error into a region-level text-matching decision, so the method also suggests a cheap route to conflict resolution in other dense prediction tasks.
  • The consistency signal survives across backbones: the paper reports gains with ViT-B/16, ViT-L/14, OpenCLIP ViT-L/14, and ResNet-based CLIP, the last using DINO features as the affinity source.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An untested extension is to treat $SC(M_i,\tilde{M}_i)$ as a per-class confidence score instead of a hard threshold, which would let downstream modules keep low-confidence classes rather than dropping them outright.
  • The two thresholds are set per dataset from its semantic complexity; a natural follow-up is a rule that predicts them from vocabulary size or scene density so the module stays training-free on new benchmarks without manual tuning.
  • Because the redundancy signal is about activation geometry rather than CLIP's text encoder, the same purification may transfer to non-CLIP dense open-vocabulary models if they produce class heatmaps and a class-agnostic affinity.
  • A direct stress test would feed FreeCP a vocabulary deliberately enriched with near-duplicate classes; if the ambiguity groups grow but the per-group argmax still resolves them, the LLM description stage is carrying the load.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes FreeCP, a training-free module that post-processes class activation maps of open-vocabulary segmentors. FreeCP first refines the CAM using CLIP self-attention, then discards classes whose pre/post-refinement activation maps have low spatial consistency (redundancy purification), then groups classes with mutually high refined-map overlap and reclassifies ambiguous local regions using LLM-generated fine-grained descriptions (ambiguity purification). Experiments on eight benchmark variants show consistent mIoU improvements over MaskCLIP, GEM, ClearCLIP, and SCLIP, and ablations indicate that redundancy purification accounts for most of the gain.

Significance. If the empirical claims are robust, FreeCP would be a useful training-free plug-in for CLIP-based OVSS, because it directly targets a known failure mode of overcomplete vocabularies and requires no gradient updates. The paper's strengths include its clear problem diagnosis, the breadth of the benchmark evaluation, the plug-in experiments across four base segmentors, and the cross-architecture ablations. However, the currently decisive evidence is the ablation table showing recovery from a 27.5 mIoU refinement collapse, and that recovery is controlled by undisclosed per-dataset thresholds; the ambiguity-purification component, which is the paper's more distinctive contribution, shows only marginal gains. These issues need to be addressed before the central 'plug-and-play, training-free' claim is fully supported.

major comments (4)
  1. [§4.2, §3.3.1, §3.3.2] The thresholds T_rp and T_ap are chosen per dataset using 'prior knowledge' about semantic complexity, but their numerical values are never reported and no sensitivity analysis is given. This is load-bearing because Tab. 3 shows that the entire recovery from the 27.5 mIoU of '+Refine' to the 65.8 mIoU of 'RP' on VOC21 is mediated by where T_rp is set. Without reported threshold values and sweeps over both thresholds, the consistent gains in Tab. 2 cannot be separated from per-dataset threshold tuning. Please report the exact thresholds for all benchmarks, provide sensitivity or plateau analyses, and report per-class or per-dataset precision/recall for the redundancy filter to show that small or rare classes are not being sacrificed.
  2. [§3.2, Eq. (5)] The paper's central heuristic is that high spatial consistency between M_i and its affinity-refined version indicates a true positive class, while low consistency indicates redundancy. This is asserted as an observation on the benchmarks used for evaluation and is not supported by a mechanistic derivation, a synthetic experiment, or an out-of-sample test. A precision-recall analysis of the SC signal on present versus absent classes, or an experiment with randomly permuted or held-out vocabularies, would establish that the signal itself, rather than the threshold chosen for each dataset, is responsible for the gains. As written, the claim that the heuristic will transfer to new datasets, vocabularies, or backbones is not yet supported.
  3. [Tab. 3] The contribution of ambiguity purification is marginal in the ablations: adding AP to RP changes mIoU by 0.0, 0.2, 0.0, 0.1, 0.6, and 0.8 across VOC21, PC60, Object, City, ADE, and Stuff. Given that the introduction and abstract identify visual-language ambiguity as a core problem, the paper should report how many ambiguity groups are discovered, how often reclassification changes the label, and the resulting group-level accuracy. Without such evidence, most of the measured improvement is attributable to redundancy purification alone, and the two-stage framing is not fully validated.
  4. [Eq. (4)] The spatial consistency metric is written as SC(X,Y) = P[X·Y] / P[X+Y−X·Y], but the operator P[·] is never defined, and the activation maps are continuous rather than binary. If this is meant to be a sum over pixels, it should be stated explicitly; if it is a soft IoU on continuous maps, that should also be stated. Every downstream threshold and grouping decision depends on this quantity, so the current ambiguity in its definition is a reproducibility issue.
minor comments (4)
  1. [Eq. (1)] The class activation map is normalized with a softmax over all K classes at each pixel, making the values for each class relative to the full vocabulary. The paper should clarify how this relative normalization interacts with the absolute threshold T_rp, since the same visual evidence can produce different activation magnitudes depending on vocabulary size.
  2. [Tab. 1 vs. Tab. 3] The baseline numbers differ between Tab. 1 and Tab. 3 for the same benchmarks (e.g., VOC21 59.4 vs. 59.8, PC60 29.7 vs. 31.6). Please specify the exact configuration for each table so that readers can reconcile the refinement-with/without-GT analysis with the later ablations.
  3. [§3.3.1 and §3.3.2] The symbol P is used both for the patch size in Eq. (1) and for an operator in Eq. (4), which is confusing. Please use distinct notation for the counting/summation operation.
  4. [Related Works and Tab. 2] RIM is cited as [49] in the Related Works but listed as [48] in Tab. 2; please make the citation numbering consistent. Additionally, the asterisk for CaR* could be explained in the table caption to avoid confusion with FreeCP's own post-processing statement.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: FreeCP's pipeline is self-contained and its reported gains are not equal to its inputs by construction.

full rationale

FreeCP's derivation chain contains no step that reduces a predicted quantity to a fitted parameter or to a self-citation. Activation maps M_j are computed from CLIP text-image similarities (Eq. 1), refined maps are produced by multiplying M_i with the CLIP self-attention matrix (Eqs. 2-3), spatial consistency is a standard IoU between two already-defined maps (Eqs. 4-5), redundancy and ambiguity filters are thresholded heuristics on that consistency (Eqs. 5-7), and ambiguity resolution uses independently generated LLM text descriptions compared with CLIP visual features (Eq. 8). None of these equations defines its output in terms of its own conclusion, and no load-bearing claim rests on a citation from the authors' prior work. The method is evaluated against external baselines on eight benchmarks, so the central empirical claim is self-contained. The per-dataset threshold configuration described in Sec. 4.2 is a genuine generalization and sensitivity concern, but it is not circular: the thresholds are hand-set by dataset prior rather than fitted parameters renamed as predictions, and the reported mIoU values are measured outcomes rather than logical consequences of an identity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central mechanism rests on two heuristics introduced by this paper: that refinement-induced change in a class activation map marks the class as absent, and that inter-class refined-map overlap marks ambiguity. Both are empirical assumptions validated only on the same eight benchmark variants used to set the two thresholds. The paper introduces no new theoretical entities, but the contribution effectively buys its gains with two per-dataset free thresholds, an unspecified layer count for the affinity average, and hand-chosen crop and description counts, which a reader should count as part of the method's cost.

free parameters (5)
  • T_rp (redundancy purification threshold) = not reported
    Each class is removed when SC(M_i, M~_i) < T_rp (Eq. 5, Sec. 3.3.1). Sec. 4.2 states thresholds are set per dataset by prior knowledge of semantic complexity; no values or sensitivity analysis are given.
  • T_ap (ambiguity purification threshold) = not reported
    Pairs with SC(M~_i, M~_j) > T_ap are declared ambiguous (Eq. 7, Sec. 3.3.2). Set per dataset by hand; no values or sensitivity analysis.
  • Number of attention layers L in affinity average = unspecified ('multiple layers')
    Eq. 2 averages self-attention matrices over L layers; L is never stated, so the refinement operator is underspecified.
  • Crop resize size for ambiguity regions = 112 x 112 (example)
    Ambiguous regions are cropped and resized to a specified shape before re-encoding (Sec. 3.3.2); the shape is a hand-chosen hyperparameter affecting the AP stage.
  • Number of LLM descriptions per class = 15
    Five answers for each of three instructions are averaged to form the fine-grained text feature (Sec. 4.2); chosen by hand.
assumptions (5)
  • domain assumption CLIP self-attention matrices capture semantic affinity among image patches
    Eqs. 2 and 3 use averaged self-attention as the affinity for refinement, following prior weakly supervised segmentation work [2, 33, 43].
  • ad hoc to paper Consistency between raw and refined activation maps indicates class presence
    Sec. 3.2 asserts 'If a class shows strong consistency... it is more likely to be a true positive'; this is the load-bearing heuristic of the redundancy stage, validated only on the benchmarks where thresholds were tuned.
  • ad hoc to paper High inter-class refined-map overlap indicates visual-language ambiguity
    Sec. 3.3.2 defines ambiguity groups via SC(M~_i, M~_j) > T_ap; the correspondence between overlap and confusion is assumed, not derived.
  • domain assumption LLM-generated fine-grained descriptions are more discriminative than class names for ambiguous crops
    Sec. 3.3.2 and Tab. 4 rely on Vicuna descriptions outperforming templates; the paper shows this empirically but assumes it transfers to new classes and datasets.
  • domain assumption Evaluation with the full dataset vocabulary is the correct OVSS protocol
    All experiments use the full class vocabulary per dataset rather than per-image class sets; Fig. 1 shows this is a deliberate and load-bearing choice, since GT-vocabulary evaluation yields much higher mIoU.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Training-Free Class Purification for Open-Vocabulary Semantic Segmentation." pith.science (2026). https://pith.science/paper/AP4IOYPR

@misc{pith2026250800557,
  author       = {Pith},
  title        = {Pith review of: Training-Free Class Purification for Open-Vocabulary Semantic Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AP4IOYPR}},
  note         = {Machine review of arXiv:2508.00557}
}
read the original abstract

Fine-tuning pre-trained vision-language models has emerged as a powerful approach for enhancing open-vocabulary semantic segmentation (OVSS). However, the substantial computational and resource demands associated with training on large datasets have prompted interest in training-free methods for OVSS. Existing training-free approaches primarily focus on modifying model architectures and generating prototypes to improve segmentation performance. However, they often neglect the challenges posed by class redundancy, where multiple categories are not present in the current test image, and visual-language ambiguity, where semantic similarities among categories create confusion in class activation. These issues can lead to suboptimal class activation maps and affinity-refined activation maps. Motivated by these observations, we propose FreeCP, a novel training-free class purification framework designed to address these challenges. FreeCP focuses on purifying semantic categories and rectifying errors caused by redundancy and ambiguity. The purified class representations are then leveraged to produce final segmentation predictions. We conduct extensive experiments across eight benchmarks to validate FreeCP's effectiveness. Results demonstrate that FreeCP, as a plug-and-play module, significantly boosts segmentation performance when combined with other OVSS methods.

Figures

Figures reproduced from arXiv: 2508.00557 by the authors.

Figure 1
Figure 1. Evidence that overcomplete vocabulary affects OVSS performance. (a) Visualization of two types of problems: Class Redundancy and Visual-Language Ambiguity. (b) Performance comparison between full vocabulary and only GT vocabulary. (OVSS) has been introduced to segment objects using ar￾bitrary classes described by text. Large-scale vision-language models (e.g. CLIP [38] and ALIGN [23]) have demonstrated impressive tr… view at source ↗
Figure 2
Figure 2. Motivation of the proposed FreeCP method. CAM and its refined version facilitate the identification of class redun￾dancy and visual-language ambiguity by analyzing spatial distri￾bution patterns. The reported values represent intersection-over￾union (IoU), which quantify the consistency between activation maps, providing insights into the degree of similarity. . Motivated by these findings, we propose a training-Fre… view at source ↗
Figure 3
Figure 3. Overview of the proposed FreeCP method. Based on a ViT version of CLIP model, the methodology includes three sequential stages: First, image self-affinity and image-text affinity are derived through the image encoder EI and text encoder ET of CLIP. We then leverage image-text affinity to generate class-specific activations M and refined activations M˜ with image self-affinity. Subsequently, we formulate spatial cons… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Overview of Class Purification. For all classes in the vocabulary, we first conduct (a) Redundancy Purification, eliminat￾ing classes whose intra-class spatial consistency falls below the predefined threshold. Subsequently, the remaining classes undergo (b) Ambiguity P…
Figure 5
Figure 5. Figure 5: Visualization of COCO Stuff and ADE20k dataset. Our FreeCP can effectively eliminate redundancy and resolve ambiguity [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

67 extracted references · 64 canonical work pages

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  2. [2]

    Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation

    Jiwoon Ahn and Suha Kwak. Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 4981–4990, 2018. 4

    work page 2018

  3. [3]

    Single-stage semantic segmentation from image labels

    Nikita Araslanov and Stefan Roth. Single-stage semantic segmentation from image labels. In IEEE Conf. Comput. Vis. Pattern Recog., pages 4253–4262, 2020. 6

    work page 2020

  4. [4]

    Training-free open- vocabulary segmentation with offline diffusion-augmented prototype generation

    Luca Barsellotti, Roberto Amoroso, Marcella Cornia, Lorenzo Baraldi, and Rita Cucchiara. Training-free open- vocabulary segmentation with offline diffusion-augmented prototype generation. In IEEE Conf. Comput. Vis. Pattern Recog., 2024. 1, 3, 6, 7

    work page 2024

  5. [5]

    Grounding everything: Emerging localization properties in vision-language transformers

    Walid Bousselham, Felix Petersen, Vittorio Ferrari, and Hilde Kuehne. Grounding everything: Emerging localization properties in vision-language transformers. In IEEE Conf. Comput. Vis. Pattern Recog., 2024. 3, 6, 7

    work page 2024

  6. [6]

    Coco- stuff: Thing and stuff classes in context

    Holger Caesar, Jasper Uijlings, and Vittorio Ferrari. Coco- stuff: Thing and stuff classes in context. InIEEE Conf. Com- put. Vis. Pattern Recog., pages 1209–1218, 2018. 1, 6

    work page 2018

  7. [7]

    Mixreorg: Cross-modal mixed patch reorganization is a good mask learner for open-world semantic segmentation

    Kaixin Cai, Pengzhen Ren, Yi Zhu, Hang Xu, Jianzhuang Liu, Changlin Li, Guangrun Wang, and Xiaodan Liang. Mixreorg: Cross-modal mixed patch reorganization is a good mask learner for open-world semantic segmentation. In Int. Conf. Comput. Vis., pages 1196–1205, 2023. 3

    work page 2023

  8. [8]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 8

    work page 2021

Show all 67 references
  1. [9]

    Learn- ing to generate text-grounded mask for open-world seman- tic segmentation from only image-text pairs

    Junbum Cha, Jonghwan Mun, and Byungseok Roh. Learn- ing to generate text-grounded mask for open-world seman- tic segmentation from only image-text pairs. In IEEE Conf. Comput. Vis. Pattern Recog., pages 11165–11174, 2023. 1, 3

    2023

  2. [10]

    Conceptual 12M: Pushing web-scale image-text pre-training to recognize long-tail visual concepts

    Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. Conceptual 12M: Pushing web-scale image-text pre-training to recognize long-tail visual concepts. In IEEE Conf. Comput. Vis. Pattern Recog., 2021. 1

    2021

  3. [11]

    Exploring open-vocabulary semantic segmentation from clip vision encoder distillation only

    Jun Chen, Deyao Zhu, Guocheng Qian, Bernard Ghanem, Zhicheng Yan, Chenchen Zhu, Fanyi Xiao, Sean Chang Cu- latana, and Mohamed Elhoseiny. Exploring open-vocabulary semantic segmentation from clip vision encoder distillation only. In Int. Conf. Comput. Vis., pages 699–710, 2023. 3

    2023

  4. [12]

    Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolu- tion, and fully connected crfs

    Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolu- tion, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell., 40(4):834–848, 2017. 1

    2017

  5. [13]

    Self-supervised image-specific prototype exploration for weakly supervised semantic segmentation

    Qi Chen, Lingxiao Yang, Jian-Huang Lai, and Xiaohua Xie. Self-supervised image-specific prototype exploration for weakly supervised semantic segmentation. InIEEE Conf. Comput. Vis. Pattern Recog., pages 4288–4298, 2022. 1

    2022

  6. [14]

    Masked-attention mask transformer for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexan- der Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. InIEEE Conf. Comput. Vis. Pattern Recog., pages 1290–1299, 2022. 1, 8

    2022

  7. [15]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhang- hao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yong- hao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, 2023. 6

    2023

  8. [16]

    Cat-seg: Cost aggregation for open- vocabulary semantic segmentation

    Seokju Cho, Heeseong Shin, Sunghwan Hong, Seungjun An, Seungjun Lee, Anurag Arnab, Paul Hongsuck Seo, and Seungryong Kim. Cat-seg: Cost aggregation for open- vocabulary semantic segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., 2024. 3

    2024

  9. [17]

    The cityscapes dataset for semantic urban scene understanding

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In IEEE Conf. Comput. Vis. Pattern Recog., pages 3213–3223, 2016. 6

    2016

  10. [18]

    De- coupling zero-shot semantic segmentation

    Jian Ding, Nan Xue, Gui-Song Xia, and Dengxin Dai. De- coupling zero-shot semantic segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 11583–11592, 2022. 3

    2022

  11. [19]

    Open- vocabulary universal image segmentation with maskclip

    Zheng Ding, Jieke Wang, and Zhuowen Tu. Open- vocabulary universal image segmentation with maskclip. In Int. Conf. Mach. Learn., pages 8090–8102, 2023. 3

    2023

  12. [20]

    The pascal visual object classes challenge: A retrospective

    Mark Everingham, SM Ali Eslami, Luc Van Gool, Christo- pher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes challenge: A retrospective. Int. J. Comput. Vis., 111(1):98–136, 2015. 6

    2015

  13. [21]

    Conservative-progressive collab- orative learning for semi-supervised semantic segmentation

    Siqi Fan, Fenghua Zhu, Zunlei Feng, Yisheng Lv, Mingli Song, and Fei-Yue Wang. Conservative-progressive collab- orative learning for semi-supervised semantic segmentation. IEEE Trans. Image Process., 2023. 1

    2023

  14. [22]

    Scal- ing open-vocabulary image segmentation with image-level labels

    Golnaz Ghiasi, Xiuye Gu, Yin Cui, and Tsung-Yi Lin. Scal- ing open-vocabulary image segmentation with image-level labels. In Eur. Conf. Comput. Vis., pages 540–557. Springer,

  15. [23]

    Scaling up visual and vision-language representa- tion learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representa- tion learning with noisy text supervision. In Int. Conf. Mach. Learn., pages 4904–4916. PMLR, 2021. 1 9

    2021

  16. [24]

    Plug-and-play, dense-label-free extraction of open-vocabulary semantic segmentation from vision- language models

    Luo Jiayun, Siddhesh Khandelwal, Leonid Sigal, and Boyang Li. Plug-and-play, dense-label-free extraction of open-vocabulary semantic segmentation from vision- language models. In IEEE Conf. Comput. Vis. Pattern Recog., 2024. 3

    2024

  17. [25]

    In defense of lazy visual grounding for open-vocabulary semantic segmentation

    Dahyun Kang and Minsu Cho. In defense of lazy visual grounding for open-vocabulary semantic segmentation. In Eur. Conf. Comput. Vis., 2024. 2, 6, 7

    2024

  18. [26]

    Diffusion models for open-vocabulary seg- mentation

    Laurynas Karazijav, Iro Laina, Andrea Vedaldi, and Chris- tian Rupprecht. Diffusion models for open-vocabulary seg- mentation. In Eur. Conf. Comput. Vis., 2024. 1, 3, 6, 7

    2024

  19. [27]

    Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick. Segment anything. In Int. Conf. Comput. Vis., pages 4015–4026, 2023. 3, 8

    2023

  20. [28]

    Clearclip: Decom- posing clip representations for dense vision-language infer- ence

    Mengcheng Lan, Chaofeng Chen, Yiping Ke, Xinjiang Wang, Litong Feng, and Wayne Zhang. Clearclip: Decom- posing clip representations for dense vision-language infer- ence. In Eur. Conf. Comput. Vis., 2024. 1, 3, 6, 7

    2024

  21. [29]

    Proxyclip: Proxy at- tention improves clip for open-vocabulary segmentation

    Mengcheng Lan, Chaofeng Chen, Yiping Ke, Xinjiang Wang, Litong Feng, and Wayne Zhang. Proxyclip: Proxy at- tention improves clip for open-vocabulary segmentation. In Eur. Conf. Comput. Vis., 2024. 1, 3, 6, 7

    2024

  22. [30]

    Language-driven semantic seg- mentation

    Boyi Li, Kilian Q Weinberger, Serge Belongie, Vladlen Koltun, and Rene Ranftl. Language-driven semantic seg- mentation. In Int. Conf. Learn. Represent., 2022. 3

    2022

  23. [31]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In Int. Conf. Mach. Learn., 2022. 3

    2022

  24. [32]

    Clip surgery for better explainability with enhancement in open- vocabulary tasks

    Yi Li, Hualiang Wang, Yiqun Duan, and Xiaomeng Li. Clip surgery for better explainability with enhancement in open- vocabulary tasks. arXiv preprint arXiv:2304.05653, 2023. 1, 3

    2023 arXiv

  25. [33]

    Clip is also an ef- ficient segmenter: A text-driven approach for weakly super- vised semantic segmentation

    Yuqi Lin, Minghao Chen, Wenxiao Wang, Boxi Wu, Ke Li, Binbin Lin, Haifeng Liu, and Xiaofei He. Clip is also an ef- ficient segmenter: A text-driven approach for weakly super- vised semantic segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 15305–15314, 2023. 2, 4, 6

    2023

  26. [34]

    Open-world semantic seg- mentation via contrasting and clustering vision-language embedding

    Quande Liu, Youpeng Wen, Jianhua Han, Chunjing Xu, Hang Xu, and Xiaodan Liang. Open-world semantic seg- mentation via contrasting and clustering vision-language embedding. In Eur. Conf. Comput. Vis. , pages 275–292. Springer, 2022. 1

    2022

  27. [35]

    Segclip: Patch aggregation with learnable centers for open-vocabulary semantic segmentation

    Huaishao Luo, Junwei Bao, Youzheng Wu, Xiaodong He, and Tianrui Li. Segclip: Patch aggregation with learnable centers for open-vocabulary semantic segmentation. In Int. Conf. Mach. Learn., pages 23033–23044. PMLR, 2023. 1, 3

    2023

  28. [36]

    The role of context for object detection and se- mantic segmentation in the wild

    Roozbeh Mottaghi, Xianjie Chen, Xiaobai Liu, Nam-Gyu Cho, Seong-Whan Lee, Sanja Fidler, Raquel Urtasun, and Alan Yuille. The role of context for object detection and se- mantic segmentation in the wild. InIEEE Conf. Comput. Vis. Pattern Recog., pages 891–898, 2014. 6

    2014

  29. [37]

    Dinov2: Learning robust visual features without super- vision

    Maxime Oquab, Timoth ´ee Darcet, Th´eo Moutakanni, Huy V V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without super- vision. Transactions on Machine Learning Research, 2023. 3

    2023

  30. [38]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In Int. Conf. Mach. Learn., pages 8748–8763. PMLR,

  31. [39]

    Per- ceptual grouping in contrastive vision-language models

    Kanchana Ranasinghe, Brandon McKinzie, Sachin Ravi, Yinfei Yang, Alexander Toshev, and Jonathon Shlens. Per- ceptual grouping in contrastive vision-language models. In Int. Conf. Comput. Vis., pages 5571–5584, 2023. 3

    2023

  32. [40]

    Viewco: Discovering text-supervised segmentation masks via multi-view semantic consistency

    Pengzhen Ren, Changlin Li, Hang Xu, Yi Zhu, Guangrun Wang, Jianzhuang Liu, Xiaojun Chang, and Xiaodan Liang. Viewco: Discovering text-supervised segmentation masks via multi-view semantic consistency. In Int. Conf. Learn. Represent., 2023. 3

    2023

  33. [41]

    Zero- guidance segmentation using zero segment labels

    Pitchaporn Rewatbowornwong, Nattanat Chatthee, Ekapol Chuangsuwanich, and Supasorn Suwajanakorn. Zero- guidance segmentation using zero segment labels. In Int. Conf. Comput. Vis., pages 1162–1172, 2023. 2, 6, 7

    2023

  34. [42]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In IEEE Conf. Comput. Vis. Pattern Recog., pages 10684–10695, 2022. 3

    2022

  35. [43]

    Learn- ing affinity from attention: End-to-end weakly-supervised semantic segmentation with transformers

    Lixiang Ru, Yibing Zhan, Baosheng Yu, and Bo Du. Learn- ing affinity from attention: End-to-end weakly-supervised semantic segmentation with transformers. In IEEE Conf. Comput. Vis. Pattern Recog., pages 16846–16855, 2022. 1, 2, 4

    2022

  36. [44]

    Ex- plore the potential of clip for training-free open vocabulary semantic segmentation

    Tong Shao, Zhuotao Tian, Hang Zhao, and Jingyong Su. Ex- plore the potential of clip for training-free open vocabulary semantic segmentation. In Eur. Conf. Comput. Vis., 2024. 3, 6, 7

    2024

  37. [45]

    Reco: Re- trieve and co-segment for zero-shot transfer

    Gyungin Shin, Weidi Xie, and Samuel Albanie. Reco: Re- trieve and co-segment for zero-shot transfer. In Adv. Neural Inform. Process. Syst., pages 33754–33767, 2022. 3, 6, 7

    2022

  38. [46]

    Clip as rnn: Segment countless visual concepts with- out training endeavor

    Shuyang Sun, Runjia Li, Philip Torr, Xiuye Gu, and Siyang Li. Clip as rnn: Segment countless visual concepts with- out training endeavor. In IEEE Conf. Comput. Vis. Pattern Recog., 2024. 3, 6, 7

    2024

  39. [47]

    Sclip: Rethinking self-attention for dense vision-language inference

    Feng Wang, Jieru Mei, and Alan Yuille. Sclip: Rethinking self-attention for dense vision-language inference. In Eur. Conf. Comput. Vis., 2024. 1, 2, 3, 6, 7

    2024

  40. [48]

    Image-to-image matching via foundation models: A new perspective for open-vocabulary semantic segmentation

    Yuan Wang, Rui Sun, Naisong Luo, Yuwen Pan, and Tianzhu Zhang. Image-to-image matching via foundation models: A new perspective for open-vocabulary semantic segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., 2024. 1, 6, 7

    2024

  41. [49]

    Image-text co- decomposition for text-supervised semantic segmentation

    Ji-Jia Wu, Andy Chia-Hao Chang, Chieh-Yu Chuang, Chun-Pei Chen, Yu-Lun Liu, Min-Hung Chen, Hou-Ning Hu, Yung-Yu Chuang, and Yen-Yu Lin. Image-text co- decomposition for text-supervised semantic segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., 2024. 3

    2024

  42. [50]

    Clipself: Vision trans- former distills itself for open-vocabulary dense prediction

    Size Wu, Wenwei Zhang, Lumin Xu, Sheng Jin, Xiangtai Li, Wentao Liu, and Chen Change Loy. Clipself: Vision trans- former distills itself for open-vocabulary dense prediction. In Int. Conf. Learn. Represent., 2024. 3 10

    2024

  43. [51]

    Clip-dinoiser: Teaching clip a few dino tricks for open- vocabulary semantic segmentation

    Monika Wysocza ´nska, Oriane Sim´eoni, Micha¨el Ramamon- jisoa, Andrei Bursuc, Tomasz Trzci ´nski, and Patrick P ´erez. Clip-dinoiser: Teaching clip a few dino tricks for open- vocabulary semantic segmentation. In Eur. Conf. Comput. Vis., 2024. 3

    2024

  44. [52]

    Segformer: Simple and ef- ficient design for semantic segmentation with transformers

    Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, and Ping Luo. Segformer: Simple and ef- ficient design for semantic segmentation with transformers. In Adv. Neural Inform. Process. Syst. , pages 12077–12090,

  45. [53]

    Rewrite caption semantics: Bridging seman- tic gaps for language-supervised semantic segmentation

    Yun Xing, Jian Kang, Aoran Xiao, Jiahao Nie, Ling Shao, and Shijian Lu. Rewrite caption semantics: Bridging seman- tic gaps for language-supervised semantic segmentation. In Adv. Neural Inform. Process. Syst., 2024. 3

    2024

  46. [54]

    Groupvit: Semantic segmentation emerges from text supervision

    Jiarui Xu, Shalini De Mello, Sifei Liu, Wonmin Byeon, Thomas Breuel, Jan Kautz, and Xiaolong Wang. Groupvit: Semantic segmentation emerges from text supervision. In IEEE Conf. Comput. Vis. Pattern Recog. , pages 18134– 18144, 2022. 1

    2022

  47. [55]

    Learning open-vocabulary semantic segmentation models from natural language supervision

    Jilan Xu, Junlin Hou, Yuejie Zhang, Rui Feng, Yi Wang, Yu Qiao, and Weidi Xie. Learning open-vocabulary semantic segmentation models from natural language supervision. In IEEE Conf. Comput. Vis. Pattern Recog., pages 2935–2944,

  48. [56]

    Open-vocabulary panop- tic segmentation with text-to-image diffusion models

    Jiarui Xu, Sifei Liu, Arash Vahdat, Wonmin Byeon, Xiao- long Wang, and Shalini De Mello. Open-vocabulary panop- tic segmentation with text-to-image diffusion models. In IEEE Conf. Comput. Vis. Pattern Recog., pages 2955–2966,

  49. [57]

    A simple baseline for open- vocabulary semantic segmentation with pre-trained vision- language model

    Mengde Xu, Zheng Zhang, Fangyun Wei, Yutong Lin, Yue Cao, Han Hu, and Xiang Bai. A simple baseline for open- vocabulary semantic segmentation with pre-trained vision- language model. In Eur. Conf. Comput. Vis., pages 736–753. Springer, 2022. 1

    2022

  50. [58]

    Side adapter network for open-vocabulary semantic segmentation

    Mengde Xu, Zheng Zhang, Fangyun Wei, Han Hu, and Xi- ang Bai. Side adapter network for open-vocabulary semantic segmentation. In IEEE Conf. Comput. Vis. Pattern Recog. , pages 2945–2954, 2023. 3

    2023

  51. [59]

    A simple framework for text- supervised semantic segmentation

    Muyang Yi, Quan Cui, Hao Wu, Cheng Yang, Osamu Yoshie, and Hongtao Lu. A simple framework for text- supervised semantic segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 7071–7080, 2023. 3

    2023

  52. [60]

    Convolutions die hard: Open-vocabulary seg- mentation with single frozen convolutional clip

    Qihang Yu, Ju He, Xueqing Deng, Xiaohui Shen, and Liang- Chieh Chen. Convolutions die hard: Open-vocabulary seg- mentation with single frozen convolutional clip. InAdv. Neu- ral Inform. Process. Syst., 2024. 3

    2024

  53. [61]

    Uncovering prototypical knowledge for weakly open- vocabulary semantic segmentation

    Fei Zhang, Tianfei Zhou, Boyang Li, Hao He, Chaofan Ma, Tianjiao Zhang, Jiangchao Yao, Ya Zhang, and Yanfeng Wang. Uncovering prototypical knowledge for weakly open- vocabulary semantic segmentation. In Adv. Neural Inform. Process. Syst., 2024. 3

    2024

  54. [62]

    For- mulating discrete probability flow through optimal transport

    Pengze Zhang, Hubery Yin, Chen Li, and Xiaohua Xie. For- mulating discrete probability flow through optimal transport. In Adv. Neural Inform. Process. Syst., 2023. 3

    2023

  55. [63]

    Instance-specific and model-adaptive supervision for semi-supervised semantic segmentation

    Zhen Zhao, Sifan Long, Jimin Pi, Jingdong Wang, and Lup- ing Zhou. Instance-specific and model-adaptive supervision for semi-supervised semantic segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., pages 23705–23714, 2023. 1

    2023

  56. [64]

    Learning deep features for discrimina- tive localization

    Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discrimina- tive localization. In IEEE Conf. Comput. Vis. Pattern Recog., pages 2921–2929, 2016. 2, 3

    2016

  57. [65]

    Semantic under- standing of scenes through the ade20k dataset

    Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fi- dler, Adela Barriuso, and Antonio Torralba. Semantic under- standing of scenes through the ade20k dataset. Int. J. Com- put. Vis., 127:302–321, 2019. 6

    2019

  58. [66]

    Extract free dense labels from clip

    Chong Zhou, Chen Change Loy, and Bo Dai. Extract free dense labels from clip. In Eur. Conf. Comput. Vis. , pages 696–712, 2022. 1, 3, 6, 7

    2022

  59. [67]

    Conditional prompt learning for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Zi- wei Liu. Conditional prompt learning for vision-language models. In IEEE Conf. Comput. Vis. Pattern Recog., 2022. 3 11

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.