Pith. sign in

REVIEW 4 major objections 4 minor 96 references

DeCLIP: Decoupled Learning for Open-Vocabulary Dense Perception

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read DeCLIP shows that decoupling CLIP's self-attention into content and context streams fixes its dense-prediction weakness, giving state-of-the-art open-vocabulary detection and segmentation.

desk verdict DeCLIP's decoupled-distillation recipe is worth taking seriously, and the benchmark results are consistently strong; the main caveat is that the key Table 1 comparison doesn't isolate Q-Q attention from decoupling. read the letter →

arxiv 2505.04410 v1 pith:6UUVK3R6 submitted 2025-05-07 cs.CV

Junjie Wang , Bin Chen , Yulin Li , Bin Kang , Yichi Chen , Zhuotao Tian This is my paper · ORCID
classification cs.CV
keywords open-vocabularydetectionsemanticsegmentationCLIPfine-tuningdecoupledattentionself-distillationvisionfoundationmodelsdensepredictionproxytokenphenomenon
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that CLIP's weak performance on dense visual tasks has a specific cause: in deep layers, CLIP's image tokens stop attending to their semantic neighbors and cluster onto a few 'proxy' background tokens, so the resulting features lack local discriminability and spatial consistency. To fix this, DeCLIP splits the last self-attention block into two parallel feature streams: 'content' features, trained by aligning pooled region features with image-crop [CLS] representations, and 'context' features, trained to reproduce the token-correlation structure of a vision foundation model such as DINOv2. The decoupling is key: doing both distillations on the same features hurts region classification, while separating them improves both tasks. The paper shows large gains on open-vocabulary object detection and semantic segmentation benchmarks, including new state-of-the-art results when DeCLIP is used as the backbone of CAT-Seg. The result is a general unsupervised pre-fine-tuning recipe: upgrading CLIP's local features for dense perception without any dense labels.

What carries the argument

The load-bearing object is the decoupled attention module at CLIP's last block, which splits the representation into context features X_context = Proj_q(X) and content features X_content = Proj(Attn_context V), where Attn_context = SoftMax(X_context X_context^T / $\sqrt$(d)). This separates the query side (which tokens relate to which) from the value side (what semantic content each token carries), so two teachers can act without conflict: a cosine-similarity loss pulls pooled content-region features toward CLIP's own crop-level [CLS] embeddings, and an L2 loss pulls the pairwise cosine correlation matrix of the context tokens toward that of a vision foundation model such as DINOv2. The total objective is L_total = L_content + $\lambda$ * L_context with $\lambda$ = 0.25.

What would settle it

Run DeCLIP's two distillation losses on EVA-CLIP while keeping the original Q-K attention in the last block; if most of the reported mIoU and mAP gains disappear, the Q-Q attention replacement rather than the decoupled distillation is the active ingredient. Also check whether EVA-CLIP's attention maps show the proxy-token pattern before fine-tuning; if they do not, the causal story needs revision.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that CLIP's dense features are corrupted by a 'proxy token' phenomenon: starting around layer 7, both the [CLS] token and ordinary image tokens concentrate their attention on a few background tokens, no matter where the query token lies, so tokens from the same object fail to correlate. The paper claims this is why CLIP transfers poorly to detection and segmentation, and that the fix is to decouple the final self-attention into a context stream, which defines spatial and semantic relations, and a content stream, which carries visual-language semantics. The content stream is disciplined by self-distillation, pulling pooled region features toward CLIP's own crop-level [CLS] embeddings, while the context stream is disciplined by matching its pairwise token correlation volume to that of a vision foundation model such as DINOv2. Trained this way on COCO images alone, DeCLIP improves region classification at every resolution tested and lifts open-vocabulary detection and segmentation across multiple benchmarks.

Load-bearing premise

The load-bearing premise is that the proxy-token attention pattern is the root cause of CLIP's poor dense prediction, and that the decoupled Q-Q attention plus distillation repairs that cause; the paper does not isolate the Q-Q change from the known training-free trick, and the attention analysis was run on a different CLIP model than the one it fine-tunes.

Editorial extensions

If this is right

  • On OV-COCO, DeCLIP raises novel-class AP of F-ViT from 37.6 to 41.1 with ViT-B/16 and of OV-DQUO from 39.2 to 46.1, with similar gains on OV-LVIS rare classes.
  • As a drop-in backbone for CAT-Seg, DeCLIP with ViT-B/16 nearly matches prior state-of-the-art results that use much larger encoders such as ConvNeXt-L, and DeCLIP with ViT-L/14 sets new state-of-the-art numbers on open-vocabulary semantic segmentation.
  • On training-free VLM-feature segmentation, DeCLIP averages 41.9 mIoU across eight benchmarks, outperforming existing training-free methods and improving on every individual benchmark.
  • Decoupling the distillation matters: combining self-distillation and VFM distillation without decoupling hurts region classification (COCO Thing mAcc drops 3.9), while the decoupled version improves both region classification and segmentation.
  • The recipe is teacher-agnostic: DINO, SAM, and DINOv2 all work as context teachers, with DINOv2 giving the best balance between region classification and segmentation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not isolate the Q-Q attention replacement from the decoupled distillation; because the training-free Q-Q trick is known to improve dense CLIP features on its own, part of DeCLIP's gain may come from that trick rather than from the decoupled losses. A control experiment with standard Q-K attention and the same two losses would settle this.
  • The proxy-token diagnosis is demonstrated on one CLIP variant, while DeCLIP is fine-tuned on EVA-CLIP; whether EVA-CLIP shows the same proxy-token pattern is not shown, so the causal story may be incomplete.
  • If the decoupling is the key, the recipe should transfer to other CLIP-style models and other context teachers; using a boundary-sensitive teacher such as SAM could sharpen instance edges in segmentation, a natural extension the paper does not test.
  • The content loss relies on CLIP's own crop-level [CLS] as teacher, so DeCLIP's ceiling is partly set by CLIP's image-level accuracy; extending content distillation to a stronger text-grounded teacher might push region classification further.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. DeCLIP is an unsupervised pre-fine-tuning method that modifies the final self-attention block of a CLIP image encoder to produce two features: a context feature X_context, formed by Q-Q self-attention, and a content feature X_content, formed by applying that attention to V. The content feature is distilled toward CLIP's own crop-level [CLS] representations, while the context feature is distilled toward the correlation volumes of a frozen vision foundation model such as DINOv2, SAM, or DINO. The paper reports consistent improvements over strong baselines in open-vocabulary detection with F-ViT and OV-DQUO, open-vocabulary semantic segmentation with CAT-Seg, training-free VLM-feature segmentation, and region classification. It also contributes an attention-visualization analysis of a 'proxy token' phenomenon in CLIP's image tokens.

Significance. If the decoupling mechanism is responsible for the gains, the method is a valuable general recipe: it requires only unlabeled images, transfers across detection and segmentation frameworks, and is supported by a broad benchmark suite, a code release, and sanity checks against EVA-CLIP and ClearCLIP. The empirical scope is the paper's main strength: Tables 2-5 cover multiple detectors, segmenters, resolutions, and datasets, and the gains are consistent even when modest. The central caveat is that Table 1, the key evidence for the decoupling conclusion, changes two variables at once (attention formulation and loss structure), so the mechanism-level claim is not yet isolated. I did not find a circularity problem: the content branch uses the frozen CLIP itself as teacher and the context branch uses an external VFM; no evaluation labels are used in training.

major comments (4)
  1. [Table 1; Sec. 3.1, Eqs. (2) and (6)] The row 'Self+VFM Distillation' and the row 'Self+VFM+Decouple' differ in two simultaneous ways: the attention is changed from Q-K^T to Q-Q^T and residual connections are removed, and the single feature is split into content and context branches with separate losses. Since SCLIP and ClearCLIP already demonstrate that the Q-Q attention change alone improves dense CLIP features, the reported gains of +5.5 mAcc and +6.7 mIoU cannot be attributed to the decoupled distillation without an additional control. Please add an ablation that keeps the decoupled Q-Q attention fixed and removes only the VFM context loss (L_context=0), and ideally a second variant that keeps the combined loss on a single Q-K feature. This is the minimal experiment needed to support the paper's central claim.
  2. [Sec. 2.2, Fig. 3(a); Supp. Tables 9-10] The 'proxy token' phenomenon is visualized for OpenAI CLIP, but DeCLIP is trained from EVA-CLIP. The supplementary sanity checks show that EVA-CLIP is not a drop-in equivalent: vanilla EVA-CLIP already outperforms OpenAI CLIP on VLM-feature segmentation (18.9 vs 14.1 average mIoU in Table 10), and EVA-ClearCLIP underperforms ClearCLIP. Please provide the same attention-map analysis for EVA-CLIP, or restrict the causal narrative to the backbone actually used in the experiments.
  3. [Sec. 3.1, Fig. 5] The paper does not explicitly state which feature is used as the backbone representation at inference. Equation (5) defines X_content as a function of Attn_context and V, and X_context is also available, but the downstream detectors and segmenters are said to use 'DeCLIP' as the image encoder without specifying whether they consume X_content, X_context, or a combination. Please state the inference path explicitly, including how the final projection and residual/FFN operations are treated after the modified last attention block.
  4. [Sec. 3.1, Eqs. (4)-(6)] The claim that content and context features are optimized 'without interference' is not exact: X_content = Proj(Attn_context * V), so the content loss propagates gradients into X_context through the attention weights, and both losses share parameters. The decoupling is at the loss level rather than a full separation of the computational graph. Please clarify this point and discuss whether the optimization-conflict story is consistent with the actual gradient structure.
minor comments (4)
  1. [Table 1, row 2] The row labeled 'Self+VFM Distillation [36]' cites reference [36], which is Segment Anything; this citation does not correspond to a distillation scheme and should be corrected or replaced with an explicit statement that the row was implemented by the authors.
  2. [Sec. 3.2, Eq. (7)] The paragraph begins with 'we employ an image patching method' using a lowercase 'we'; this should be 'We employ' and the sentence should be reworded for clarity.
  3. [Supp. Table 11 and Sec. 7.2] There are minor typos in the supplementary material: 'Sentitivity Analysis' should be 'Sensitivity Analysis', and 'OVSS based on VLM feautures' should be 'features'.
  4. [Sec. 4.3 and Supp. Sec. 9.2] The number of sub-regions k in the content distillation is randomly sampled from [1,6], but no sensitivity study is reported for this choice; a short ablation would improve reproducibility and confidence in the robustness of the content loss.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DeCLIP's gains are held-out empirical results from self- and external-VFM distillation, not fitted or self-referential predictions.

full rationale

DeCLIP's claimed derivation chain is an empirical pipeline rather than a derivation that reduces to its own inputs. The paper observes CLIP's attention behavior (the 'proxy' token phenomenon), proposes an architectural decoupling of the last self-attention block into content and context features, and trains those features with two distillation losses: a content loss aligning region features with the same CLIP model's crop CLS embeddings, and a context loss aligning correlation volumes from an external vision foundation model such as DINOv2. The evaluation then occurs on held-out open-vocabulary detection and segmentation benchmarks, which are not used to define any loss or architectural component. No equation in the method section equals a benchmark metric, no fitted parameter is renamed as a prediction, and no load-bearing result is imported solely through self-citation. The use of CLIP itself as teacher for the content branch is self-distillation by design, but it is a training objective that improves region-text alignment, not a way of encoding the downstream benchmark. The paper also transparently credits the Q-Q attention modification to prior training-free methods (SCLIP, ClearCLIP), so the decoupled attention is presented as an acknowledged design choice rather than a derived theorem. The possible confound in Table 1—where the final row changes both the attention mechanism and the decoupled distillation—and the fact that the motivating attention maps are shown for OpenAI CLIP while DeCLIP is trained on EVA-CLIP, are experimental validity concerns, not circularity. The sanity checks in Tables 9 and 10 partially address the EVA-CLIP concern, and the only self-citations (OV-DQUO as a baseline) are not load-bearing evidence for the central claim. The central claims are genuine held-out empirical improvements, so the paper is self-contained against external benchmarks and receives a circularity score of 0.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The method relies on assumptions about the proxy-token phenomenon, the transferability of the Q-Q attention trick, and the suitability of VFM correlations as a spatial teacher. These are reasonable empirical assumptions supported by ablations and prior work, but they are not proven from first principles.

free parameters (6)
  • lambda (context loss weight) = 0.25
    Balances L_content and L_context in Eq. (10); chosen from sensitivity analysis (appendix Table 11); defaults to 0.25 in all experiments.
  • F-ViT temperature for OV-COCO = 45
    Set to 45 for F-ViT integration on OV-COCO; temperature affects logit scaling in detection and was adjusted by the authors (Sec. 9.2).
  • F-ViT temperature for OV-LVIS = 90
    Set to 90 for F-ViT on OV-LVIS (Sec. 9.2).
  • OV-DQUO temperature = 50
    Set to 50 for OV-DQUO on both OV-COCO and OV-LVIS (Sec. 9.2).
  • fine-tuned attention layers = 12 (all blocks)
    Ablation (appendix Table 8) shows region classification best at 12 layers; segmentation peaks at 6; authors choose 12 to balance tasks.
  • sub-region count for content distillation = random m,n in [1,6]
    The image is divided into k = m x n sub-regions with m,n sampled from [1,6] (Sec. 9.2); a distribution rather than a single fitted value.
assumptions (6)
  • domain assumption VFM feature correlations are a suitable teacher for CLIP's context features, improving spatial consistency without harming vision-language alignment.
    Assumed in Sec. 3.2; empirically supported by ablations (Table 6) but not derived.
  • domain assumption The proxy-token phenomenon observed in OpenAI CLIP attention maps also occurs in EVA-CLIP, the backbone used for DeCLIP.
    The motivation (Sec. 2.2) visualizes OpenAI CLIP; DeCLIP is trained on EVA-CLIP (Sec. 9.2); no direct evidence for EVA-CLIP is provided.
  • domain assumption Replacing standard Q-K attention with Q-Q attention in the final block preserves enough vision-language alignment for open-vocabulary tasks.
    The content branch uses Attn = softmax(Q Q^T / sqrt(d)) (Eq. 6); prior work SCLIP/ClearCLIP support this, but in DeCLIP it is coupled with distillation and not ablated alone.
  • domain assumption Self-distillation from image-crop CLS tokens increases region-level discriminability.
    Inherited from CLIPSelf [68]; assumed in Sec. 3.2, not re-derived.
  • standard math Cosine similarity and L2 losses in Eqs. (7) and (9) define well-behaved gradients for distillation.
    Standard differentiable losses; no proof given.
  • domain assumption Matching token counts by adjusting resolutions (CLIP at 1024, VFM at 896) gives valid correspondences for correlation distillation.
    Required for the L_context loss in Eq. (9); implemented in Sec. 9.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DeCLIP: Decoupled Learning for Open-Vocabulary Dense Perception." pith.science (2026). https://pith.science/paper/6UUVK3R6

@misc{pith2026250504410,
  author       = {Pith},
  title        = {Pith review of: DeCLIP: Decoupled Learning for Open-Vocabulary Dense Perception},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6UUVK3R6}},
  note         = {Machine review of arXiv:2505.04410}
}
read the original abstract

Dense visual prediction tasks have been constrained by their reliance on predefined categories, limiting their applicability in real-world scenarios where visual concepts are unbounded. While Vision-Language Models (VLMs) like CLIP have shown promise in open-vocabulary tasks, their direct application to dense prediction often leads to suboptimal performance due to limitations in local feature representation. In this work, we present our observation that CLIP's image tokens struggle to effectively aggregate information from spatially or semantically related regions, resulting in features that lack local discriminability and spatial consistency. To address this issue, we propose DeCLIP, a novel framework that enhances CLIP by decoupling the self-attention module to obtain ``content'' and ``context'' features respectively. The ``content'' features are aligned with image crop representations to improve local discriminability, while ``context'' features learn to retain the spatial correlations under the guidance of vision foundation models, such as DINO. Extensive experiments demonstrate that DeCLIP significantly outperforms existing methods across multiple open-vocabulary dense prediction tasks, including object detection and semantic segmentation. Code is available at \textcolor{magenta}{https://github.com/xiaomoguhz/DeCLIP}.

Figures

Figures reproduced from arXiv: 2505.04410 by the authors.

Figure 1
Figure 1. DeCLIP outperforms previous state-of-the-art models on [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Quantitative and qualitative comparisons between our method and CLIP. (a) Performance comparisons of open-vocabulary dense predictions on COCO [43]. (b) Attention map comparisons, with the anchor image token marked in red. with the correlations among other image tokens, leading to suboptimal performance in dense prediction tasks. Specifically, we have observed that in deeper layers (be￾hind the 9th layer), the [CLS]… view at source ↗
Figure 3
Figure 3. Visualization of attention maps across different encoding layers of CLIP and VFM. The attention weights are calculated at a low resolution, then averaged across different heads, and finally upsampled to the original image resolution for visualization. The anchor image token is marked in red. We observe the occurrence of the “proxy” token phenomenon in CLIP, but not in VFM. Furthermore, when the position of the ancho… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Pre-fine-tuning methods for adapting CLIP to dense prediction tasks. Existing work considers establishing region-text alignment through cost-effective methods via: (a) using images as pseudo regions or (b) using self-distillation on image patches. The former regards th…
Figure 5
Figure 5. Figure 5: Illustration of the DeCLIP framework. We decouple CLIP’s final attention module into context and content features for distillation, avoiding optimization conflicts between feature correlations and visual-language alignment. CLIP itself serves as the teacher for content…
Figure 6
Figure 6. Figure 6: Comparisons between DeCLIP and existing methods in terms of open-vocabulary region classification ability at different resolu [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Qualitative comparisons of attention maps between [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Visualization of the “proxy” token phenomenon in the attention maps of the CLIP visual encoder. Specifically, the input image resolution is 224*224. We extract the attention weights from each attention block of CLIP and average them across the multi￾head dimension (aft…
Figure 9
Figure 9. Figure 9: Qualitative comparison of feature correlations between DeCLIP and existing pre-fine-tuning approaches [68, 85]. Specif￾ically, the input image resolution is 336*336. We extract the output features from each attention block of CLIP, where each feature F ∈ R 441×D. Then,…
Figure 10
Figure 10. Figure 10: Qualitative comparison of the open-vocabulary semantic segmentation results between DeCLIP and existing approaches [ [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: Comprehensive comparison of attention maps between CLIP and DeCLIP. The left side presents images of various styles [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

96 extracted references · 59 canonical work pages

  1. [1]

    Mim-refiner: A contrastive learning boost from intermediate pre-trained representations

    Benedikt Alkin, Lukas Miklautz, Sepp Hochreiter, and Jo- hannes Brandstetter. Mim-refiner: A contrastive learning boost from intermediate pre-trained representations. arXiv preprint arXiv:2402.10093, 2024. 21

    arXiv 2024

  2. [2]

    Multi-label cluster discrimination for vi- sual representation learning

    Xiang An, Kaicheng Yang, Xiangzi Dai, Ziyong Feng, and Jiankang Deng. Multi-label cluster discrimination for vi- sual representation learning. In European Conference on Computer Vision, pages 428–444. Springer, 2025. 21

    2025

  3. [3]

    Coco- stuff: Thing and stuff classes in context

    Holger Caesar, Jasper Uijlings, and Vittorio Ferrari. Coco- stuff: Thing and stuff classes in context. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1209–1218, 2018. 17, 20

    2018

  4. [4]

    End- to-end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End- to-end object detection with transformers. In European conference on computer vision, pages 213–229. Springer,

  5. [5]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9650–9660, 2021. 4, 7, 8, 21

    2021

  6. [6]

    Learn- ing to generate text-grounded mask for open-world semantic segmentation from only image-text pairs

    Junbum Cha, Jonghwan Mun, and Byungseok Roh. Learn- ing to generate text-grounded mask for open-world semantic segmentation from only image-text pairs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11165–11174, 2023. 8

    2023

  7. [7]

    Enhanced training of query- based object detection via selective query recollection

    Fangyi Chen, Han Zhang, Kai Hu, Yu-Kai Huang, Chenchen Zhu, and Marios Savvides. Enhanced training of query- based object detection via selective query recollection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 23756– 23765, 2023. 1

    2023

  8. [8]

    Rtgen: Generating region-text pairs for open-vocabulary object detection

    Fangyi Chen, Han Zhang, Zhantao Yang, Hao Chen, Kai Hu, and Marios Savvides. Rtgen: Generating region-text pairs for open-vocabulary object detection. arXiv preprint arXiv:2405.19854, 2024. 1, 17

    arXiv 2024

Show all 96 references
  1. [9]

    Exploring open-vocabulary semantic segmentation from clip vision encoder distilla- tion only

    Jun Chen, Deyao Zhu, Guocheng Qian, Bernard Ghanem, Zhicheng Yan, Chenchen Zhu, Fanyi Xiao, Sean Chang Cu- latana, and Mohamed Elhoseiny. Exploring open-vocabulary semantic segmentation from clip vision encoder distilla- tion only. In Proceedings of the IEEE/CVF International ...

    2023

  2. [10]

    An empiri- cal study of training self-supervised vision transformers

    Xinlei Chen, Saining Xie, and Kaiming He. An empiri- cal study of training self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9640–9649, 2021. 21

    2021

  3. [11]

    Frozenseg: Harmonizing frozen foundation models for open-vocabulary segmentation

    Xi Chen, Haosen Yang, Sheng Jin, Xiatian Zhu, and Hongxun Yao. Frozenseg: Harmonizing frozen foundation models for open-vocabulary segmentation. arXiv preprint arXiv:2409.03525, 2024. 1, 8, 21

    2024 arXiv

  4. [12]

    Masked-attention mask transformer for universal image segmentation

    Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1290–1299, 2022. 1

    2022

  5. [13]

    Reproducible scaling laws for contrastive language-image learning

    Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuh- mann, Ludwig Schmidt, and Jenia Jitsev. Reproducible scaling laws for contrastive language-image learning. In Proceedings of the IEEE/CVF Conference on Computer Visi...

    2023

  6. [14]

    Cat- seg: Cost aggregation for open-vocabulary semantic seg- mentation

    Seokju Cho, Heeseong Shin, Sunghwan Hong, Anurag Arnab, Paul Hongsuck Seo, and Seungryong Kim. Cat- seg: Cost aggregation for open-vocabulary semantic seg- mentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4113– 4123, 2024. ...

    2024

  7. [15]

    The cityscapes dataset for semantic urban scene understanding

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern reco...

    2016

  8. [16]

    Vision transformers need registers

    Timoth ´ee Darcet, Maxime Oquab, Julien Mairal, and Pi- otr Bojanowski. Vision transformers need registers. arXiv preprint arXiv:2309.16588, 2023. 13, 21

    2023 arXiv

  9. [17]

    De- coupling zero-shot semantic segmentation

    Jian Ding, Nan Xue, Gui-Song Xia, and Dengxin Dai. De- coupling zero-shot semantic segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11583–11592, 2022. 8, 20, 21

    2022

  10. [18]

    De- coupling zero-shot semantic segmentation

    Jian Ding, Nan Xue, Gui-Song Xia, and Dengxin Dai. De- coupling zero-shot semantic segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11583–11592, 2022. 20

    2022

  11. [19]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 2, 13

    2010 arXiv

  12. [20]

    Learning to prompt for open-vocabulary ob- ject detection with vision-language model

    Yu Du, Fangyun Wei, Zihe Zhang, Miaojing Shi, Yue Gao, and Guoqi Li. Learning to prompt for open-vocabulary ob- ject detection with vision-language model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14084–14093, 2022. 7, 17

    2022

  13. [21]

    The pascal visual object classes (voc) challenge

    Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88:303–338, 2010. 20

    2010

  14. [22]

    Eva-02: A visual representation for neon genesis

    Yuxin Fang, Quan Sun, Xinggang Wang, Tiejun Huang, Xin- long Wang, and Yue Cao. Eva-02: A visual representation for neon genesis. Image and Vision Computing, 149:105171,

  15. [23]

    Scal- ing open-vocabulary image segmentation with image-level labels

    Golnaz Ghiasi, Xiuye Gu, Yin Cui, and Tsung-Yi Lin. Scal- ing open-vocabulary image segmentation with image-level labels. In European Conference on Computer Vision, pages 540–557. Springer, 2022. 20

    2022

  16. [24]

    Open-vocabulary object detection via vision and language knowledge distillation

    Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, and Yin Cui. Open-vocabulary object detection via vision and language knowledge distillation. arXiv preprint arXiv:2104.13921,

    arXiv

  17. [25]

    Lvis: A dataset for large vocabulary instance segmentation

    Agrim Gupta, Piotr Dollar, and Ross Girshick. Lvis: A dataset for large vocabulary instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5356–5364, 2019. 17, 18

    2019

  18. [26]

    Global knowledge calibration for fast open-vocabulary segmentation

    Kunyang Han, Yong Liu, Jun Hao Liew, Henghui Ding, Ji- ajun Liu, Yitong Wang, Yansong Tang, Yujiu Yang, Jiashi Feng, Yao Zhao, et al. Global knowledge calibration for fast open-vocabulary segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision,...

    2023

  19. [27]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 2, 13

    2016

  20. [28]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017. 6, 7

    2017

  21. [29]

    Proxydet: Synthesizing proxy novel classes via classwise mixup for open-vocabulary object de- tection

    Joonhyun Jeong, Geondo Park, Jayeon Yoo, Hyungsik Jung, and Heesu Kim. Proxydet: Synthesizing proxy novel classes via classwise mixup for open-vocabulary object de- tection. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 2462–2470, 2024. 17

    2024

  22. [30]

    Learning mask-aware clip representations for zero-shot segmentation

    Siyu Jiao, Yunchao Wei, Yaowei Wang, Yao Zhao, and Humphrey Shi. Learning mask-aware clip representations for zero-shot segmentation. Advances in Neural Information Processing Systems, 36:35631–35653, 2023. 1, 3, 4, 8, 21

    2023

  23. [31]

    Collaborative vision-text rep- resentation optimizing for open-vocabulary segmentation

    Siyu Jiao, Hongguang Zhu, Jiannan Huang, Yao Zhao, Yun- chao Wei, and Humphrey Shi. Collaborative vision-text rep- resentation optimizing for open-vocabulary segmentation. In European Conference on Computer Vision, pages 399–416. Springer, 2025. 3, 21

    2025

  24. [32]

    Diffusion models for zero-shot open-vocabulary segmentation

    Laurynas Karazija, Iro Laina, Andrea Vedaldi, and Christian Rupprecht. Diffusion models for zero-shot open-vocabulary segmentation. arXiv preprint arXiv:2306.09316, 2023. 20, 21

    2023 arXiv

  25. [33]

    Con- trastive feature masking open-vocabulary vision transformer

    Dahun Kim, Anelia Angelova, and Weicheng Kuo. Con- trastive feature masking open-vocabulary vision transformer. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 15556–15566, 2023. 7, 17, 20

    2023

  26. [34]

    Region- aware pretraining for open-vocabulary object detection with vision transformers

    Dahun Kim, Anelia Angelova, and Weicheng Kuo. Region- aware pretraining for open-vocabulary object detection with vision transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11144–11154, 2023. 7, 17

    2023

  27. [35]

    Region- aware pretraining for open-vocabulary object detection with vision transformers

    Dahun Kim, Anelia Angelova, and Weicheng Kuo. Region- aware pretraining for open-vocabulary object detection with vision transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11144–11154, 2023. 7, 17, 20

    2023

  28. [36]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4015–4026, 2023. 4...

    2023

  29. [37]

    F-vlm: Open-vocabulary object detec- tion upon frozen vision and language models

    Weicheng Kuo, Yin Cui, Xiuye Gu, AJ Piergiovanni, and Anelia Angelova. F-vlm: Open-vocabulary object detec- tion upon frozen vision and language models. arXiv preprint arXiv:2209.15639, 2022. 1, 7, 17, 20

    2022 arXiv

  30. [38]

    Clearclip: Decom- posing clip representations for dense vision-language infer- ence

    Mengcheng Lan, Chaofeng Chen, Yiping Ke, Xinjiang Wang, Litong Feng, and Wayne Zhang. Clearclip: Decom- posing clip representations for dense vision-language infer- ence. arXiv preprint arXiv:2407.12442, 2024. 5, 6, 8, 15, 17, 18

    2024 arXiv

  31. [39]

    Language-driven semantic seg- mentation

    Boyi Li, Kilian Q Weinberger, Serge Belongie, Vladlen Koltun, and Ren ´e Ranftl. Language-driven semantic seg- mentation. arXiv preprint arXiv:2201.03546, 2022. 3, 20, 21

    2022 arXiv

  32. [40]

    Mask dino: To- wards a unified transformer-based framework for object de- tection and segmentation

    Feng Li, Hao Zhang, Huaizhe Xu, Shilong Liu, Lei Zhang, Lionel M Ni, and Heung-Yeung Shum. Mask dino: To- wards a unified transformer-based framework for object de- tection and segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, ...

    2023

  33. [41]

    Scaling language-image pre-training via masking

    Yanghao Li, Haoqi Fan, Ronghang Hu, Christoph Feichten- hofer, and Kaiming He. Scaling language-image pre-training via masking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23390– 23400, 2023. 1

    2023

  34. [42]

    Open-vocabulary semantic segmentation with mask-adapted clip

    Feng Liang, Bichen Wu, Xiaoliang Dai, Kunpeng Li, Yinan Zhao, Hang Zhang, Peizhao Zhang, Peter Vajda, and Di- ana Marculescu. Open-vocabulary semantic segmentation with mask-adapted clip. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pag...

    2023

  35. [43]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceeding...

    2014

  36. [44]

    DAB-DETR: Dynamic anchor boxes are better queries for DETR

    Shilong Liu, Feng Li, Hao Zhang, Xiao Yang, Xianbiao Qi, Hang Su, Jun Zhu, and Lei Zhang. DAB-DETR: Dynamic anchor boxes are better queries for DETR. In International Conference on Learning Representations, 2022. 1

    2022

  37. [45]

    A convnet for the 2020s

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feicht- enhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 2, 13

    2022

  38. [46]

    Decoupled weight decay regularization

    I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 20

    2017 arXiv

  39. [47]

    Codet: Co-occurrence guided region-word alignment for open-vocabulary object detection

    Chuofan Ma, Yi Jiang, Xin Wen, Zehuan Yuan, and Xiaojuan Qi. Codet: Co-occurrence guided region-word alignment for open-vocabulary object detection. Advances in Neural Information Processing Systems, 36, 2024. 7, 17

    2024

  40. [48]

    The role of context for object detection and se- mantic segmentation in the wild

    Roozbeh Mottaghi, Xianjie Chen, Xiaobai Liu, Nam-Gyu Cho, Seong-Whan Lee, Sanja Fidler, Raquel Urtasun, and Alan Yuille. The role of context for object detection and se- mantic segmentation in the wild. In Proceedings of the IEEE conference on computer vision and pattern recog...

    2014

  41. [49]

    Open vocabulary semantic segmentation with patch aligned contrastive learning

    Jishnu Mukhoti, Tsung-Yu Lin, Omid Poursaeed, Rui Wang, Ashish Shah, Philip HS Torr, and Ser-Nam Lim. Open vocabulary semantic segmentation with patch aligned contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1941...

    2023

  42. [50]

    Silc: Improving vision language pretraining with self-distillation

    Muhammad Ferjad Naeem, Yongqin Xian, Xiaohua Zhai, Lukas Hoyer, Luc Van Gool, and Federico Tombari. Silc: Improving vision language pretraining with self-distillation. In European Conference on Computer Vision, pages 38–55. Springer, 2025. 5

    2025

  43. [51]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 4, 7, ...

    2023 arXiv

  44. [52]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of th...

    2021

  45. [53]

    Am-radio: Agglomerative vision foundation model reduce all domains into one

    Mike Ranzinger, Greg Heinrich, Jan Kautz, and Pavlo Molchanov. Am-radio: Agglomerative vision foundation model reduce all domains into one. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12490–12500, 2024. 21

    2024

  46. [54]

    Sam 2: Segment anything in images and videos

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714, 2024. 4, 7, 21

    2024 arXiv

  47. [55]

    Faster r-cnn: Towards real-time object detection with re- gion proposal networks

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with re- gion proposal networks. Advances in neural information processing systems, 28, 2015. 1, 20

    2015

  48. [56]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 17, 19

    2022

  49. [57]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, pa...

    2015

  50. [58]

    Objects365: A large-scale, high-quality dataset for object detection

    Shuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. Objects365: A large-scale, high-quality dataset for object detection. In Proceedings of the IEEE/CVF international conference on computer vision, pages 8430–8439, 2019. 7, 17, 18, 19

    2019

  51. [59]

    Explore the potential of clip for training-free open vocab- ulary semantic segmentation

    Tong Shao, Zhuotao Tian, Hang Zhao, and Jingyong Su. Explore the potential of clip for training-free open vocab- ulary semantic segmentation. In European Conference on Computer Vision, pages 139–156. Springer, 2025. 5, 6, 13

    2025

  52. [60]

    Reco: Re- trieve and co-segment for zero-shot transfer

    Gyungin Shin, Weidi Xie, and Samuel Albanie. Reco: Re- trieve and co-segment for zero-shot transfer. Advances in Neural Information Processing Systems, 35:33754–33767,

  53. [61]

    Eva-clip: Improved training techniques for clip at scale

    Quan Sun, Yuxin Fang, Ledell Wu, Xinlong Wang, and Yue Cao. Eva-clip: Improved training techniques for clip at scale. arXiv preprint arXiv:2303.15389, 2023. 1, 15, 20, 21

    2023 arXiv

  54. [62]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017. 2

    2017

  55. [63]

    Sclip: Rethink- ing self-attention for dense vision-language inference

    Feng Wang, Jieru Mei, and Alan Yuille. Sclip: Rethink- ing self-attention for dense vision-language inference. arXiv preprint arXiv:2312.01597, 2023. 5, 6, 8, 17, 18

    2023 arXiv

  56. [64]

    Sam-clip: Merging vision foundation models to- wards semantic and spatial understanding

    Haoxiang Wang, Pavan Kumar Anasosalu Vasu, Fartash Faghri, Raviteja Vemulapalli, Mehrdad Farajtabar, Sachin Mehta, Mohammad Rastegari, Oncel Tuzel, and Hadi Pouransari. Sam-clip: Merging vision foundation models to- wards semantic and spatial understanding. In Proceedings of t...

    2024

  57. [65]

    Ov-dquo: Open- vocabulary detr with denoising text query training and open-world unknown objects supervision

    Junjie Wang, Bin Chen, Bin Kang, Yulin Li, YiChi Chen, Weizhi Xian, and Huifeng Chang. Ov-dquo: Open- vocabulary detr with denoising text query training and open-world unknown objects supervision. arXiv preprint arXiv:2405.17913, 2024. 1, 6, 7, 15, 17, 20, 21

    2024 arXiv

  58. [66]

    Object-aware distillation pyramid for open-vocabulary object detection

    Luting Wang, Yi Liu, Penghui Du, Zihan Ding, Yue Liao, Qiaosong Qi, Biaolong Chen, and Si Liu. Object-aware distillation pyramid for open-vocabulary object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11186–11196, 2023. 20, 21

    2023

  59. [67]

    Aligning bag of regions for open- vocabulary object detection

    Size Wu, Wenwei Zhang, Sheng Jin, Wentao Liu, and Chen Change Loy. Aligning bag of regions for open- vocabulary object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15254–15264, 2023. 1, 7, 17, 20, 21

    2023

  60. [68]

    CLIPSelf: Vision transformer distills itself for open-vocabulary dense predic- tion

    Size Wu, Wenwei Zhang, Lumin Xu, Sheng Jin, Xiangtai Li, Wentao Liu, and Chen Change Loy. CLIPSelf: Vision transformer distills itself for open-vocabulary dense predic- tion. In The Twelfth International Conference on Learning Representations, 2024. 1, 4, 5, 6, 7, 15, 16, 17, ...

    2024

  61. [69]

    Clim: Contrastive language- image mosaic for region representation

    Size Wu, Wenwei Zhang, Lumin Xu, Sheng Jin, Wentao Liu, and Chen Change Loy. Clim: Contrastive language- image mosaic for region representation. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 6117– 6125, 2024. 1, 4, 17, 21

    2024

  62. [70]

    Cora: Adapting clip for open-vocabulary detection with region prompting and anchor pre-matching

    Xiaoshi Wu, Feng Zhu, Rui Zhao, and Hongsheng Li. Cora: Adapting clip for open-vocabulary detection with region prompting and anchor pre-matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7031–7040, 2023. 1, 4, 7, 17, 18, 20, 21

    2023

  63. [71]

    Clip-dinoiser: Teaching clip a few dino tricks.arXiv preprint arXiv:2312.12359, 2023

    Monika Wysocza ´nska, Oriane Sim´eoni, Micha¨el Ramamon- jisoa, Andrei Bursuc, Tomasz Trzci ´nski, and Patrick P ´erez. Clip-dinoiser: Teaching clip a few dino tricks.arXiv preprint arXiv:2312.12359, 2023. 5, 8

    2023 arXiv

  64. [72]

    Sed: A simple encoder-decoder for open- vocabulary semantic segmentation

    Bin Xie, Jiale Cao, Jin Xie, Fahad Shahbaz Khan, and Yanwei Pang. Sed: A simple encoder-decoder for open- vocabulary semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3426–3436, 2024. 3, 21

    2024

  65. [73]

    Groupvit: Semantic segmentation emerges from text supervision

    Jiarui Xu, Shalini De Mello, Sifei Liu, Wonmin Byeon, Thomas Breuel, Jan Kautz, and Xiaolong Wang. Groupvit: Semantic segmentation emerges from text supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18134–18144, 2022. 8, 20

    2022

  66. [74]

    Open-vocabulary panoptic segmentation with text-to-image diffusion models

    Jiarui Xu, Sifei Liu, Arash Vahdat, Wonmin Byeon, Xi- aolong Wang, and Shalini De Mello. Open-vocabulary panoptic segmentation with text-to-image diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2955–2966, 2023. 8

    2023

  67. [75]

    A simple baseline for open- vocabulary semantic segmentation with pre-trained vision- language model

    Mengde Xu, Zheng Zhang, Fangyun Wei, Yutong Lin, Yue Cao, Han Hu, and Xiang Bai. A simple baseline for open- vocabulary semantic segmentation with pre-trained vision- language model. In European Conference on Computer Vision, pages 736–753. Springer, 2022. 8

    2022

  68. [76]

    Side adapter network for open-vocabulary semantic segmentation

    Mengde Xu, Zheng Zhang, Fangyun Wei, Han Hu, and Xi- ang Bai. Side adapter network for open-vocabulary semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2945– 2954, 2023. 8

    2023

  69. [77]

    Masq- clip for open-vocabulary universal image segmentation

    Xin Xu, Tianyi Xiong, Zheng Ding, and Zhuowen Tu. Masq- clip for open-vocabulary universal image segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 887–898, 2023. 3, 21

    2023

  70. [78]

    Convolutions die hard: Open-vocabulary seg- mentation with single frozen convolutional clip

    Qihang Yu, Ju He, Xueqing Deng, Xiaohui Shen, and Liang- Chieh Chen. Convolutions die hard: Open-vocabulary seg- mentation with single frozen convolutional clip. Advances in Neural Information Processing Systems, 36, 2024. 1, 8, 21

    2024

  71. [79]

    Open-vocabulary sam: Segment and recognize twenty-thousand classes interactively

    Haobo Yuan, Xiangtai Li, Chong Zhou, Yining Li, Kai Chen, and Chen Change Loy. Open-vocabulary sam: Segment and recognize twenty-thousand classes interactively. In ECCV,

  72. [80]

    Open-vocabulary seman- tic segmentation using test-time distillation

    Nir Zabari and Yedid Hoshen. Open-vocabulary seman- tic segmentation using test-time distillation. In European Conference on Computer Vision, pages 56–72. Springer,

  73. [81]

    Open-vocabulary detr with conditional matching

    Yuhang Zang, Wei Li, Kaiyang Zhou, Chen Huang, and Chen Change Loy. Open-vocabulary detr with conditional matching. In European Conference on Computer Vision, pages 106–122. Springer, 2022. 7, 17, 20

    2022

  74. [82]

    Open-vocabulary object detection using captions

    Alireza Zareian, Kevin Dela Rosa, Derek Hao Hu, and Shih- Fu Chang. Open-vocabulary object detection using captions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14393–14402, 2021. 1, 18

    2021

  75. [83]

    Exploring region- word alignment in built-in detector for open-vocabulary ob- ject detection

    Heng Zhang, Qiuyu Zhao, Linyu Zheng, Hao Zeng, Zhi- wei Ge, Tianhao Li, and Sulong Xu. Exploring region- word alignment in built-in detector for open-vocabulary ob- ject detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1697...

    2024

  76. [84]

    Shiyu Zhao, Samuel Schulter, Long Zhao, Zhixing Zhang, Vijay Kumar B G, Yumin Suh, Manmohan Chandraker, and Dimitris N. Metaxas. Taming self-training for open- vocabulary object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CV...

    2024

  77. [85]

    Regionclip: Region- based language-image pretraining

    Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chun- yuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, et al. Regionclip: Region- based language-image pretraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...

    2022

  78. [86]

    Semantic under- standing of scenes through the ade20k dataset

    Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fi- dler, Adela Barriuso, and Antonio Torralba. Semantic under- standing of scenes through the ade20k dataset. International Journal of Computer Vision, 127:302–321, 2019. 17, 20

    2019

  79. [87]

    Extract free dense labels from clip

    Chong Zhou, Chen Change Loy, and Bo Dai. Extract free dense labels from clip. In European Conference on Computer Vision, pages 696–712. Springer, 2022. 8, 17, 18

    2022

  80. [88]

    ibot: Image bert pre-training with online tokenizer

    Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan Yuille, and Tao Kong. ibot: Image bert pre-training with online tokenizer. arXiv preprint arXiv:2111.07832,

    arXiv

  81. [89]

    Detecting twenty-thousand classes using image-level supervision

    Xingyi Zhou, Rohit Girdhar, Armand Joulin, Philipp Kr¨ahenb¨uhl, and Ishan Misra. Detecting twenty-thousand classes using image-level supervision. In European Conference on Computer Vision, pages 350–368. Springer,

  82. [90]

    A survey on open- vocabulary detection and segmentation: Past, present, and future

    Chaoyang Zhu and Long Chen. A survey on open- vocabulary detection and segmentation: Past, present, and future. arXiv preprint arXiv:2307.09220, 2023. 20

    2023 arXiv

  83. [91]

    Deformable detr: Deformable trans- formers for end-to-end object detection

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable trans- formers for end-to-end object detection. arXiv preprint arXiv:2010.04159, 2020. 1 DeCLIP: Decoupled Learning for Open-Vocabulary Dense Perception Supplementary Material O...

    2010 arXiv

  84. [92]

    global view

    Details of Proxy Token Phenomenon This section primarily supplements the details of the proxy token phenomenon observed in CLIP, offering deeper in- sights into the rationale behind our proposed DeCLIP. Observation. As stated in the main paper, ViT-based [19] CLIP utilizes the...

  85. [93]

    Additional Experiments 7.1. Ablation Studies In this section, we conduct a thorough ablation study on DeCLIP, encompassing the examination of various Xcontext implementations, the variation in the number of fine-tuning layers, the impact of the hyperparameterλ in the loss func...

  86. [94]

    bird” rather than to be “background

    Additional Qualitative Analysis This section further presents a qualitative experimental analysis of our proposed DeCLIP method in comparison to existing methods, including feature correlation analysis, semantic segmentation results, and attention map compar- isons, thereby pr...

  87. [95]

    Details of Experimental Settings In this section, we present further details and configurations utilized in our experiments. 9.1. Datasets and Evaluation Protocols Open-Vocabulary Detection. Following established set- tings [68, 70, 82], we evaluated our model on the OV-COCO [...

    2012

  88. [96]

    Related Work 10.1. Open-Vocabulary Dense Prediction Open-vocabulary dense prediction aims to detect and seg- ment visual concepts from novel categories using tex- tual descriptions, extending beyond the base categories on which the model was trained. According to recent sur- v...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.