Pith. sign in

REVIEW 2 major objections 5 minor 65 references

CLIPix turns CLIP's own classification activations into clean pixel masks for any named object, without support images or closed-set training.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 07:29 UTC pith:S6AYIIB2

load-bearing objection Clean engineering win: Grad-CAM-style CLIP activations + two simple modules beat affinity and contrastive baselines on standard open-set binary segmentation, with solid ablations and edge-friendly numbers. the 2 major comments →

arxiv 2607.05253 v2 pith:S6AYIIB2 submitted 2026-07-06 cs.CV

Repurposing CLIP to Localize at Pixel Level

Jiaxiang Fang , Shiqiang Ma , Jing Wang , Siyu Chen , Fei Guo , Shengfeng He This is my paper
classification cs.CV
keywords CLIPpixel-level localizationbinary open-set semantic segmentationnoise-resistant correctionlocalization embeddingzero-shot segmentationvision-language models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

CLIP is already good at deciding whether an image contains a named object, but its features are global and noisy when used for pixel-by-pixel masks. CLIPix shows that the regions CLIP attends to while computing that decision can be extracted as initial localization maps, cleaned of global bias by a Noise-Resistant Correction step that reweights patch features and the class token, then expanded into full-body detail by a Localization Embedding module that fuses those maps with image features. The resulting system needs only a class-name text prompt, keeps CLIP frozen so open-set generalization is preserved, and reports state-of-the-art zero-shot binary open-set scores on the standard PASCAL-5i and COCO-20i benchmarks. A reader cares because it converts an existing, widely deployed image-level model into a practical open-vocabulary segmenter without the usual overhead of mask proposals, support sets, or heavy fine-tuning.

Core claim

The paper establishes that tracing the gradient-weighted response of CLIP's key visual-encoder layers during classification yields object-specific attentive regions that, after Noise-Resistant Correction of the input embeddings and class token and Localization Embedding into the decoder features, become sufficiently accurate pixel-level cues for binary open-set segmentation of arbitrary categories.

What carries the argument

Noise-Resistant Correction (NRC): the initial activation map L1 reweights the patch embeddings E and constructs a pseudo-target prototype that updates the class token C; the encoder is re-run to produce a cleaned map L2. Localization Embedding then expands L1/L2 into full-body features via cascaded activation and cross-attention.

Load-bearing premise

The assumption that CLIP's final-layer classification activations already highlight enough of the true object (even for unseen classes) that a simple reweighting of patches and class token can remove the remaining global noise and yield usable full-body masks.

What would settle it

On a held-out open-set split, replace the gradient-derived L1 maps with pure noise or with affinity-matrix baselines while keeping NRC and Localization Embedding fixed; if mIoU collapses to the level of the uncorrected affinity baseline, the claim that classification activations are the decisive localization source is falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper introduces CLIPix, a framework that repurposes a frozen CLIP (ViT-B/16) for binary open-set semantic segmentation. It extracts class-specific response activation maps L1 from CLIP’s visual-language classification score S via a Grad-CAM-style formula (Eq. 2) on key layers, then applies Noise-Resistant Correction (NRC) that reweights patch embeddings E and mixes the class token C with a pooled prototype (Eqs. 3–4) to produce a refined map L2. A Localization Embedding (LE) strategy further constructs prototypes, performs cascaded activation of image features (from an auxiliary CNN encoder), and uses cross-attention (Eqs. 5–9) to decode full-body masks from a class-name text prompt alone. Extensive experiments claim state-of-the-art zero-shot mIoU/FBIoU on PASCAL-5i (80.7/88.4) and COCO-20i (61.8/78.8), with ablations, multi-class training-free extension, efficiency numbers, and per-class results supporting the components.

Significance. If the reported gains hold under the stated zero-shot protocol, CLIPix is a practically useful contribution: it converts CLIP’s image-level open-set strength into competitive pixel-level localization without support images, heavy prompt engineering, or SAM-scale models, while remaining lightweight enough for MobileNet/EfficientNet backbones and edge settings. The clean isolation of NRC (+10 mIoU) and LE (+7.7 mIoU) in Table IV, the visual denoising in Fig. 5, the training-free multi-class lift in Table VI, and the public code are concrete strengths that make the work reproducible and extensible. The result sits usefully between pure CLIP affinity methods and foundation-model few-shot pipelines.

major comments (2)
  1. §III.C and Implementation Details: the LE pathway relies on an auxiliary pretrained CNN (ResNet-50 / MobileNet / EfficientNet) whose features F_vis are activated by the CLIP-derived maps; the central claim of “repurposing CLIP” therefore depends on a hybrid architecture whose contribution is not fully isolated from the CLIP localization cues. An ablation that replaces the CNN features with CLIP’s own intermediate maps (or freezes the CNN) would clarify how much of the SOTA margin is truly attributable to the NRC/LE repurposing versus the extra encoder capacity.
  2. Tables I–II and §IV.C: several zero-shot baselines (e.g., SAZS, LSeg, PMGA-Net) are compared against few-shot methods that receive support masks; while the paper correctly flags the distinction, the narrative repeatedly frames CLIPix as “surpassing” the few-shot numbers. Because the protocols differ, the strongest claim that can be made is superiority among zero-shot methods; the cross-setting ranking should be presented more cautiously or moved to a separate “oracle” column.
minor comments (5)
  1. Eq. (2) and surrounding text: the Grad-CAM-style summation is standard; a short citation to the original Grad-CAM paper (or DenseCLIP’s related use) would help readers locate the technique.
  2. Fig. 2 and §III.B: the diagram shows “Repeating Step (b)” after the NRC update, but the text does not specify whether the second forward pass freezes all CLIP weights or allows any gradient flow; a one-sentence clarification would remove ambiguity.
  3. Tables I–II, IV, VIII–IX: no standard deviations or multiple-run statistics are reported; even a single-run note would strengthen confidence in the 1–2 point margins over the next-best zero-shot entries.
  4. §IV.H and Fig. 8: the discussion of failure modes on “instrumental” objects is insightful; adding a quantitative breakdown (e.g., mean mIoU on natural vs. man-made classes) would make the observation more actionable for future work.
  5. Minor typographical issues: “repurposes” vs. “repurpose” inconsistency in the abstract/intro, “Pottedplant” spacing in Fig. 1, and occasional missing spaces around math operators (e.g., α1,α2).

Circularity Check

0 steps flagged

No circularity: empirical method with held-out evaluation; localization maps and modules do not reduce claims to inputs by construction.

full rationale

CLIPix extracts L1 via the Grad-CAM-style formula (Eq. 2) from frozen CLIP classification logits, refines to L2 by reweighting patch embeddings E and class token C (Eqs. 3-4), then embeds via LE (Eqs. 5-9) into a lightweight decoder trained by cross-entropy. All performance claims (Tables I-III, VI-IX) are measured mIoU/FBIoU on held-out folds of PASCAL-5i/COCO-20i against external baselines; ablations (Tables IV-V, Figs. 5-7) isolate module contributions without fitting a parameter that is then re-reported as a prediction. No equation equates a claimed result to its own fitted input; CLIP is an independent pretrained model; self-citations (e.g., PMGA-Net) are only for comparison, not load-bearing uniqueness or ansatz. The derivation chain is therefore self-contained and non-circular.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 0 invented entities

The central empirical claim rests on CLIP's pretrained behavior plus a handful of hand-chosen scalars and architectural choices; no new physical entities are postulated. The free parameters and domain assumptions listed below are exactly those required for the reported numbers to be obtained.

free parameters (4)
  • α1, α2 (class-token mixing weights) = 0.5 / 0.5
    Fixed at 0.5/0.5 after a one-dimensional sweep (Fig. 7); the peak performance depends on this balance.
  • key CLIP layer for activation extraction = final layer
    Chosen as the final layer after ablation (Fig. 6); earlier layers degrade mIoU substantially.
  • learning rate / epochs / batch size = 1e-4 / 200|50 / 16
    AdamW 1e-4, 200 epochs (PASCAL) / 50 epochs (COCO), batch 16; standard but still free choices that affect the reported SOTA.
  • high-response threshold (multi-class extension)
    Used to filter localization maps before attention-map rewriting; value not reported, yet performance gains depend on it.
axioms (3)
  • domain assumption CLIP's final-layer classification activations (Eq. 2) encode object-specific spatial regions that remain informative for unseen classes after simple reweighting.
    Stated as the core motivation in §III.B; without it the entire localization pipeline collapses.
  • ad hoc to paper Global feature bias of CLIP can be sufficiently mitigated by Hadamard reweighting of patch embeddings and a linear mix of the original class token with a pooled prototype (Eqs. 3-4).
    Introduced without theoretical guarantee; validated only by the ablation in Table V.
  • domain assumption Training and test class sets are strictly disjoint and the evaluation protocol of PASCAL-5i / COCO-20i is a faithful open-set test.
    Standard few-shot segmentation fold construction adopted in §IV.A.

pith-pipeline@v1.1.0-grok45 · 26895 in / 2615 out tokens · 29145 ms · 2026-07-11T07:29:34.951352+00:00 · methodology

0 comments
read the original abstract

Large-scale Vision-Language Models like CLIP have demonstrated impressive open-set localization capabilities at the image level. However, adapting this capability to pixel-level dense prediction poses challenges due to global feature biases. In this paper, we introduce CLIPix, a simple yet effective framework that repurposes CLIP to perform pixel-level localization. By tracing back CLIP's classification process, CLIPix identifies object-specific attentive regions and repurposes them as pixel-level localization cues. To address noise introduced by global biases, we propose a Noise-Resistant Correction strategy, refining these cues for more precise segmentation. Additionally, we introduce a Localization Embedding strategy to integrate both localization and enriched detail information, enabling accurate, high-resolution segmentation. Our approach preserves CLIP's generalization strength and unlocks its potential for segmenting arbitrary objects. Extensive experiments on the PASCAL and COCO datasets demonstrate that CLIPix achieves state-of-the-art performance, underscoring its effectiveness.

Figures

Figures reproduced from arXiv: 2607.05253 by Fei Guo, Jiaxiang Fang, Jing Wang, Shengfeng He, Shiqiang Ma, Siyu Chen.

Figure 1
Figure 1. Figure 1: The overview of our CLIPix framework. CLIPix repurposes CLIP for precise pixel-level localization while preserving its inherent generalization [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The overview of our CLIPix framework (a). CLIPix repurposes CLIP for accurate pixel-level localization while preserving its natural generalization, [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative results of the proposed method and baseline approach on PASCAL-5 [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of foundation model application modes. (a) Segmentation [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Ablation studies of key layer selection on performance impact. The [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Ablation studies on the impact of different parameter settings for [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Visualized analysis of failure cases in complex scenarios. From left [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

65 extracted references · 8 linked inside Pith

  1. [1]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo,et al., “Segment anything,” inProceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4015–4026, 2023. 1, 5, 6, 7

    2023

  2. [2]

    Exploiting efficientsam and temporal coher- ence for audio-visual segmentation,

    Y . Zhu, K. Li, and Z. Yang, “Exploiting efficientsam and temporal coher- ence for audio-visual segmentation,”IEEE Transactions on Multimedia,

  3. [3]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark,et al., “Learning transferable visual models from natural language supervision,” inInternational conference on machine learning, pp. 8748–8763, PMLR, 2021. 1, 2, 3, 10

    2021

  4. [4]

    Visual and textual prior guided mask assemble for few-shot segmentation and beyond,

    S. Chen, F. Meng, R. Zhang, H. Qiu, H. Li, Q. Wu, and L. Xu, “Visual and textual prior guided mask assemble for few-shot segmentation and beyond,”IEEE Transactions on Multimedia, 2024. 1, 2, 3, 5, 6, 7

    2024

  5. [5]

    Matcher: Segment anything with one shot using all-purpose feature matching,

    Y . Liu, M. Zhu, H. Li, H. Chen, X. Wang, and C. Shen, “Matcher: Segment anything with one shot using all-purpose feature matching,” arXiv preprint arXiv:2305.13310, 2024. 1, 3, 7, 8

    Pith/arXiv arXiv 2024

  6. [6]

    Personalize segment anything model with one shot,

    R. Zhang, Z. Jiang, Z. Guo, S. Yan, J. Pan, X. Ma, H. Dong, P. Gao, and H. Li, “Personalize segment anything model with one shot,”arXiv preprint arXiv:2305.03048, 2023. 1, 3, 7, 8

    Pith/arXiv arXiv 2023

  7. [7]

    Vrp-sam: Sam with visual reference prompt,

    Y . Sun, J. Chen, S. Zhang, X. Zhang, Q. Chen, G. Zhang, E. Ding, J. Wang, and Z. Li, “Vrp-sam: Sam with visual reference prompt,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23565–23574, 2024. 1, 2, 3, 7, 8

    2024

  8. [8]

    Denseclip: Language-guided dense prediction with context-aware prompting,

    Y . Rao, W. Zhao, G. Chen, Y . Tang, Z. Zhu, G. Huang, J. Zhou, and J. Lu, “Denseclip: Language-guided dense prediction with context-aware prompting,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18082–18091, 2022. 1, 3, 8

    2022

  9. [9]

    Spikeclip: A contrastive language-image pretrained spiking neural network,

    T. Li, W. Liu, C. Lv, Y . Gu, J. Xu, C. Zhang, M. Wu, X. Zheng, and X. Huang, “Spikeclip: A contrastive language-image pretrained spiking neural network,”arXiv preprint arXiv:2310.06488, 2023. 1

    Pith/arXiv arXiv 2023

  10. [10]

    Spklip: Aligning spike video streams with natural language,

    Y . Gao, M. Jin, Z. Yu, T. Huang, and G. Chen, “Spklip: Aligning spike video streams with natural language,”arXiv preprint arXiv:2505.12656,

    arXiv

  11. [11]

    Self-supervised high-order information bottleneck learning of spiking neural network for robust event-based optical flow estimation,

    S. Yang, B. Linares-Barranco, Y . Wu, and B. Chen, “Self-supervised high-order information bottleneck learning of spiking neural network for robust event-based optical flow estimation,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 1

    2024

  12. [12]

    Language-driven semantic segmentation,

    B. Li, K. Q. Weinberger, S. Belongie, V . Koltun, and R. Ranftl, “Language-driven semantic segmentation,”arXiv preprint arXiv:2201.03546, 2022. 1, 3, 5, 6, 7, 10 IEEE TRANSACTIONS ON MULTIMEDIA 13

    Pith/arXiv arXiv 2022

  13. [13]

    Delving into shape-aware zero-shot semantic segmentation,

    X. Liu, B. Tian, Z. Wang, R. Wang, K. Sheng, B. Zhang, H. Zhao, and G. Zhou, “Delving into shape-aware zero-shot semantic segmentation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2999–3009, June 2023. 1, 3, 5, 6, 7, 10

    2023

  14. [14]

    Decoupling zero-shot semantic segmentation,

    J. Ding, N. Xue, G.-S. Xia, and D. Dai, “Decoupling zero-shot semantic segmentation,” inProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 11583–11592, 2022. 1

    2022

  15. [15]

    Vision-language models for vision tasks: A survey,

    J. Zhang, J. Huang, S. Jin, and S. Lu, “Vision-language models for vision tasks: A survey,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 2

    2024

  16. [16]

    Vqgan-clip: Open domain image generation and editing with natural language guidance,

    K. Crowson, S. Biderman, D. Kornis, D. Stander, E. Hallahan, L. Cas- tricato, and E. Raff, “Vqgan-clip: Open domain image generation and editing with natural language guidance,” inEuropean Conference on Computer Vision, pp. 88–105, Springer, 2022. 2

    2022

  17. [17]

    Clip-forge: Towards zero-shot text-to-shape generation,

    A. Sanghi, H. Chu, J. G. Lambourne, Y . Wang, C.-Y . Cheng, M. Fumero, and K. R. Malekshan, “Clip-forge: Towards zero-shot text-to-shape generation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18603–18613, 2022. 2

    2022

  18. [18]

    Clip-driven semantic discovery network for visible-infrared person re-identification,

    X. Yu, N. Dong, L. Zhu, H. Peng, and D. Tao, “Clip-driven semantic discovery network for visible-infrared person re-identification,”IEEE Transactions on Multimedia, 2025. 2

    2025

  19. [19]

    Clip-based modality compensation for visible-infrared image re-identification,

    G. Hu, Y . Lv, J. Zhang, Q. Wu, and Z. Wen, “Clip-based modality compensation for visible-infrared image re-identification,”IEEE Trans- actions on Multimedia, 2024. 2

    2024

  20. [20]

    Maple: Multi-modal prompt learning,

    M. U. Khattak, H. Rasheed, M. Maaz, S. Khan, and F. S. Khan, “Maple: Multi-modal prompt learning,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19113– 19122, 2023. 2

    2023

  21. [21]

    Clip-kd: An empirical study of clip model distillation,

    C. Yang, Z. An, L. Huang, J. Bi, X. Yu, H. Yang, B. Diao, and Y . Xu, “Clip-kd: An empirical study of clip model distillation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15952–15962, 2024. 2

    2024

  22. [22]

    Learning to prompt for vision- language models,

    K. Zhou, J. Yang, C. C. Loy, and Z. Liu, “Learning to prompt for vision- language models,”International Journal of Computer Vision, vol. 130, no. 9, pp. 2337–2348, 2022. 2

    2022

  23. [23]

    Conditional prompt learning for vision-language models,

    K. Zhou, J. Yang, C. C. Loy, and Z. Liu, “Conditional prompt learning for vision-language models,” inProceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 16816–16825,

  24. [24]

    Zero-shot semantic segmentation,

    M. Bucher, T.-H. Vu, M. Cord, and P. P ´erez, “Zero-shot semantic segmentation,”Advances in Neural Information Processing Systems, vol. 32, 2019. 2, 6, 7

    2019

  25. [25]

    Dual-guided fre- quency prototype network for few-shot semantic segmentation,

    C. Wen, H. Huang, Y . Ma, F. Yuan, and H. Zhu, “Dual-guided fre- quency prototype network for few-shot semantic segmentation,”IEEE Transactions on Multimedia, vol. 26, pp. 8874–8888, 2024. 2

    2024

  26. [26]

    Semantic projection network for zero-and few-label semantic segmentation,

    Y . Xian, S. Choudhury, Y . He, B. Schiele, and Z. Akata, “Semantic projection network for zero-and few-label semantic segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8256–8265, 2019. 2

    2019

  27. [27]

    Towards open-vocabulary semantic segmentation without semantic la- bels,

    H. Shin, C. Kim, S. Hong, S. Cho, A. Arnab, P. H. Seo, and S. Kim, “Towards open-vocabulary semantic segmentation without semantic la- bels,”arXiv preprint arXiv:2409.19846, 2024. 3

    Pith/arXiv arXiv 2024

  28. [28]

    Hypercorrelation squeeze for few-shot segmentation,

    J. Min, D. Kang, and M. Cho, “Hypercorrelation squeeze for few-shot segmentation,” inProceedings of the IEEE/CVF international conference on computer vision, pp. 6941–6952, 2021. 6, 7

    2021

  29. [29]

    Self-support few-shot semantic segmentation,

    Q. Fan, W. Pei, Y .-W. Tai, and C.-K. Tang, “Self-support few-shot semantic segmentation,” inEuropean Conference on Computer Vision, pp. 701–719, Springer, 2022. 6

    2022

  30. [30]

    Learning what not to segment: A new perspective on few-shot segmentation,

    C. Lang, G. Cheng, B. Tu, and J. Han, “Learning what not to segment: A new perspective on few-shot segmentation,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8057–8067, 2022. 6

    2022

  31. [31]

    Mianet: Aggregating unbiased instance and general information for few-shot semantic seg- mentation,

    Y . Yang, Q. Chen, Y . Feng, and T. Huang, “Mianet: Aggregating unbiased instance and general information for few-shot semantic seg- mentation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7131–7140, 2023. 6, 7

    2023

  32. [32]

    Hierarchical dense correlation distillation for few-shot segmentation,

    B. Peng, Z. Tian, X. Wu, C. Wang, S. Liu, J. Su, and J. Jia, “Hierarchical dense correlation distillation for few-shot segmentation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, pp. 23641–23651, 2023. 6, 7

    2023

  33. [33]

    Hybrid mamba for few-shot segmentation,

    Q. Xu, X. Liu, L. Zhu, G. Lin, C. Long, Z. Li, and R. Zhao, “Hybrid mamba for few-shot segmentation,”Advances in Neural Information Processing Systems, vol. 37, pp. 73858–73883, 2024. 6, 7

    2024

  34. [34]

    Eliminating feature ambiguity for few-shot segmentation,

    Q. Xu, G. Lin, C. C. Loy, C. Long, Z. Li, and R. Zhao, “Eliminating feature ambiguity for few-shot segmentation,” inEuropean Conference on Computer Vision, pp. 416–433, Springer, 2024. 6, 7

    2024

  35. [35]

    Addressing background context bias in few-shot segmentation through iterative modulation,

    L. Zhu, T. Chen, J. Yin, S. See, and J. Liu, “Addressing background context bias in few-shot segmentation through iterative modulation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3370–3379, 2024. 6

    2024

  36. [36]

    Rethinking prior information generation with clip for few-shot segmentation,

    J. Wang, B. Zhang, J. Pang, H. Chen, and W. Liu, “Rethinking prior information generation with clip for few-shot segmentation,” inPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3941–3951, 2024. 6

    2024

  37. [37]

    Llafs++: Few-shot image segmentation with large language models,

    L. Zhu, T. Chen, D. Ji, P. Xu, J. Ye, and J. Liu, “Llafs++: Few-shot image segmentation with large language models,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 47, no. 9, pp. 7715– 7732, 2025. 6

    2025

  38. [38]

    Dsv-lfs: Unifying llm-driven semantic cues with visual features for robust few-shot segmentation,

    A. Karimi and C. Poullis, “Dsv-lfs: Unifying llm-driven semantic cues with visual features for robust few-shot segmentation,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4584–4594, June 2025. 6

    2025

  39. [39]

    Prior guided feature enrichment network for few-shot segmentation,

    Z. Tian, H. Zhao, M. Shu, Z. Yang, R. Li, and J. Jia, “Prior guided feature enrichment network for few-shot segmentation,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 02, pp. 1050– 1065, 2022. 6

    2022

  40. [40]

    Holistic prototype activation for few- shot segmentation,

    G. Cheng, C. Lang, and J. Han, “Holistic prototype activation for few- shot segmentation,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 4, pp. 4650–4666, 2022. 6

    2022

  41. [41]

    Base and meta: A new perspective on few-shot segmentation,

    C. Lang, G. Cheng, B. Tu, C. Li, and J. Han, “Base and meta: A new perspective on few-shot segmentation,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. 6

    2023

  42. [42]

    Image segmentation using text and image prompts,

    T. L ¨uddecke and A. Ecker, “Image segmentation using text and image prompts,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7086–7096, 2022. 6, 10

    2022

  43. [43]

    Uniboost: Un- supervised unimodal pre-training for boosting zero-shot vision-language tasks,

    Y . Sun, Z. Zhong, Q. Fan, C.-K. Tang, and Y .-W. Tai, “Uniboost: Un- supervised unimodal pre-training for boosting zero-shot vision-language tasks,”arXiv preprint arXiv:2306.04715, 2023. 6

    Pith/arXiv arXiv 2023

  44. [44]

    Prompt-and-transfer: Dynamic class-aware enhancement for few-shot segmentation,

    H. Bi, Y . Feng, W. Diao, P. Wang, Y . Mao, K. Fu, H. Wang, and X. Sun, “Prompt-and-transfer: Dynamic class-aware enhancement for few-shot segmentation,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 6, 7

    2024

  45. [45]

    Doubly deformable aggregation of covariance matrices for few-shot segmentation,

    Z. Xiong, H. Li, and X. X. Zhu, “Doubly deformable aggregation of covariance matrices for few-shot segmentation,” inEuropean Conference on Computer Vision, pp. 133–150, Springer, 2022. 7

    2022

  46. [46]

    Semantic projection network for zero-and few-label semantic segmentation,

    Y . Xian, S. Choudhury, Y . He, B. Schiele, and Z. Akata, “Semantic projection network for zero-and few-label semantic segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8256–8265, 2019. 7

    2019

  47. [47]

    One-shot learning for semantic segmentation,

    A. Shaban, S. Bansal, Z. Liu, I. Essa, and B. Boots, “One-shot learning for semantic segmentation,”arXiv preprint arXiv:1709.03410, 2017. 5

    Pith/arXiv arXiv 2017

  48. [48]

    Feature weighting and boosting for few-shot segmentation,

    K. Nguyen and S. Todorovic, “Feature weighting and boosting for few-shot segmentation,” inProceedings of the IEEE/CVF International Conference on Computer Vision, pp. 622–631, 2019. 5

    2019

  49. [49]

    Mobilenetv2: Inverted residuals and linear bottlenecks,

    M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520, 2018. 5, 6, 10

    2018

  50. [50]

    Efficientnet: Rethinking model scaling for con- volutional neural networks,

    M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for con- volutional neural networks,” inInternational Conference on Machine Learning, pp. 6105–6114, PMLR, 2019. 5, 6, 10

    2019

  51. [51]

    Improved baselines with visual instruction tuning,

    H. Liu, C. Li, Y . Li, and Y . J. Lee, “Improved baselines with visual instruction tuning,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 26296–26306, June 2024. 6

    2024

  52. [52]

    BLIP-2: bootstrapping language- image pre-training with frozen image encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. Hoi, “BLIP-2: bootstrapping language- image pre-training with frozen image encoders and large language models,” inICML, 2023. 6

    2023

  53. [53]

    Images speak in images: A generalist painter for in-context visual learning,

    X. Wang, W. Wang, Y . Cao, C. Shen, and T. Huang, “Images speak in images: A generalist painter for in-context visual learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6830–6839, 2023. 7, 8

    2023

  54. [54]

    Seggpt: Towards segmenting everything in context,

    X. Wang, X. Zhang, Y . Cao, W. Wang, C. Shen, and T. Huang, “Seggpt: Towards segmenting everything in context,” inProceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1130– 1140, 2023. 7, 8

    2023

  55. [55]

    Llafs: When large language models meet few-shot segmentation,

    L. Zhu, T. Chen, D. Ji, J. Ye, and J. Liu, “Llafs: When large language models meet few-shot segmentation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3065– 3075, 2024. 8

    2024

  56. [56]

    Foreground- covering prototype generation and matching for sam-aided few-shot segmentation,

    S. Park, S. Lee, H. S. Seong, J. Yoo, and J.-P. Heo, “Foreground- covering prototype generation and matching for sam-aided few-shot segmentation,”arXiv preprint arXiv:2501.00752, 2025. 8 IEEE TRANSACTIONS ON MULTIMEDIA 14

    Pith/arXiv arXiv 2025

  57. [57]

    Sclip: Rethinking self-attention for dense vision-language inference,

    F. Wang, J. Mei, and A. Yuille, “Sclip: Rethinking self-attention for dense vision-language inference,” inEuropean Conference on Computer Vision, pp. 315–332, Springer, 2024. 10

    2024

  58. [58]

    The role of context for object detection and semantic segmentation in the wild,

    R. Mottaghi, X. Chen, X. Liu, N.-G. Cho, S.-W. Lee, S. Fidler, R. Urtasun, and A. Yuille, “The role of context for object detection and semantic segmentation in the wild,” inProceedings of the IEEE conference on computer vision and pattern recognition, pp. 891–898,

  59. [59]

    The cityscapes dataset for semantic urban scene understanding,

    M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Be- nenson, U. Franke, S. Roth, and B. Schiele, “The cityscapes dataset for semantic urban scene understanding,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3213– 3223, 2016. 10

    2016

  60. [60]

    Semantic understanding of scenes through the ade20k dataset,

    B. Zhou, H. Zhao, X. Puig, T. Xiao, S. Fidler, A. Barriuso, and A. Torralba, “Semantic understanding of scenes through the ade20k dataset,”International Journal of Computer Vision, vol. 127, no. 3, pp. 302–321, 2019. 10

    2019

  61. [61]

    Groupvit: Semantic segmentation emerges from text supervision,

    J. Xu, S. De Mello, S. Liu, W. Byeon, T. Breuel, J. Kautz, and X. Wang, “Groupvit: Semantic segmentation emerges from text supervision,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18134–18144, 2022. 10

    2022

  62. [62]

    Clip-dinoiser: Teaching clip a few dino tricks for open-vocabulary semantic segmentation,

    M. Wysocza ´nska, O. Sim ´eoni, M. Ramamonjisoa, A. Bursuc, T. Trzci´nski, and P. P´erez, “Clip-dinoiser: Teaching clip a few dino tricks for open-vocabulary semantic segmentation,” inEuropean Conference on Computer Vision, pp. 320–337, Springer, 2024. 10

    2024

  63. [63]

    Learning to generate text-grounded mask for open-world semantic segmentation from only image-text pairs,

    J. Cha, J. Mun, and B. Roh, “Learning to generate text-grounded mask for open-world semantic segmentation from only image-text pairs,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11165–11174, 2023. 10

    2023

  64. [64]

    Clearclip: Decomposing clip representations for dense vision-language inference,

    M. Lan, C. Chen, Y . Ke, X. Wang, L. Feng, and W. Zhang, “Clearclip: Decomposing clip representations for dense vision-language inference,” inEuropean Conference on Computer Vision, pp. 143–160, Springer,

  65. [65]

    Resclip: Residual attention for training-free dense vision-language inference,

    Y . Yang, J. Deng, W. Li, and L. Duan, “Resclip: Residual attention for training-free dense vision-language inference,” inProceedings of the Computer Vision and Pattern Recognition Conference, pp. 29968–29978,