Pith. sign in

REVIEW 4 major objections 6 minor 66 references

Toward Modality Gap: Vision Prototype Learning for Weakly-supervised Semantic Segmentation with CLIP

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that CLIP's text prototypes cannot optimally localize visual objects because of an inherent modality gap, and that learning class-specific vision prototypes under text supervision plus regional semantic contrast produces…

desk verdict Solid empirical WSSS paper with real plug-in gains, but the theory section is under-developed and should be fixed or cut before publication. read the letter →

arxiv 2412.19650 v1 pith:ERVMKQG5 submitted 2024-12-27 cs.CV cs.LG

Zhongxing Xu , Feilong Tang , Zhe Chen , Yingxue Su , Zhiyi Zhao , Ge Zhang , Jionglong Su , Zongyuan Ge This is my paper · ORCID
classification cs.CVcs.LG
keywords weaklysupervisedsemanticsegmentationCLIPmodalitygapvisionprototypelearningregionalcontrastpseudo-maskgenerationPASCALVOCMSCOCO
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CLIP-based weakly supervised semantic segmentation has relied on optimizing text prompts to obtain text prototypes that activate object regions, but this paper claims that the text and vision spaces of CLIP are separated by an inherent modality gap that contrastive loss cannot close. The authors try to establish that the optimal category prototypes for pixel-level localization must therefore live in the vision space, and they propose Vision Prototype Learning (VPL): learn class-specific vision prototypes by gradient descent under a KL-divergence constraint with text prototypes, then use those vision prototypes to generate pseudo-masks and supervise a decoder. A regional semantic contrast loss aligns masked-average-pooled region embeddings with the vision prototypes. If correct, this shifts CLIP-based WSSS from prompt engineering to vision-space prototype learning and yields state-of-the-art pseudo-mask and segmentation results on PASCAL VOC 2012 and MS COCO 2014.

What carries the argument

The central object is the set of learnable vision prototypes W = {w_n} in the CLIP vision embedding space, replacing text prototypes Z as the source of class activation. Theorems 2 and 3 argue that a strongly convex KL objective in W can be minimized by gradient descent, with Eq. 10 bounding the gradient update and Eq. 11 iterating it, and the choice of temperature tau_I larger than CLIP's tau_T (0.03 vs 0.01) follows from Proposition 1 and is confirmed by the gap metric in Table 5. The second mechanism is regional semantic contrast, which pools region embeddings by masked average pooling (Eq. 12) and applies a bidirectional contrastive loss (Eq. 13) so that region embeddings and vision prototypes are pulled together and pushed away from negatives. Together, these turn text-derived activation maps into a seed for learning vision-space representations rather than an end product.

What would settle it

On PASCAL VOC 2012 val, compare VPL against a text-only baseline with identical ensemble prompts, CRF post-processing, and decoder training, but with text prototypes projected into the vision space through a linear map learned on a small labeled split. If the projected-text baseline reaches or exceeds VPL's 78.5% mIoU, the claim that optimal prototypes must be learned within the vision space would be falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that the modality gap between CLIP text and vision embeddings is inherent: Theorem 1 bounds the distance between an optimal vision prototype W* and a text prototype Z from below by a constant plus alignment noise, so minimizing contrastive loss cannot eliminate it, and the best prototypes for vision tasks are therefore not text prototypes but vision prototypes. The method learns W' by minimizing KL divergence between the distribution induced by text prototypes and the distribution induced by learnable vision prototypes (Eq. 7), updates it by projected gradient descent using Eq. 10-11, and uses the resulting W' to generate GradCAMs and pseudo-masks. A regional semantic contrast loss (Eq. 13) then contrasts region embeddings with these vision prototypes during decoder training. Empirically, plugging VPL into CLIP-ES and CLIP-CPAL raises seed mIoU by 5.5 and 5.9 points and final segmentation mIoU to 78.5% and 79.3% on VOC val, 77.8% and 79.0% on VOC test, and 49.2% and 49.8% on COCO val, which the paper reports as state-of-the-art.

Load-bearing premise

The load-bearing premise is that the GradCAM pseudo-masks produced by text prototypes, despite activating co-occurring or background regions, carry enough correct localization signal that learning vision prototypes under KL supervision yields genuinely better prototypes than the text prototypes themselves.

Editorial extensions

If this is right

  • VPL can be added to existing CLIP-based WSSS pipelines without changing their architecture, improving seed and pseudo-mask quality by several IoU points.
  • Text-side prompt engineering in CLIP-based WSSS has a ceiling set by the modality gap, so further gains should come from vision-space adaptation rather than better prompts.
  • The temperature used to learn vision prototypes should be larger than CLIP's text temperature, matching the theoretical relationship tau_T = sqrt(epsilon) tau_I.
  • Regional semantic contrast with vision prototypes yields more compact and separable feature clusters, which leads to better final segmentation under noisy pseudo-labels.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the modality gap is truly irreducible by contrastive loss, then similar vision-prototype learning could benefit other CLIP-based dense prediction tasks, such as open-vocabulary segmentation and referring expression segmentation, where text prototypes are currently the default.
  • Because VPL's vision prototypes are anchored to text-derived pseudo-masks, the method inherits any blind spots of text GradCAM; a natural extension is to break this anchor with multiple prompt sources or self-training iterations that let vision prototypes escape the text distribution.
  • A cheap comparison, applying a learned linear map from text prototypes into the vision embedding space, would test whether VPL's gains come from crossing modalities or from optimization within the vision space.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Vision Prototype Learning (VPL), a weakly-supervised semantic segmentation framework built on CLIP. The authors argue that text prototypes suffer from an inherent modality gap and therefore cannot optimally localize objects; they propose learning class-specific vision prototypes in vision space, supervised by text-prototype distributions via a KL-divergence objective. Vision prototypes replace text prototypes for GradCAM-based pseudo-label generation, and a regional semantic contrast module aligns region embeddings with vision prototypes during decoder training. Experiments on PASCAL VOC 2012 and MS COCO 2014 report state-of-the-art or improved mIoU over strong baselines such as CLIP-ES and CLIP-CPAL, with ablations showing gains from the vision prototypes, the contrast module, and the temperature setting.

Significance. If the theoretical and empirical claims hold, the paper would make a useful contribution: it identifies a real practical issue, the text-vision modality gap in CLIP-based WSSS, and it offers a simple plug-and-play module that improves several existing pipelines on held-out VOC and COCO benchmarks. The empirical evaluation is externally benchmarked, the ablations isolate the proposed components, and the temperature prediction tau_I > tau_T is concretely tested and confirmed. However, the theoretical core that motivates the method is not currently verifiable: the proofs are placed in appendices B.1-B.6 that are absent from the submitted version, and the stated theorems contain notational and logical gaps. Because the paper explicitly claims a theoretical demonstration that optimal vision prototypes cannot be achieved in text space, this is a load-bearing weakness rather than a purely cosmetic one.

major comments (4)
  1. [Preliminary; Methodology (Hypothesis 1 through Theorem 3)] The manuscript repeatedly relies on proofs in Appendices B.1, B.3, B.4, B.5, and B.6 (for example, the claim that small temperature does not pull text and vision spaces together, Proposition 1, Theorems 1-3), but none of these appendices appears in the submitted v1. Without those proofs, the theoretical statements cannot be checked, and the central claim that the modality gap is inherent and that optimal vision prototypes cannot be achieved in text space is unsupported as written.
  2. [Methodology, Theorem 1, Eq. (6)] Equation (6) states Delta_gap = ||Z - W*||_F^2 >= c_perp + epsilon, but c_perp is described as a constant vector while epsilon is a scalar and Delta_gap is a scalar Frobenius norm. Adding a vector and a scalar is not defined, and no operational definition is given for c_perp or epsilon. This makes the lower-bound theorem ill-typed and prevents the claimed conclusion that 'minimizing the distance between Z and W is difficult due to the inherent modality gap' from being established.
  3. [Methodology, Proposition 1, Eq. (5)] Proposition 1 assumes z^x_n = w*_n, which is exactly the condition that the vision component of the text prototype equals the optimal vision classifier. The subsequent equality p^{n'}_{i,k} = p^n_{i,k} then follows by construction from the assumed identity and the temperature relation, rather than demonstrating that the text space can or cannot recover optimal vision prototypes. The remark that the assumptions are 'difficult to hold' is an assertion, not a proof, so the paper does not actually establish that optimal vision prototypes cannot be achieved in text space.
  4. [Methodology, Theorem 3, Eqs. (7)-(11)] There is a mismatch between the objective used in the algorithm and the objective used to define the target W*. Equation (9) defines W* as the minimizer of L(m, W), where m is a GradCAM pseudo-mask distribution generated through text prototypes Z, but the algorithm optimizes the KL objective L(P', W) in Eq. (7), where P' is the distribution estimated by the text prototypes. The bound in Eq. (10) involves a term with P' - Y and needs an argument that P' is close to the true labels Y; the paper's own motivation is that text prototypes misactivate co-occurring categories such as railroad and train, so no such closeness is established. Consequently, the theorem does not explain how the learned W' approaches the optimal W*, and the stated theoretical guarantee is unsupported.
minor comments (6)
  1. [Abstract] The opening sentence 'research powerful cross-modal semantic understanding capabilities' is grammatically incomplete; it should be something like 'exhibit powerful cross-modal semantic understanding capabilities'.
  2. [Experiments, Table 4] Table 4 contains a typo: one row is labeled 'Baselinee' and another 'Baselines', and the notation VPLs/VPLe for single versus ensemble prompts is not defined in the caption.
  3. [Experiments, Table 5] The 'Gap' metric used in Table 5 is not formally defined; the text says it measures the difference between prototypes and features, but the exact computation should be stated for reproducibility.
  4. [Experiments, Comparisons With State-of-the-Art Methods] The sentence 'The proposed CLIP-CPAL does not modify the architecture of the CLIP network' appears to refer to VPL, not CLIP-CPAL, since CLIP-CPAL is a prior method; this should be corrected.
  5. [Algorithm 1 and Implementation Details] The implementation section mentions 'standard projected gradient descent' and the algorithm is described as 'convex optimization', but Algorithm 1 line 4 only performs an unconstrained gradient step and includes no projection or bounded-norm operation; this discrepancy should be clarified.
  6. [Methodology, Eq. (10)] Equation (10) uses the notation nabla_{W'*} L(W) to denote ||W'* - W*||_2, which is not a gradient; the notation should be changed to avoid confusion with the actual gradient used in Eq. (11).

Circularity Check

2 steps flagged · score 4.0 of 10

The empirical evaluation is external and non-circular, but the theoretical optimality claim is partly circular: the 'optimal vision prototypes' and their training target are both derived from the very text prototypes the method aims to replace.

  1. self definitional [Methodology, Theorem 3 (Eqs. 9-11) and Eq. (7)]
    "Theorem 3 (Learning Vision Prototypes) . Assume L (P ′, W) is a µ-strongly convex function in W , we have: W ′∗ = arg min W L (P ′, W) ; W ∗ = arg min W L(m, W), (9) where mi is the GradCAM as the pseudo mask distribution for xi, and is generated through text prototypes Z. We compute the Eq. (7) as − P i,n,k P n′ i,k log(P n i,k), and we compute the gradient direction to adjust W ′∗ by the standard gradient descent."

    The 'optimal' vision prototype W* is defined as the minimizer of L(m,W), where the pseudo-mask m is itself the GradCAM generated through text prototypes Z. The learned W'* is obtained by minimizing KL(P'||P), with P' being the distribution estimated by the same text prototypes Z. Consequently, Theorem 3's convergence claim W'* → W* only establishes consistency between two text-prototype-derived quantities. It cannot show that vision prototypes overcome the co-occurrence and incompleteness errors attributed to text prototypes; the superiority of W over Z is not a logical consequence of the stated equations.

  2. fitted input called prediction [Methodology, Eq. (7); Experiments, Table 3 (Ablation II)]
    "Our objective function is constructed by measuring the KL-divergence between the distributions of text prototypes and vision prototypes as: min W L (P ′, W) = X i KL P n′ i ∥P n i , (7) where P n′ i and P n i respectively represent the distributions estimated by the text prototypes Z and the learnable vision prototypes W ."

    The learnable vision prototypes W are fitted to the text-prototype distribution P' as the fixed target. Since the paper's own motivation states that text prototypes 'erroneously activate similar or highly co-occurring categories (e.g., railroad and train)', any localization map produced by W is statistically anchored to the text-prototype predictions through Eq. (7). The ablation step II attributes the +4.9% gain to 'vision prototypes capture fine-grained semantic details better than text prototypes', but the stated objective only enforces fidelity to P'; the gain is not explained by the KL mechanism and must come from auxiliary choices, making the theoretical account of the improvement circular.

full rationale

The benchmark results are not circular: VPL is evaluated on held-out PASCAL VOC and COCO against external methods, and the ablation study isolates the contributions of each component. The circularity is confined to the theoretical framing. In Eq. (7), P' from text prototypes is used as the training target for the vision prototypes, while Theorem 3 defines W* from text-prototype-generated GradCAM masks; hence both the objective and the claimed optimum are anchored to the same text prototypes the paper argues are inadequate. The proof of Theorem 3 (Appendix B.6) is not present, and Eq. (10)'s bound depends on P' - Y without any argument that P' is close to Y; the paper's own railroad/train example suggests the opposite. These issues undermine the theoretical demonstration but do not invalidate the external empirical comparisons. The self-citations to CLIP-CPAL and related work are used as baselines, not as load-bearing authority for the central claim, so they do not raise the score beyond the partial definitional circularity documented above.

Assumptions & free parameters 5 free parameters · 6 assumptions · 2 invented entities

The method contributes a new training scheme but leans on several unverified modeling assumptions: a decomposition of text prototypes, strong convexity of the KL objective, intra-class compactness, and linear representability of classes by single vision prototypes. The empirical benchmark results are external and provide the main independent support.

free parameters (5)
  • Temperature tau_I for vision prototype learning = 0.03 (best on VOC val, Table 5)
    Temperature used in the softmax when learning vision prototypes; chosen by a validation sweep from 0.01 to 0.05 and directly affects the measured modality gap and mIoU.
  • Seed mask threshold Phi = 0.4 (best on VOC val, Table 6)
    Threshold for converting continuous GradCAM outputs to 0-1 pseudo masks; chosen by a sweep from 0.2 to 0.7.
  • Gradient descent iterations T_w = 3000 (best on VOC val, Table 7)
    Number of iterations used to optimize the vision prototype; chosen by a sweep from 1000 to 3500.
  • Phase-1 learning rate alpha = 10
    Learning rate for projected gradient descent on the vision prototype; reported in Implementation Details without an ablation.
  • Loss coefficients lambda_ce, lambda_RSC = 1, 1
    Weights in the final loss Eq. (14); set equal to 1 with no reported sensitivity analysis.
assumptions (6)
  • ad hoc to paper Hypothesis 1: each class prototype z_n decomposes into orthogonal vision-space and residual components z_n = sqrt(epsilon) z^x_n + sqrt(1-epsilon) z^perp_n
    Introduced in Eq. (4) specifically to support the analysis; no independent justification that such a decomposition with unit norms and orthogonality holds for CLIP embeddings.
  • ad hoc to paper Proposition 1 assumes z^x_n = w*_n, i.e., the vision component of the text prototype equals the optimal vision classifier
    Used in Eq. (5) to show equivalence; this is the condition the paper ultimately argues cannot hold, so it is a conditional assumption rather than a proved fact.
  • ad hoc to paper Theorem 3 assumes L(P', W) is mu-strongly convex in W
    No proof or argument that the KL divergence between softmax distributions is strongly convex; the gradient-descent convergence statement rests on this.
  • domain assumption Theorem 2 assumes intra-class compactness, ||x_i - w_{m_i,k}||_2 -> 0
    Used to justify instance-level regularization; compactness of CLIP vision features is asserted rather than measured.
  • domain assumption CLIP's modality gap is inherent and cannot be reduced by minimizing contrastive loss
    Adopted from cited prior work (Liang et al. 2022); the paper's Theorem 1 is meant to quantify this, but the proof is in a missing appendix.
  • domain assumption Frozen CLIP ViT-B/16 features are linearly separable enough for a single per-class vision prototype
    The method models each class with one prototype vector W; this assumes the relevant class structure is roughly linear in CLIP's feature space.
invented entities (2)
  • Class-specific vision prototypes W
    purpose: Replace text prototypes as the basis for GradCAM localization
    The prototypes are internal learned vectors; no external handle is provided beyond the paper's reported benchmark results.
  • Modality-gap decomposition constants c_perp and epsilon
    purpose: Quantify the lower bound on text-vs-vision prototype distance in Theorem 1
    Defined in Eq. (6) but not given an operational measurement procedure; the bound Delta_gap >= c_perp + epsilon mixes a vector and a scalar, so it is not a well-defined falsifiable claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Toward Modality Gap: Vision Prototype Learning for Weakly-supervised Semantic Segmentation with CLIP." pith.science (2026). https://pith.science/paper/ERVMKQG5

@misc{pith2026241219650,
  author       = {Pith},
  title        = {Pith review of: Toward Modality Gap: Vision Prototype Learning for Weakly-supervised Semantic Segmentation with CLIP},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ERVMKQG5}},
  note         = {Machine review of arXiv:2412.19650}
}
read the original abstract

The application of Contrastive Language-Image Pre-training (CLIP) in Weakly Supervised Semantic Segmentation (WSSS) research powerful cross-modal semantic understanding capabilities. Existing methods attempt to optimize input text prompts for improved alignment of images and text, by finely adjusting text prototypes to facilitate semantic matching. Nevertheless, given the modality gap between text and vision spaces, the text prototypes employed by these methods have not effectively established a close correspondence with pixel-level vision features. In this work, our theoretical analysis indicates that the inherent modality gap results in misalignment of text and region features, and that this gap cannot be sufficiently reduced by minimizing contrast loss in CLIP. To mitigate the impact of the modality gap, we propose a Vision Prototype Learning (VPL) framework, by introducing more representative vision prototypes. The core of this framework is to learn class-specific vision prototypes in vision space with the help of text prototypes, for capturing high-quality localization maps. Moreover, we propose a regional semantic contrast module that contrasts regions embedding with corresponding prototypes, leading to more comprehensive and robust feature learning. Experimental results show that our proposed framework achieves state-of-the-art performance on two benchmark datasets.

Figures

Figures reproduced from arXiv: 2412.19650 by the authors.

Figure 1
Figure 1. The main idea proposed in this paper is to reduce [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed Weakly-supervised Vision Prototype Learning (VPL), which consists of two main com [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Qualitative results on Pascal VOC 2012 val set. (a) In￾put images. (b) Results from CLIP-ES. (c) Results by our CLIP￾ES+VPL. (d) Ground truth. Our method produces more accurate responses and as a plug-and-play method. with pseudo-labels generated by our method, compared with related works. Our CLIP-ES+VPL achieves sate-of-the-art (SOTA) results on VOC, with an mIoU of 78.5% on the val set and 77.8% on the test set. … view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Feature embedding visualizations of (a) our framework without RSC, and (b) our framework on the Pascal VOC 2012 val set using t-SNE (Van der Maaten and Hinton 2008). ing the effectiveness of vision prototypes. Analysis of Hyper-parameters. We conduct a hyperparam￾eter …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 48 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Ahn, J.; Cho, S.; and Kwak, S. 2019. Weakly supervised learning of instance segmentation with inter-pixel relations. In CVPR

    work page 2019

  4. [4]

    Chen, L.; Lei, C.; Li, R.; Li, S.; Zhang, Z.; and Zhang, L. 2023. FPR: False Positive Rectification for Weakly Supervised Semantic Segmentation. In ICCV

    work page 2023

  5. [5]

    Chen, Z.; Pan, Y.; Ye, Y.; Lu, M.; and Xia, Y. 2024. Each test image deserves a specific prompt: Continual test-time adaptation for 2d medical image segmentation. In CVPR

    2024

  6. [6]

    Chen, Z.; and Sun, Q. 2023. Extracting Class Activation Maps from Non-Discriminative Features as well. In CVPR

    work page 2023

  7. [7]

    Cheng, Z.; Qiao, P.; Li, K.; Li, S.; Wei, P.; Ji, X.; Yuan, L.; Liu, C.; and Chen, J. 2023. Out-of-candidate rectification for weakly supervised semantic segmentation. In CVPR

    work page 2023

  8. [8]

    Cho, H.; Yoon, S.-H.; Kweon, H.; and Yoon, K.-J. 2024. Finding Meaning in Points: Weakly Supervised Semantic Segmentation for Event Cameras. ECCV

    work page 2024

Show all 66 references
  1. [9]

    K.; Winn, J.; and Zisserman, A

    Everingham, M.; Van Gool, L.; Williams, C. K.; Winn, J.; and Zisserman, A. 2010. The pascal visual object classes (voc) challenge. IJCV

    2010

  2. [10]

    Hariharan, B.; Arbel \'a ez, P.; Bourdev, L.; Maji, S.; and Malik, J. 2011. Semantic contours from inverse detectors. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision, 991--998

    2011

  3. [11]

    Hu, M.; Xia, P.; Wang, L.; Yan, S.; Tang, F.; Xu, Z.; Luo, Y.; Song, K.; Leitner, J.; Cheng, X.; et al. 2025. Ophnet: A large-scale video benchmark for ophthalmic surgical workflow understanding. In ECCV

    2025

  4. [12]

    Hu, M.; Yan, S.; Xia, P.; Tang, F.; Li, W.; Duan, P.; Zhang, L.; and Ge, Z. 2024 a . Diffusion Model Driven Test-Time Image Adaptation for Robust Skin Lesion Classification. arXiv preprint arXiv:2405.11289

    2024 arXiv

  5. [13]

    Hu, M.; Yuan, K.; Shen, Y.; Tang, F.; Xu, X.; Zhou, L.; Li, W.; Chen, Y.; Xu, Z.; Peng, Z.; et al. 2024 b . OphCLIP: Hierarchical Retrieval-Augmented Learning for Ophthalmic Surgical Video-Language Pretraining. arXiv preprint arXiv:2411.15421

    2024 arXiv

  6. [14]

    Hu, X.; Zhang, K.; Xia, L.; Chen, A.; Luo, J.; Sun, Y.; Wang, K.; Qiao, N.; Zeng, X.; Sun, M.; et al. 2024 c . ReCLIP: Refine contrastive language image pre-training with source free domain adaptation. In WACV

    2024

  7. [15]

    a henb \

    Kr \"a henb \"u hl, P.; and Koltun, V. 2011. Efficient inference in fully connected crfs with gaussian edge potentials. NeurIPS

    2011

  8. [16]

    Kweon, H.; Yoon, S.-H.; and Yoon, K.-J. 2023. Weakly Supervised Semantic Segmentation via Adversarial Learning of Classifier and Reconstructor. In CVPR

    2023

  9. [17]

    Lee, J.; Kim, E.; and Yoon, S. 2021. Anti-adversarially manipulated attributions for weakly and semi-supervised semantic segmentation. In CVPR

    2021

  10. [18]

    Lee, J.; Yi, J.; Shin, C.; and Yoon, S. 2021. Bbam: Bounding box attribution map for weakly supervised semantic and instance segmentation. In CVPR

    2021

  11. [19]

    Li, J.; Jie, Z.; Wang, X.; Wei, X.; and Ma, L. 2022. Expansion and shrinkage of localization for weakly-supervised semantic segmentation. NeurIPS

    2022

  12. [20]

    W.; Zhang, Y.; Kwon, Y.; Yeung, S.; and Zou, J

    Liang, V. W.; Zhang, Y.; Kwon, Y.; Yeung, S.; and Zou, J. Y. 2022. Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning. NeurIPS

    2022

  13. [21]

    Lin, D.; Dai, J.; Jia, J.; He, K.; and Sun, J. 2016. Scribblesup: Scribble-supervised convolutional networks for semantic segmentation. In CVPR

    2016

  14. [22]

    Lin, T.-Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Doll \'a r, P.; and Zitnick, C. L. 2014. Microsoft coco: Common objects in context. In ECCV

    2014

  15. [23]

    Lin, Y.; Chen, M.; Wang, W.; Wu, B.; Li, K.; Lin, B.; Liu, H.; and He, X. 2023. Clip is also an efficient segmenter: A text-driven approach for weakly supervised semantic segmentation. In CVPR

    2023

  16. [24]

    Liu, F.; Liu, Y.; Kong, Y.; Xu, K.; Zhang, L.; Yin, B.; Hancke, G.; and Lau, R. 2023. Referring image segmentation using text supervision. In ICCV

    2023

  17. [25]

    Long, J.; Shelhamer, E.; and Darrell, T. 2015. Fully convolutional networks for semantic segmentation. In CVPR

    2015

  18. [26]

    Ouali, Y.; Bulat, A.; Matinez, B.; and Tzimiropoulos, G. 2023. Black box few-shot adaptation for vision-language models. In ICCV

    2023

  19. [27]

    Peng, Z.; Wang, G.; Xie, L.; Jiang, D.; Shen, W.; and Tian, Q. 2023. USAGE: A Unified Seed Area Generation Paradigm for Weakly Supervised Semantic Segmentation. In ICCV

    2023

  20. [28]

    Qian, Q.; Xu, Y.; and Hu, J. 2023. Intra-Modal Proxy Learning for Zero-Shot Visual Categorization with CLIP. NeurIPS

    2023

  21. [29]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In ICML, 8748--8763. PMLR

    2021

  22. [30]

    Rong, S.; Tu, B.; Wang, Z.; and Li, J. 2023. Boundary-Enhanced Co-Training for Weakly Supervised Semantic Segmentation. In CVPR

    2023

  23. [31]

    Ru, L.; Zheng, H.; Zhan, Y.; and Du, B. 2023. Token contrast for weakly-supervised semantic segmentation. In CVPR

    2023

  24. [32]

    Sun, W.; Zhang, Y.; Qin, Z.; Liu, Z.; Cheng, L.; Wang, F.; Zhong, Y.; and Barnes, N. 2023. All-pairs Consistency Learning forWeakly Supervised Semantic Segmentation. In ICCV

    2023

  25. [33]

    Tang, F.; Trinh, M.; Duong, A.; Ly, A.; Stapleton, F.; Chen, Z.; Ge, Z.; and Razzak, I. 2024 a . Discriminating retinal microvascular and neuronal differences related to migraines: Deep Learning based Crossectional Study. arXiv preprint arXiv:2408.07293

    2024 arXiv

  26. [34]

    Tang, F.; Xu, Z.; Huang, Q.; Wang, J.; Hou, X.; Su, J.; and Liu, J. 2023. DuAT: Dual-aggregation transformer network for medical image segmentation. In PRCV

    2023

  27. [35]

    Tang, F.; Xu, Z.; Qu, Z.; Feng, W.; Jiang, X.; and Ge, Z. 2024 b . Hunting Attributes: Context Prototype-Aware Learning for Weakly Supervised Semantic Segmentation. In CVPR

    2024

  28. [36]

    Trinh, M.; Tang, F.; Ly, A.; Duong, A.; Stapleton, F.; Ge, Z.; and Razzak, I. 2024. Sight for sore heads--using cnns to diagnose migraines. Investigative Ophthalmology & Visual Science

    2024

  29. [37]

    Udandarao, V.; Gupta, A.; and Albanie, S. 2023. Sus-x: Training-free name-only transfer of vision-language models. In ICCV

    2023

  30. [38]

    Van der Maaten, L.; and Hinton, G. 2008. Visualizing data using t-SNE. JMLR

    2008

  31. [39]

    Wan, Z.; Zhang, P.; Wang, Y.; Yong, S.; Stepputtis, S.; Sycara, K.; and Xie, Y. 2024. Sigma: Siamese mamba network for multi-modal semantic segmentation. arXiv preprint arXiv:2404.04256

    2024 arXiv

  32. [40]

    Wang, C.; Xu, R.; Xu, S.; Meng, W.; and Zhang, X. 2023. Treating Pseudo-labels Generation as Image Matting for Weakly Supervised Semantic Segmentation. In ICCV

    2023

  33. [41]

    Wang, J.; Huang, Q.; Tang, F.; Meng, J.; Su, J.; and Song, S. 2022. Stepwise feature fusion: Local guides global. In MICCAI. Springer

    2022

  34. [42]

    Wang, Y.; Zhang, J.; Kan, M.; Shan, S.; and Chen, X. 2020. Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation. In CVPR

    2020

  35. [43]

    Wei, J.; and Zhang, X. 2024. Dopra: Decoding over-accumulation penalization and re-allocation in specific weighting layer. In Proceedings of the 32nd ACM International Conference on Multimedia, 7065--7074

    2024

  36. [44]

    Wu, F.; He, J.; Yin, Y.; Hao, Y.; Huang, G.; and Cheng, L. 2024 a . Masked Collaborative Contrast for Weakly Supervised Semantic Segmentation. In WACV

    2024

  37. [45]

    Wu, Y.; Li, X.; Dai, S.; Li, J.; Liu, T.; and Xie, S. 2023. Hierarchical semantic contrast for weakly supervised semantic segmentation. In IJCAI

    2023

  38. [46]

    Wu, Y.; Ye, X.; Yang, K.; Li, J.; and Li, X. 2024 b . DuPL: Dual Student with Trustworthy Progressive Learning for Robust Weakly Supervised Semantic Segmentation. In CVPR, 3534--3543

    2024

  39. [47]

    Xie, J.; Hou, X.; Ye, K.; and Shen, L. 2022. Clims: Cross language image matching for weakly supervised semantic segmentation. In CVPR

    2022

  40. [48]

    Xiong, X.; Wu, Z.; Tan, S.; Li, W.; Tang, F.; Chen, Y.; Li, S.; Ma, J.; and Li, G. 2024. Sam2-unet: Segment anything 2 makes strong encoder for natural and medical image segmentation. arXiv preprint arXiv:2408.08870

    2024

  41. [49]

    Xu, L.; Bennamoun, M.; Boussaid, F.; Laga, H.; Ouyang, W.; and Xu, D. 2024 a . Mctformer+: Multi-class token transformer for weakly supervised semantic segmentation. TPAMI

    2024

  42. [50]

    Xu, L.; Ouyang, W.; Bennamoun, M.; Boussaid, F.; and Xu, D. 2022. Multi-class token transformer for weakly supervised semantic segmentation. In CVPR

    2022

  43. [51]

    Xu, L.; Ouyang, W.; Bennamoun, M.; Boussaid, F.; and Xu, D. 2023. Learning Multi-Modal Class-Specific Tokens for Weakly Supervised Dense Object Localization. In CVPR

    2023

  44. [52]

    Xu, Z.; Tang, F.; Chen, Z.; Zhou, Z.; Wu, W.; Yang, Y.; Liang, Y.; Jiang, J.; Cai, X.; and Su, J. 2024 b . Polyp-Mamba: Polyp Segmentation with Visual Mamba. In MICCAI. Springer

    2024

  45. [53]

    Yang, X.; and Gong, X. 2024. Foundation Model Assisted Weakly Supervised Semantic Segmentation. In WACV

    2024

  46. [54]

    Yang, Y.; Chen, H.; Liu, Z.; Lyu, Y.; Zhang, B.; Wu, S.; Wang, Z.; and Ren, K. 2023. Action recognition with multi-stream motion modeling and mutual information maximization. arXiv preprint arXiv:2306.07576

    2023 arXiv

  47. [55]

    Yang, Z.; Fu, K.; Duan, M.; Qu, L.; Wang, S.; and Song, Z. 2024. Separate and Conquer: Decoupling Co-occurrence via Decomposition and Representation for Weakly Supervised Semantic Segmentation. In CVPR

    2024

  48. [56]

    Ye, Y.; Xie, Y.; Zhang, J.; Chen, Z.; Wu, Q.; and Xia, Y. 2024. Continual self-supervised learning: Towards universal multi-modal medical data representation learning. In CVPR

    2024

  49. [57]

    Yoon, S.-H.; Kweon, H.; Cho, J.; Kim, S.; and Yoon, K.-J. 2022. Adversarial erasing framework via triplet with gated pyramid pooling layer for weakly supervised semantic segmentation. In ECCV

    2022

  50. [58]

    Yuan, X.; Shen, C.; Yan, S.; Zhang, X.; Xie, L.; Wang, W.; Guan, R.; Wang, Y.; and Ye, J. 2024. Instance-adaptive Zero-shot Chain-of-Thought Prompting. arXiv preprint arXiv:2409.20441

    2024 arXiv

  51. [59]

    Zhang, B.; Yu, S.; Wei, Y.; Zhao, Y.; and Xiao, J. 2024 a . Frozen CLIP: A Strong Backbone for Weakly Supervised Semantic Segmentation. In CVPR

    2024

  52. [60]

    Zhang, P.; Wang, Y.; Liu, Y.; Tu, Z.; and Lu, H. 2024 b . Magic tokens: Select diverse tokens for multi-modal object re-identification. In CVPR

    2024

  53. [61]

    Zhang, X.; Quan, Y.; Gu, C.; Shen, C.; Yuan, X.; Yan, S.; Cheng, H.; Wu, K.; and Ye, J. 2024 c . Seeing Clearly by Layer Two: Enhancing Attention Heads to Alleviate Hallucination in LVLMs. arXiv preprint arXiv:2411.09968

    2024 arXiv

  54. [62]

    Zhang, X.; Shen, C.; Yuan, X.; Yan, S.; Xie, L.; Wang, W.; Gu, C.; Tang, H.; and Ye, J. 2024 d . From Redundancy to Relevance: Enhancing Explainability in Multimodal Large Language Models. arXiv preprint arXiv:2406.06579

    2024 arXiv

  55. [63]

    Zhao, X.; Tang, F.; Wang, X.; and Xiao, J. 2024 a . SFC: Shared Feature Calibration in Weakly Supervised Semantic Segmentation. In AAAI

    2024

  56. [64]

    Zhao, X.; Yang, Z.; Dai, T.; Zhang, B.; and Xiao, J. 2024 b . PSDPM: Prototype-based Secondary Discriminative Pixels Mining for Weakly Supervised Semantic Segmentation. In CVPR, 3437--3446

    2024

  57. [65]

    Zhou, B.; Khosla, A.; Lapedriza, A.; Oliva, A.; and Torralba, A. 2016. Learning deep features for discriminative localization. In CVPR

    2016

  58. [66]

    Zhou, T.; Zhang, M.; Zhao, F.; and Li, J. 2022. Regional semantic contrast and aggregation for weakly supervised semantic segmentation. In CVPR

    2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.