Pith. sign in

REVIEW 3 major objections 5 minor 49 references

DiSa: Directional Saliency-Aware Prompt Learning for Generalizable Vision-Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DiSa claims that adding cross-interactive saliency and directional prototype regularization to CLIP prompt learning improves generalization to novel classes and unseen domains, reporting an average base-to-novel harmonic mean of 81.11.

desk verdict A useful incremental prompt-learning recipe whose main masking mechanism is not reproducible as written, but the empirical pattern is credible enough to warrant a careful revision. read the letter →

arxiv 2505.19373 v1 pith:FAGMHV7K submitted 2025-05-26 cs.CV

Niloufar Alipour Talemi , Hossein Kashiani , Hossein R. Nowdeh , Fatemeh Afghah This is my paper
classification cs.CV
keywords vision-languagemodelspromptlearningfew-shotdomaingeneralizationsaliency-awaremaskingdirectionalregularizationCLIPadaptation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes two regularization strategies for prompt learning in vision-language models: a cross-interactive regularization that aligns prompted and frozen encoders across modalities, and a directional regularization that aligns prompted visual features with class-mean prototypes computed from the frozen encoder. The central claim is that these regularizers prevent prompt overfitting and improve generalization to novel classes and unseen domains. If correct, this offers a practical way to adapt CLIP to downstream classification tasks with small per-task cost while retaining more of its pretrained generalization. The paper reports consistent gains over prior prompt-learning methods across eleven benchmarks.

What carries the argument

Three objects carry the argument. First, the Cross-Interactive Regularization loss, defined as $L_{CIR} = D_{KL}(q_{f_p g_o}, q_{f_o g_p})$, where $q_{f_p g_o}$ is the cosine-similarity score between prompted image embeddings and frozen text embeddings and $q_{f_o g_p}$ is the same score between frozen image embeddings and prompted text embeddings; this is paired with a score-consistency term $L_{SR} = D_{KL}(q_{f_p g_p}, q_{f_o g_o})$. Second, saliency-aware masking: the paper computes per-patch importance scores $\alpha_n$ by averaging attention over heads between the text encoder's CLS token and image patch tokens, then randomly masks half of the 50% least important patches before passing the image through the prompted encoder. Third, directional regularization, $L_{DiR} = 1 - \cos(f_p, m_i)$, where $m_i$ is the class-mean embedding of frozen image features for class $i$; this aligns orientation while leaving magnitude flexible. The paper positions CIR as the main departure from prior score- or feature-regularization methods, and the directional term as the replacement for strict proximity constraints.

What would settle it

Run the DiSa training pipeline on any of the 11 benchmarks with the saliency-masking term disabled but all other losses active: if the average harmonic mean does not drop by roughly the 0.63% novel-class margin reported in Table 4, the masking mechanism is not doing the claimed work. Separately, inspect the model's attention graph: Eq. 6 requires a cross-attention operation between the text encoder's CLS query and image patch keys, and standard CLIP provides no such operation, so the method as written can only be reproduced if the code release reveals an alternate attention path.

Watch

Extended reading notes

Core claim

On the paper's own terms, DiSa is a prompt-learning method that starts from an IVLP baseline and adds two complementary regularizers. The Cross-Interactive Regularization (CIR) term aligns cross-modal score distributions: the prompted image encoder's embeddings with the frozen text encoder's embeddings, and vice versa, using KL divergence, plus a score-consistency term between prompted and frozen same-modal predictions. A saliency-aware masking step uses the frozen model's guidance to identify and randomly mask less informative image patches, forcing the prompted encoder to rely on salient regions. The directional regularization aligns the prompted visual embedding's direction with a class-wise prototype, the mean frozen image embedding for that class, using a cosine-similarity loss rather than strict distance matching. The paper reports average base-to-novel harmonic mean 81.11, cross-dataset average 67.25, and domain-generalization average 61.43, each above the prior best values in the corresponding tables.

Load-bearing premise

The load-bearing premise is that the frozen text encoder's CLS token can attend to image patch tokens to score patch saliency, but in the standard CLIP architecture the text and image encoders are separate and no such cross-attention path is defined, so the masking component is not buildable exactly as written unless that path exists or the intended computation is inside the image encoder.

Editorial extensions

If this is right

  • Prompt tuning with DiSa yields higher novel-class accuracy than prior prompt-learning methods across 11 datasets, with an average base-to-novel harmonic mean of 81.11.
  • The gains are largest in low-shot settings: the paper reports average improvements of 1.18%, 0.95%, 0.80%, 0.45%, and 0.36% for 1, 2, 4, 8, and 16 shots, consistent with reduced overfitting.
  • An ImageNet-trained DiSa model transfers to 10 other datasets with average accuracy 67.25 and to ImageNet domain variants with average accuracy 61.43, both above the listed alternatives.
  • Inference cost is the same as the IVLP baseline (162.8 GFLOPs, 1380 FPS on SUN397), while training adds only about 0.11x GFLOPs, so the generalization gains do not come from extra inference compute.
  • Ablations show that directional alignment with class prototypes outperforms norm-only and MSE-based alignment, and that moderate saliency masking (25-35% of least informative patches) helps novel classes while heavier masking hurts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the saliency-masking formula as written requires an attention path from the text encoder's CLS token to image patch keys, which does not exist in the standard CLIP dual-encoder architecture; if the intended computation is image-encoder self-attention between the image CLS token and patch keys, the method would be implementable and the reported gains could be tested directly.
  • Beyond the paper: the directional regularizer may be helping by preventing representation collapse rather than by preserving orientation per se, since the ablations show norm alignment hurts and MSE alignment is weaker than cosine alignment; separating those explanations would require controlling for embedding norm explicitly.
  • Beyond the paper: because class prototypes are built only from base-class training features, the method's novel-class gains implicitly assume that base-class prototype directions transfer to novel classes; a direct test is to shrink the base-class split and check whether the novel-class advantage shrinks accordingly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes DiSa, a prompt-learning method for CLIP that augments the IVLP baseline with two regularizers: (i) a cross-interactive regularization (CIR) that aligns prompted/frozen cross-modal prediction scores and includes a saliency-aware masking of image patches, and (ii) a directional regularization that aligns prompted visual features with class-mean prototypes of the frozen image encoder via cosine similarity. The method is evaluated on 11 datasets under base-to-novel generalization, cross-dataset transfer, domain generalization, and few-shot classification, reporting average gains over prior prompt-learning methods. The paper also presents ablations and sensitivity studies for the proposed components.

Significance. If the reported results are reproducible, DiSa demonstrates a consistent though modest improvement over strong prompt-learning baselines, e.g., average base-to-novel harmonic mean 81.11 vs. 80.48 for CoPrompt and 79.97 for PromptSRC; cross-dataset average 67.25 vs. 67.00; domain-generalization average 61.43 vs. 60.80. The ablations in Table 4 support the contribution of each component, and the sensitivity studies are thorough. However, the paper does not release code, reports no error bars despite three-run averages, and the saliency-masking mechanism as written is incompatible with the standard CLIP architecture; these issues currently prevent verification of the central claim.

major comments (3)
  1. [Section 3.2.1, Eq. (6)] The saliency-masking mechanism as written is incompatible with the CLIP architecture used in all experiments. Equation (6) computes attention values between z_h^Q(CLS), a query from the text encoder's CLS token, and z_h^K(n), keys of image patch tokens from the image encoder. In the ViT-B/16 CLIP model used throughout the paper, the text and image encoders are separate Transformer stacks with no cross-attention between them; the text branch has no image patch keys and the image branch has no text CLS queries. The Fig. 2 caption explicitly restates this cross-encoder attention, so this is not a one-character typo. Because the masking step is one of the two named contributions, is reported to add 0.63% novel-class accuracy in Table 4, and is part of every configuration, the method cannot be built from the description as written. Please either (a) introduce and specify an explicit cross-attention module (with its own parameters and training), or (b) correct Eq. (6) and the caption to represent self-attention inside the image encoder (image CLS attending to image patches), and confirm that the reported experiments used that corrected mechanism.
  2. [Tables 1-3 and Fig. 3] All main results are stated to be averaged over three runs (Section 4.1), yet no standard deviations or error bars are reported anywhere. The reported gains are small: for example, the average base-to-novel harmonic mean improvement over CoPrompt is 0.63 points, and the cross-dataset improvement over CoPrompt is 0.25 points. Without variance information, these differences may be within run-to-run noise, especially given the batch size of 4 and 20-epoch training. Please report per-dataset and average standard deviations for the three runs in Tables 1-3 and Fig. 3, and adjust the significance claims accordingly.
  3. [Section 4.6, Tables 5 and 6] Key hyperparameters are selected on the same test benchmarks used for the final reported numbers. Table 5 picks lambda=12 by maximizing the average harmonic mean across the 11 test datasets, and Table 6 picks prompt depth by evaluating domain-generalization and base-to-novel test averages. This test-set-informed selection can overstate performance and makes the reported numbers not strictly unbiased estimates of the method's expected performance. Please either adopt a validation-set-based selection protocol (e.g., training on base classes and selecting on a held-out split of base classes, or reporting the sensitivity curves without claiming the best point as the expected result) or state explicitly that the final numbers are test-set-optimized and discuss the expected optimism. At minimum, include this caveat in the experimental setup.
minor comments (5)
  1. [Table 7] The computational cost table appears internally inconsistent. The caption says the comparison is 'on the SUN397 dataset,' but the HM column for CoOp (71.66), CoCoOp (75.83), and PromptSRC (79.97) matches the average base-to-novel harmonic means in Table 1, not the SUN397-specific HM values (72.51, 77.64, and 82.74 in Table 1(i)). DiSa's HM of 80.95 in Table 7 also matches neither the average 81.11 in Table 1(a) nor the SUN397 value 81.60 in Table 1(i). Please clarify which benchmark and training configuration (e.g., 10 vs. 20 epochs) this table refers to, and correct the caption and header accordingly.
  2. [Abstract and Section 4.2] The abstract claims DiSa 'consistently outperforms' state-of-the-art methods, but Table 1 shows that DiSa is not the best method on several datasets, e.g., StanfordCars (HM 76.77 vs. 77.71 for APEX) and Flowers102 (HM 86.15 vs. 86.40 for APEX). The claim is supported on average, but the wording should be qualified to 'on average' or 'in most settings.'
  3. [Section 4.1] The statement 'we utilize deep prompting with V=T=4 VL prompts' is ambiguous: V and T were introduced in Section 3.1 as the numbers of visual and textual prompt tokens, not as a number of transformer layers, yet the next sentence refers to 'first three transformer layers' and 'first nine transformer layers.' Please clarify how many prompt tokens are used per layer and at which layers they are inserted, and reconcile the notation.
  4. [Throughout] There are several typos and formatting inconsistencies: Table 2's header reads 'PromtSCR' instead of 'PromptSRC'; the method name is spelled both 'MaPLe' and 'Maple'; Table 4's header line is fragmented; and Fig. 4(a)'s axis label 'Masked Least Important Patches (%)' conflicts with the text's description of randomly masking within the least important subset. Please copyedit these.
  5. [Section 4.1] No code or pre-trained checkpoints are provided. Given that Eq. (6) is currently not reproducible, releasing code (or at least a detailed pseudocode for the masking step) is essential for the community to verify and build on the results.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DiSa is an empirical training recipe, and its regularization terms are not defined in terms of the reported benchmark outcomes.

full rationale

DiSa's derivation chain is a standard empirical prompt-learning pipeline. The base model is the external IVLP baseline, and the claimed contributions (CIR, saliency masking, and directional prototype alignment) are additional loss terms defined in Eqs. 2, 4, 7, and 9. These losses are optimized on training data and then evaluated on novel classes, cross-dataset targets, and domain-shifted datasets; none of the loss definitions incorporates the reported test accuracies. The class-mean prototypes in Eq. 8 are computed from training labels and frozen CLIP features, which is ordinary supervised regularization rather than circularity, because the novel-class and transfer results are not used to compute those prototypes. The paper contains no load-bearing self-citations by the authors; all cited prior methods are external baselines. The questionable cross-attention construction in Eq. 6, where a text-encoder CLS query is combined with image-patch keys, is a reproducibility or implementability defect rather than a circular reduction: it does not make any reported result equivalent to its inputs by construction. Hyperparameter selection on the evaluation benchmarks is a soundness concern, not circularity. Therefore the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new particles, forces, or latent variables are introduced. The prototype m_i is a class-mean statistic, not an invented entity. The main unverified constructs are losses and masking procedures, which are captured under assumptions and red flags.

free parameters (3)
  • lambda (directional regularization weight) = 12
    Selected from Table 5 as the value maximizing average harmonic mean across the 11 test datasets; no validation split is described.
  • Masking ratio gamma = mask half of the bottom-50% least-attended patches (25% of all patches)
    Chosen from Fig. 4 ablation on test-set accuracy; the stated implementation randomly masks half of the least important patches.
  • Prompt depth = 9 layers for base-to-novel and few-shot; 3 layers for cross-dataset and domain generalization
    Chosen from Table 6 ablation per setting using the target benchmark averages.
assumptions (4)
  • domain assumption The frozen CLIP encoders provide reliable guidance for both saliency and prototype directions.
    Saliency masking (Eq. 6) and class-mean prototypes (Eq. 8) both assume frozen CLIP features indicate which image regions and class directions are semantically important; this is plausible but empirically unverified.
  • domain assumption Attention scores between a CLS token and image patch tokens are a valid proxy for patch importance.
    The masking step sorts patch importance by attention scores; the paper provides no analysis linking these scores to classification-relevant regions beyond the ablation in Fig. 4.
  • standard math KL divergence between softmax-normalized cosine similarity distributions is an appropriate alignment objective.
    KL divergence is mathematically well-defined for these distributions, but the paper does not demonstrate its superiority over other alignment losses beyond the reported ablations.
  • domain assumption Class-mean vectors computed from 16 training samples are stable enough to serve as prototypes.
    Section 3.2.2 uses class means over few-shot training examples as anchors; with 16 samples per class these means may be noisy, and the paper does not quantify this.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DiSa: Directional Saliency-Aware Prompt Learning for Generalizable Vision-Language Models." pith.science (2026). https://pith.science/paper/FAGMHV7K

@misc{pith2026250519373,
  author       = {Pith},
  title        = {Pith review of: DiSa: Directional Saliency-Aware Prompt Learning for Generalizable Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FAGMHV7K}},
  note         = {Machine review of arXiv:2505.19373}
}
read the original abstract

Prompt learning has emerged as a powerful paradigm for adapting vision-language models such as CLIP to downstream tasks. However, existing methods often overfit to seen data, leading to significant performance degradation when generalizing to novel classes or unseen domains. To address this limitation, we propose DiSa, a Directional Saliency-Aware Prompt Learning framework that integrates two complementary regularization strategies to enhance generalization. First, our Cross-Interactive Regularization (CIR) fosters cross-modal alignment by enabling cooperative learning between prompted and frozen encoders. Within CIR, a saliency-aware masking strategy guides the image encoder to prioritize semantically critical image regions, reducing reliance on less informative patches. Second, we introduce a directional regularization strategy that aligns visual embeddings with class-wise prototype features in a directional manner to prioritize consistency in feature orientation over strict proximity. This approach ensures robust generalization by leveraging stable prototype directions derived from class-mean statistics. Extensive evaluations on 11 diverse image classification benchmarks demonstrate that DiSa consistently outperforms state-of-the-art prompt learning methods across various settings, including base-to-novel generalization, cross-dataset transfer, domain generalization, and few-shot learning.

Figures

Figures reproduced from arXiv: 2505.19373 by the authors.

Figure 1
Figure 1. Performance comparison of base-to-novel general [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed DiSa. The DiSa employs two complementary regularization approaches: saliency-aware [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Performance comparison across K-shot settings (K = 1, 2, 4, 8, 16). Our approach consistently achieves superior average [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Analysis of saliency masking and the directional regularization. (a) Accuracy vs. percentage of least informative patches [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

49 extracted references · 27 canonical work pages

  1. [1]

    Maximilian Augustin, Alexander Meinke, and Matthias Hein. 2020. Adversar- ial robustness on in-and out-distribution improves explainability. In European Conference on Computer Vision . 228–245

    work page 2020

  2. [2]

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. 2014. Food-101–mining discriminative components with random forests. In European Conference on Computer Vision. 446–461

    work page 2014

  3. [3]

    Adrian Bulat and Georgios Tzimiropoulos. 2023. LASP: Text-to-text optimization for language-aware soft prompting of vision & language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 23232– 23241

    work page 2023

  4. [4]

    Guangyi Chen, Weiran Yao, Xiangchen Song, Xinyue Li, Yongming Rao, and Kun Zhang. 2022. PLOT: Prompt learning with optimal transport for vision-language models. arXiv preprint arXiv:2210.01253 (2022)

    arXiv 2022

  5. [5]

    Hyeong Kyu Choi, Joonmyung Choi, and Hyunwoo J Kim. 2022. TokenMixup: Efficient attention-guided token-level data augmentation for transformers. Ad- vances in Neural Information Processing Systems 35 (2022), 14224–14235

    work page 2022

  6. [6]

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and An- drea Vedaldi. 2014. Describing textures in the wild. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 3606–3613

    work page 2014

  7. [7]

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Ima- geNet: A large-scale hierarchical image database. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Ieee, 248–255

    work page 2009

  8. [8]

    Mohammad Mahdi Derakhshani, Enrique Sanchez, Adrian Bulat, Victor G Turrisi da Costa, Cees GM Snoek, Georgios Tzimiropoulos, and Brais Martinez. 2023. Bayesian prompt learning for image-language model generalization. In Proceed- ings of the IEEE/CVF International Conference on Computer Vision . 15237–15246

    2023

Show all 49 references
  1. [9]

    Jian Ding, Nan Xue, Gui-Song Xia, and Dengxin Dai. 2022. Decoupling zero-shot semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 11583–11592

    2022

  2. [10]

    Li Fei-Fei, Rob Fergus, and Pietro Perona. 2004. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 178–178

    2004

  3. [11]

    Chengjian Feng, Yujie Zhong, Zequn Jie, Xiangxiang Chu, Haibing Ren, Xiaolin Wei, Weidi Xie, and Lin Ma. 2022. PromptDet: Towards open-vocabulary detection using uncurated images. In European Conference on Computer Vision . 701–717

    2022

  4. [12]

    Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. 2024. CLIP-Adapter: Better vision-language models with feature adapters. International Journal of Computer Vision 132, 2 (2024), 581–595

    2024

  5. [13]

    Wenbin He, Suphanut Jamonnak, Liang Gou, and Liu Ren. 2023. Clip-S4: Language-guided self-supervised semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 11207–11216

    2023

  6. [14]

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. 2019. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 12, 7 (2019), 2217–2226

    2019

  7. [15]

    Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. 2021. The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF Internati...

    2021

  8. [16]

    Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song

  9. [17]

    Gabriel Ilharco, Mitchell Wortsman, Samir Yitzhak Gadre, Shuran Song, Han- naneh Hajishirzi, Simon Kornblith, Ali Farhadi, and Ludwig Schmidt. 2022. Patch- ing open-vocabulary models by interpolating weights. Advances in Neural Infor- mation Processing Systems 35 (2022), 29262–29277

    2022

  10. [18]

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. 2021. Scaling up visual and vision- language representation learning with noisy text supervision. In International Conference on Machine Learning . PMLR, 4904–4916

    2021

  11. [19]

    Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. 2023. Maple: Multi-modal prompt learning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 19113–19122

    2023

  12. [20]

    Muhammad Uzair Khattak, Syed Talal Wasim, Muzammal Naseer, Salman Khan, Ming-Hsuan Yang, and Fahad Shahbaz Khan. 2023. Self-regulating prompts: Foundational model adaptation without forgetting. InProceedings of the IEEE/CVF International Conference on Computer Vision . 15190–15200

    2023

  13. [21]

    Konwoo Kim, Michael Laskin, Igor Mordatch, and Deepak Pathak. 2021. How to adapt your large-scale vision-and-language model. (2021)

    2021

  14. [22]

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 2013. 3D object repre- sentations for fine-grained categorization. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision Workshops . 554–561

    2013

  15. [23]

    Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)

    2017 arXiv

  16. [24]

    Yuning Lu, Jianzhuang Liu, Yonggang Zhang, Yajing Liu, and Xinmei Tian. 2022. Prompt distribution learning. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition . 5206–5215

    2022

  17. [25]

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi

  18. [26]

    Muhammad Ferjad Naeem, Yongqin Xian, Luc V Gool, and Federico Tombari

  19. [27]

    Maria-Elena Nilsback and Andrew Zisserman. 2008. Automated flower classifi- cation over a large number of classes. In Sixth Indian Conference on Computer Vision, Graphics & Image Processing . 722–729

    2008

  20. [28]

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. 2012. Cats and dogs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3498–3505

    2012

  21. [29]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International Conference on Machine Learni...

    2021

  22. [30]

    Yongming Rao, Wenliang Zhao, Guangyi Chen, Yansong Tang, Zheng Zhu, Guan Huang, Jie Zhou, and Jiwen Lu. 2022. DenseCLIP: Language-guided dense pre- diction with context-aware prompting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 18082–18091

    2022

  23. [31]

    Hanoona Rasheed, Muhammad Uzair Khattak, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. 2023. Fine-tuned CLIP models are efficient video learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6545–6554

    2023

  24. [32]

    Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. 2019. Do ImageNet classifiers generalize to ImageNet?. In International Conference on Machine Learning. PMLR, 5389–5400

    2019

  25. [33]

    Shuvendu Roy and Ali Etemad. 2023. Consistency-guided prompt learning for vision-language models. arXiv preprint arXiv:2306.01195 (2023)

    2023 arXiv

  26. [34]

    Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. 2012. UCF101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402 (2012)

    2012 arXiv

  27. [35]

    Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research 15, 1 (2014), 1929–1958

    2014

  28. [36]

    Maria Tsimpoukelli, Jacob L Menick, Serkan Cabi, SM Eslami, Oriol Vinyals, and Felix Hill. 2021. Multimodal few-shot learning with frozen language models. Advances in Neural Information Processing Systems 34 (2021), 200–212

    2021

  29. [37]

    Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. 2019. Learning robust global representations by penalizing local predictive power. Advances in Neural Information Processing Systems 32 (2019)

    2019

  30. [38]

    Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba

  31. [39]

    Lingxiao Yang, Ru-Yuan Zhang, Yanchen Wang, and Xiaohua Xie. 2024. MMA: Multi-Modal Adapter for Vision-Language Models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 23826–23837

    2024

  32. [40]

    Yongjin Yang, Jongwoo Ko, and Se-Young Yun. 2023. Towards Difficulty- Agnostic Efficient Transfer Learning for Vision-Language Models. arXiv preprint arXiv:2311.15569 (2023)

    2023 arXiv

  33. [41]

    Hantao Yao, Rui Zhang, and Changsheng Xu. 2024. Tcp: Textual-based class- aware prompt tuning for visual-language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 23438–23448

    2024

  34. [42]

    Lewei Yao, Runhui Huang, Lu Hou, Guansong Lu, Minzhe Niu, Hang Xu, Xiao- dan Liang, Zhenguo Li, Xin Jiang, and Chunjing Xu. 2021. Filip: Fine-grained interactive language-image pre-training. arXiv preprint arXiv:2111.07783 (2021)

    2021 arXiv

  35. [43]

    Zhaohui Zheng, Rongguang Ye, Qibin Hou, Dongwei Ren, Ping Wang, Wangmeng Zuo, and Ming-Ming Cheng. 2023. Localization distillation for object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 8 (2023), 10070– 10083

    2023

  36. [44]

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022. Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 16816–16825

    2022

  37. [45]

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022. Learning to prompt for vision-language models. International Journal of Computer Vision 130, 9 (2022), 2337–2348

    2022

  38. [2010]

    In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Sun database: Large-scale scene recognition from abbey to zoo. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 3485–3492

  39. [2013]

    arXiv preprint arXiv:1306.5151 (2013)

    Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151 (2013)

    2013 arXiv

  40. [2021]

    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Natural adversarial examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 15262–15271

  41. [2022]

    Advances in Neural Information Processing Systems 35 (2022), 12283– 12294

    I2DFormer: Learning image to document attention for zero-shot image classification. Advances in Neural Information Processing Systems 35 (2022), 12283– 12294

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.