Pith. sign in

REVIEW 4 major objections 6 minor 53 references

Dynamic Rank Adaptation for Vision-Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Dynamic Rank Adaptation claims that fine-tuning CLIP-like vision-language models with adapter capacity allocated by token importance improves accuracy on unseen new classes while keeping base-class accuracy nearly intact.

desk verdict A plausible adapter variant with a genuinely new training-time mechanism, but the dynamic-rank story does not survive contact with the inference-time equations, and the sorting is non-differentiable as written. read the letter →

arxiv 2507.05668 v1 pith:7UIAYFB2 submitted 2025-07-08 cs.CV

Jiahui Wang , Qin Xu , Bo Jiang , Bin Luo This is my paper · ORCID
classification cs.CV
keywords Vision-languagemodelsCLIPfine-tuningAdaptersDynamicrankallocationTokenimportanceBase-to-newgeneralizationFew-shotlearningPrompt
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to solve a known failure: when CLIP is fine-tuned on base classes, prompt and adapter methods often overfit and lose accuracy on unseen classes. It proposes an adapter variant, Dynamic Rank Adaptation (DRA), that gives more adaptation rank to tokens judged important and compresses unimportant ones. On 11 datasets with 16 shots per class, DRA reports average base accuracy 83.06, new-class accuracy 77.75, and harmonic mean 80.32, beating the prior adapter MMA by 0.95 on new classes and 0.45 on harmonic mean. The paper argues this shows allocating capacity by token importance preserves general representations.

What carries the argument

The load-bearing object is the token-level rank mask $\mathbf{M}'$. Token importance scores $a_t$ are computed as averaged self-attention over the low-rank adapter projection; tokens are sorted and split into $K=4$ groups with rank ratios $[1.0, 0.8, 0.6, 0.4]$. Each group keeps the top $\lfloor e_i \times r \rfloor$ channels according to a channel response score $a_c$, the per-channel variance of the projected low-rank features. The mask is applied after sorting tokens and then restored, so each sample receives its own rank allocation while the final projection path remains a standard adapter at inference.

What would settle it

Measure new-class accuracy after replacing the learned token importance scores with random scores drawn from the same Gumbel distribution while keeping the identical rank budget; if random grouping matches DRA's new-class gain over the fixed-rank baseline, the importance mechanism is not the source of the improvement.

Watch

Extended reading notes

Core claim

The central claim is that the fixed-rank bottleneck shared by all tokens limits new-class generalization, and replacing it with adaptive per-token rank allocation removes that limitation. DRA computes token importance scores from a low-rank self-attention map, sorts tokens into four groups, and assigns rank ratios [1.0, 0.8, 0.6, 0.4] so important tokens keep full capacity while unimportant tokens are compressed. A channel response mechanism keeps the most active channels of each sample when compressing, and an L1 loss pulls training output toward inference output to stabilize training. The paper reports that DRA outperforms prior methods on new-class accuracy in 7/11 datasets, on harmonic mean in 8/11 datasets, and also tops cross-dataset and domain generalization averages.

Load-bearing premise

The method assumes the self-attention scores computed inside the adapter rank tokens by true semantic importance, so the most informative tokens are the ones kept at full rank; the paper shows only a qualitative visualization, not a quantitative check.

Editorial extensions

If this is right

  • On base-to-new generalization over 11 datasets, DRA improves average new-class accuracy to 77.75 and harmonic mean to 80.32, with base accuracy 83.06 nearly unchanged from the MMA baseline.
  • DRA improves new-class accuracy on 7/11 datasets and harmonic mean on 8/11 datasets compared with the best prior method.
  • Channel response and L1 regularization each add measurable gains: removing either lowers the harmonic mean relative to the full DRA configuration.
  • Cross-dataset average accuracy (66.71) and domain generalization average accuracy (60.64) are the highest reported among the compared methods.
  • At inference, dynamic allocation is unwound: features pass through the same linear projections as a normal adapter, so DRA adds no inference-time token selection step.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the importance scores are genuinely semantic, the same dynamic-rank principle should transfer beyond classification: any transformer adapter could allocate capacity by token importance for detection, segmentation, or retrieval tasks that the paper does not test.
  • The paper's own ablations leave open the possibility that part of the gain is a regularization effect of masking; comparing DRA against random token masks with the same rank budget would separate those explanations.
  • The token importance scores could be reused as free saliency maps; evaluating them against foreground annotations would both test the core mechanism and produce a side product.
  • Because DRA converges in five epochs, the same capacity-allocation idea may reduce fine-tuning compute for larger vision-language models, though the paper only reports results on CLIP ViT-B/16.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Dynamic Rank Adaptation (DRA), an adapter-based method for few-shot fine-tuning of CLIP on base-to-new class generalization. DRA computes per-token importance scores in a low-rank space (Eq. 10), sorts tokens into K groups (Eq. 11), and assigns each group a different effective rank by masking channels in a group-dependent manner (Eqs. 14-16). A channel-variance criterion (Eq. 12) is used to retain the most informative channels, and an L1 loss (Eq. 19) aligns the training-time dynamic output with a fixed-rank inference-time path. The authors report average base/new/HM accuracies of 83.06/77.75/80.32 across 11 base-to-new datasets, claiming SOTA new-class and harmonic-mean performance, together with cross-dataset and domain-generalization results.

Significance. If the mechanistic claim were established, DRA would be a valuable contribution: token-conditional rank allocation is a plausible way to avoid overfitting to background or template tokens during CLIP adaptation, and the new-class evaluation is a genuine held-out test rather than a fit to the evaluation classes. The paper provides broad benchmark coverage (11 datasets for base-to-new, 10 for cross-dataset, 4 for domain generalization) and reports efficiency comparisons. However, the significance is currently limited because the central mechanism described in Sections 3.2.1-3.2.3 is not supported as written: the importance projections are not trainable under the stated equations, the dynamic path is absent at inference, and the ablations do not isolate the proposed components from the added linear layer and the regularization loss.

major comments (4)
  1. [§3.2.1 (Eqs. 11 and 13) and §3.2.3 (Eq. 15)] The sorting and masking operations are non-differentiable with respect to the 'learnable' importance projections W_t and W_c. Equation (11) applies arg sort to a_t + g, and Eq. (13) does the same for a_c + g; hard arg-sort has zero derivative almost everywhere, and the binary mask M in Eq. (15) has no gradient channel back to the scores. Adding Gumbel noise does not make the sorting process differentiable; a straight-through estimator or a soft-sort relaxation would be required but is never stated. Consequently, W_t and W_c cannot be trained by the given loss, and the reported token grouping is determined by the random initialization of W_t rather than by learned importance. The authors should either specify a concrete gradient estimator or compare against a variant with frozen/random W_t and W_c to demonstrate that learned grouping contributes to the results.
  2. [§3.2.3 (Eq. 18) and §3.3 (Eq. 19)] The deployed model at inference is a fixed-rank adapter with an extra linear layer W_p; it contains no sorting, no group masks, and no importance grouping. The paper states that DRA is enabled only during training, and Eq. (19) explicitly penalizes the training-time dynamic output for deviating from this static inference output (with λ_T = λ_V = 1 in the experiments). Therefore, the headline gains in Table 1 cannot be attributed to dynamic rank allocation at inference time; they may instead come from the added projection W_p or from stochastic masking acting as a training regularizer. The authors should ablate W_p alone and also compare against a variant that uses static or random masks during training to separate these effects.
  3. [§4.3.1, Table 4] The component ablation does not isolate the proposed components. 'Baseline' is not defined in the experimental settings, and the DRA-T&V rows all include W_p as part of the DRA module. Removing CR and L1 leaves a variant that differs from Baseline by both token-group masking and the extra linear layer W_p, so the observed +0.33 HM over Baseline (and the full +0.91 HM) could be entirely due to W_p. The authors should ablate W_p alone and compare DRA against random token/channel masks to establish that the learned importance scores, rather than the extra capacity or the regularization, drive the reported improvement.
  4. [§4.1.2, §4.3.2-4.3.4, Table 5, Figure 3] The hyperparameters K, the rank ratios e_i, the intermediate dimension r, and the starting layer h were chosen by maximizing harmonic mean on the same 11 datasets used for the headline comparison, and no separate validation split is reported. This selection-on-test-set procedure biases the reported averages in DRA's favor and makes the 0.95 New / 0.45 HM margin over MMA difficult to interpret. The authors should either fix hyperparameters a priori, use a validation split, or report a leave-one-dataset-out selection protocol.
minor comments (6)
  1. [§3.2.1, Eq. (10)] The softmax is written over an N×N matrix and then averaged over the row index i, but the normalization dimension and the exact indexing should be stated explicitly; the current notation is ambiguous about whether Softmax is applied row-wise or column-wise.
  2. [§3.2.3, Eq. (16)] Equation (16) contains a typo: 'RestoringSort(Sort(X′,t)⊙M′),t)' has an extra comma and parenthesis; the argument order of RestoringSort should be clarified.
  3. [§3.2.3, Eq. (18)] Equation (18) uses X on both sides of the assignment, which is confusing because the left-hand side is the adapter output while the right-hand side is the input to the adapter; the output should be renamed (e.g., X_out).
  4. [§4.3.1, Table 4] The row labels 'DRA-T&V - -', 'DRA-T&V ✓ -', and 'DRA-T&V - ✓' are not self-explanatory; the table should explicitly define which components each checkmark denotes.
  5. [§4.4, Figure 4] The histogram axes and the color scale for the token importance and rank allocation are not labeled, and the cropping of the <SOS> score is mentioned but not shown, which makes the visualization difficult to interpret.
  6. [§4.2.1, Table 1] The text claims that DRA improves over SOTA methods on new classes and harmonic mean, but the average base-class accuracy (83.06) is below LFA (83.62), MetaPrompt (83.38), and TCP (84.13); the conclusion should qualify that the improvement is specifically on new-class and harmonic-mean metrics, not on base-class accuracy.

Circularity Check

1 steps flagged · score 4.0 of 10

The headline new-class result is partly forced by hyperparameters selected on the same evaluation benchmark, while no derivation chain is itself circular.

  1. fitted input called prediction [Section 4.3.2 and 4.3.3 (Table 5, Figure 3) vs. Section 4.1.2 and 4.2.1 (Table 1)]
    "We set the number of token importance groups K to 4 and the rank ratio is set to [1.0, 0.8, 0.6, 0.4]. ... This trend continues until r = 32 when the model reaches a peak in the harmonic mean accuracy. ... Therefore, r = 32 was determined to be the optimal intermediate dimension setting."

    The configuration reported as the final DRA result in Table 1 (Base=83.06, New=77.75, HM=80.32) is exactly the configuration selected as best on the same 11-dataset benchmark in Table 5 (K=4, ratios [1.0, 0.8, 0.6, 0.4]) and Figure 3 (r=32, h=5). No separate validation split or held-out hyperparameter selection is reported; the paper only states that 'the final performance is averaged over three random seeds.' The headline claim that 'the new classes and HM of our proposed method outperform the SOTA methods' is therefore the maximum of the searched grid over the evaluation data, so the new-class/HM advantage is partly forced by the selection procedure rather than an independent prediction.

full rationale

The paper contains no self-citation chain, no imported uniqueness theorem, and no equation-level reduction in which an output is defined as its own input. The method's derivation is self-contained: token importance scores, channel response, rank grouping, and the L1 regularization are all specified with explicit equations. The main circularity concern is empirical rather than formal: the final hyperparameters (r=32, h=5, K=4, rank ratios [1.0, 0.8, 0.6, 0.4]) were chosen by harmonic-mean performance on the exact benchmarks used for the headline comparison, and the selected configuration is then presented as the method's result in Table 1. This makes the reported new-class generalization advantage partially an artifact of test-set model selection. Separately, the skeptic's points about the non-differentiability of the hard sorting mask and about the inference-time model (Eq. 18) not containing the dynamic masking path are substantive correctness and mechanism-identification concerns, but they are not circularity: an inert or mis-specified mechanism does not make the reported numbers equivalent to their inputs by construction. Overall, the central result is an empirical comparison with real held-out new classes, so the partial circularity is limited to the hyperparameter-selection loop.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several manually chosen hyperparameters, all tuned through ablation on the same evaluation benchmarks, and on two heuristic importance measures (token attention and channel variance) that are asserted rather than derived. No new physical or conceptual entities are introduced.

free parameters (6)
  • intermediate dimension r = 32
    Maximum bottleneck rank; selected by ablation on base-new HM (Fig. 3a), with r=32 the peak.
  • number of token groups K = 4
    Selected from Table 5; K=4 gives the best HM among tested values.
  • rank ratios e_i = [1.0, 0.8, 0.6, 0.4]
    Chosen from Table 5; lower ratios hurt both base and new performance.
  • starting layer h = 5 (base-new), 7 (cross-dataset and domain generalization)
    Selected from Fig. 3b for base-new; a separate value is used for ImageNet-trained models.
  • regularization weights lambda_T and lambda_V = 1.0
    Set manually; no ablation or sensitivity analysis is reported.
  • scaling factor s = 0.001
    Set manually and consistent with prior adapters; no sensitivity analysis.
assumptions (4)
  • domain assumption CLIP features (ViT-B/16) are frozen and retain their general zero-shot representations, with adaptation confined to injected adapters.
    Section 3.1.1 and implementation details; this is the standard PEFT setup inherited from prior work.
  • ad hoc to paper Token importance scores from self-attention in the low-rank space (Eq. 10) correspond to semantic informativeness.
    Core premise of token grouping; supported only by qualitative visualization in Fig. 4.
  • ad hoc to paper Channel variance (Eq. 12) identifies the most informative channels for a sample.
    Used to retain channels during rank reduction; no empirical or theoretical justification beyond the stated heuristic.
  • domain assumption Fine-tuning lower layers of CLIP is harmful, so adapters start at layer h (based on prior work [33,35]).
    Section 3.3; the starting layer is then tuned empirically in Fig. 3b.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dynamic Rank Adaptation for Vision-Language Models." pith.science (2026). https://pith.science/paper/7UIAYFB2

@misc{pith2026250705668,
  author       = {Pith},
  title        = {Pith review of: Dynamic Rank Adaptation for Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7UIAYFB2}},
  note         = {Machine review of arXiv:2507.05668}
}
read the original abstract

Pre-trained large vision-language models (VLMs) like CLIP demonstrate impressive generalization ability. Existing prompt-based and adapter-based works have made significant progress in fine-tuning VLMs but still face the challenges of maintaining strong generalization abilities, particularly towards unseen new classes. This limitation partly arises from these methods treating all tokens of the image and text encoder equally, which can lead to overfitting on less informative features (e.g., background noise, template words) and degrade the general representations that are crucial for novel concept recognition. To address this issue, we propose Dynamic Rank Adaptation (DRA), a novel adapter variant method, designed specifically to enhance new class generalization. DRA dynamically allocates adaptation ranks based on the importance of features during training to preserve general knowledge. DRA first employs token importance grouping, using sequence attention to evaluate and group tokens by their importance. Then, we adopt rank adaptation according to the importance of each token group dynamically by assigning higher feature ranks to the more important tokens. Also, we design a new channel response mechanism to prioritize the preservation and adaptation of feature channels identified as the most informative for each instance. In addition, a L1 regularization term is introduced to stabilize the training. Extensive experiments demonstrate the effectiveness and superiority of our proposed DRA over existing works, especially on enhancing the performance of new classes on various benchmarks, including base-new classes, cross-datasets evaluation and domain generalization. The source code will be published after the paper is received.

Figures

Figures reproduced from arXiv: 2507.05668 by the authors.

Figure 1
Figure 1. The motivation of the proposed method is to dynam [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The diagram of the proposed Dynamic Rank Adaptation (Left) and the Detailed view (Right). DRA dynamically assigns [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Results of ablation experiments on the impact of [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Visualization of the token importance group and [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 29 canonical work pages

  1. [1]

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. 2014. Food-101–mining discriminative components with random forests. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part VI 13. Springer, 446–461

    2014

  2. [2]

    Guangyi Chen, Weiran Yao, Xiangchen Song, Xinyue Li, Yongming Rao, and Kun Zhang. 2023. PLOT: Prompt Learning with Optimal Transport for Vision- Language Models. ICLR (2023)

    work page 2023

  3. [3]

    Shoufa Chen, Chongjian GE, Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang, and Ping Luo. 2022. AdaptFormer: Adapting Vision Transformers for Scalable Vi- sual Recognition. In Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), Vol. 35. Curran Associates, Inc., 16664–16678

    work page 2022

  4. [4]

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and An- drea Vedaldi. 2014. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition . 3606–3613

    2014

  5. [5]

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Im- ageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition . 248–255

    2009

  6. [6]

    Li Fei-Fei, Rob Fergus, and Pietro Perona. 2004. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In 2004 conference on computer vision and pattern recognition workshop. IEEE, 178–178

    2004

  7. [7]

    Chengjian Feng, Yujie Zhong, Zequn Jie, Xiangxiang Chu, Haibing Ren, Xiaolin Wei, Weidi Xie, and Lin Ma. 2022. Promptdet: Towards open-vocabulary detection using uncurated images. In European Conference on Computer Vision . Springer, 701–717

    work page 2022

  8. [8]

    Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. 2024. Clip-adapter: Better vision-language models with feature adapters. International Journal of Computer Vision 132, 2 (2024), 581–595

    2024

Show all 53 references
  1. [9]

    Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, and Yin Cui. 2021. Open-vocabulary object detection via vision and language knowledge distillation. arXiv preprint arXiv:2104.13921 (2021)

    2021 arXiv

  2. [10]

    Wenbin He, Suphanut Jamonnak, Liang Gou, and Liu Ren. 2023. Clip-s4: Language-guided self-supervised semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 11207–11216

    2023

  3. [11]

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. 2019. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 12, 7 (2019), 2217–2226

    2019

  4. [12]

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In International conference on machine learning. PMLR, 2790–2799

    2019

  5. [13]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)

    2021 arXiv

  6. [14]

    Lianyu Hu, Liqing Gao, Zekang Liu, Chi-Man Pun, and Wei Feng. 2024. COMMA: Co-articulated Multi-Modal Learning.. In AAAI Conference on Artificial Intelli- gence (AAAI). 2238–2246

    2024

  7. [15]

    Shaohan Huang, Li Dong, Wenhui Wang, Yaru Hao, Saksham Singhal, Shuming Ma, Tengchao Lv, Lei Cui, Owais Khan Mohammed, Barun Patra, and others

  8. [16]

    Sajid Javed, Arif Mahmood, Iyyakutti Iyappan Ganapathi, Fayaz Ali Dharejo, Naoufel Werghi, and Mohammed Bennamoun. 2024. Cplip: zero-shot learning for histopathology with comprehensive vision-language alignment. In Proceedings of the IEEE/CVF conference on computer vision and ...

    2024

  9. [17]

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. 2022. Visual prompt tuning. In Euro- pean Conference on Computer Vision . Springer, 709–727

    2022

  10. [18]

    Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. 2023. MaPLe: Multi-Modal Prompt Learning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 19113–19122

    2023

  11. [19]

    Muhammad Uzair Khattak, Syed Talal Wasim, Muzammal Naseer, Salman Khan, Ming-Hsuan Yang, and Fahad Shahbaz Khan. 2023. Self-regulating prompts: Foundational model adaptation without forgetting. InProceedings of the IEEE/CVF international conference on computer vision . 15190–15200

    2023

  12. [20]

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven C. H. Hoi. 2022. BLIP: Boot- strapping Language-Image Pre-training for Unified Vision-Language Understand- ing and Generation.. In International Conference on Machine Learning (ICML) . 12888–12900

    2022

  13. [21]

    Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. Comput. Surveys 55, 9 (2023), 1–35

    2023

  14. [22]

    Yuning Lu, Jianzhuang Liu, Yonggang Zhang, Yajing Liu, and Xinmei Tian. 2022. Prompt distribution learning. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition . 5206–5215

    2022

  15. [23]

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi

  16. [24]

    Maria-Elena Nilsback and Andrew Zisserman. 2008. Automated flower classifica- tion over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing . IEEE, 722–729

    2008

  17. [25]

    Yassine Ouali, Adrian Bulat, Brais Matinez, and Georgios Tzimiropoulos. 2023. Black box few-shot adaptation for vision-language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 15534–15546

    2023

  18. [26]

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. 2012. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recognition . IEEE, 3498–3505

    2012

  19. [27]

    Thinh Phan, Khoa Vo, Duy Le, Gianfranco Doretto, Donald Adjeroh, and Ngan Le

  20. [28]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, and others. 2021. Learning transferable visual models from natural language supervision. In International conference on machine l...

    2021

  21. [29]

    Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. 2017. Learning mul- tiple visual domains with residual adapters. Advances in neural information processing systems 30 (2017)

    2017

  22. [30]

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Lud- wig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. 202...

    2022

  23. [31]

    K Soomro. 2012. UCF101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402 (2012)

    2012 arXiv

  24. [32]

    Xinyu Tian, Shu Zou, Zhaoyuan Yang, and Jing Zhang. 2024. ArGue: Attribute- Guided Prompt Tuning for Vision-Language Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 28578– 28587

    2024

  25. [33]

    Shihan Wu, Ji Zhang, Pengpeng Zeng, Lianli Gao, Jingkuan Song, and Heng Tao Shen. 2024. Skip Tuning: Pre-trained Vision-Language Models are Effective and Efficient Adapters Themselves. arXiv preprint arXiv:2412.11509 (2024)

    2024 arXiv

  26. [34]

    Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba

  27. [35]

    Lingxiao Yang, Ru-Yuan Zhang, Yanchen Wang, and Xiaohua Xie. 2024. MMA: Multi-Modal Adapter for Vision-Language Models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2024

  28. [36]

    Hantao Yao, Rui Zhang, and Changsheng Xu. 2023. Visual-Language Prompt Tuning With Knowledge-Guided Context Optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 6757– 6767

    2023

  29. [37]

    Hantao Yao, Rui Zhang, and Changsheng Xu. 2024. TCP:Textual-based Class- aware Prompt tuning for Visual-Language Model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 23438–23448

    2024

  30. [38]

    Lewei Yao, Runhui Huang, Lu Hou, Guansong Lu, Minzhe Niu, Hang Xu, Xiaodan Liang, Zhenguo Li, Xin Jiang, and Chunjing Xu. 2022. FILIP: Fine-grained Inter- active Language-Image Pre-Training.. In International Conference on Learning Representations (ICLR)

    2022

  31. [39]

    Maxime Zanella and Ismail Ben Ayed. 2024. On the test-time zero-shot gen- eralization of vision-language models: Do we really need prompt learning?. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion. 23783–23793

    2024

  32. [40]

    Xiaohua Zhai, Xiao Wang, Basil Mustafa, Andreas Steiner, Daniel Keysers, Alexan- der Kolesnikov, and Lucas Beyer. 2022. Lit: Zero-shot transfer with locked-image text tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 18123–18133. Con...

    2022

  33. [41]

    Ji Zhang, Shihan Wu, Lianli Gao, Heng Tao Shen, and Jingkuan Song. 2024. DePT: Decoupled Prompt Tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 12924–12933

    2024

  34. [42]

    Renrui Zhang, Wei Zhang, Rongyao Fang, Peng Gao, Kunchang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li. 2022. Tip-Adapter: Training-Free Adaption of CLIP for Few-Shot Classification.. In European Conference on Computer Vision (ECCV) . 493–510

    2022

  35. [43]

    Cairong Zhao, Yubin Wang, Xinyang Jiang, Yifei Shen, Kaitao Song, Dongsheng Li, and Duoqian Miao. 2024. Learning Domain Invariant Prompt for Vision- Language Models. IEEE Transactions on Image Processing 33 (2024), 1348–1360

    2024

  36. [44]

    Wangbo Zhao, Jiasheng Tang, Yizeng Han, Yibing Song, Kai Wang, Gao Huang, Fan Wang, and Yang You. 2024. Dynamic Tuning Towards Parameter and In- ference Efficiency for ViT Adaptation.. In Conference on Neural Information Processing Systems (NeurIPS)

    2024

  37. [45]

    Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chunyuan Li, Noel Codella, Liu- nian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, et al. 2022. Regionclip: Region-based language-image pretraining. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern rec...

    2022

  38. [46]

    Chong Zhou, Chen Change Loy, and Bo Dai. 2022. Extract free dense labels from clip. In European Conference on Computer Vision . Springer, 696–712

    2022

  39. [47]

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022. Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 16816–16825

    2022

  40. [48]

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022. Learning to Prompt for Vision-Language Models. International Journal of Computer Vision 130, 9 (2022), 2337–2348

    2022

  41. [49]

    Beier Zhu, Yulei Niu, Yucheng Han, Yue Wu, and Hanwang Zhang. 2023. Prompt- aligned gradient for prompt tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 15659–15669

    2023

  42. [2010]

    In 2010 IEEE computer society conference on computer vision and pattern recognition

    Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE computer society conference on computer vision and pattern recognition . IEEE, 3485–3492

    2010

  43. [2013]

    arXiv preprint arXiv:1306.5151 (2013)

    Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151 (2013)

    2013 arXiv

  44. [2023]

    Advances in Neural Information Processing Systems 36 (2023), 72096–72109

    Language is not all you need: Aligning perception with language models. Advances in Neural Information Processing Systems 36 (2023), 72096–72109

    2023

  45. [2024]

    In Proceedings of the IEEE/CVF winter conference on applications of computer vision

    Zeetad: Adapting pretrained vision-language model for zero-shot end-to- end temporal action detection. In Proceedings of the IEEE/CVF winter conference on applications of computer vision . 7046–7055

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.