Pith. sign in

REVIEW 5 major objections 6 minor 91 references

Visual Instance-aware Prompt Tuning

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that replacing dataset-level visual prompts with instance-aware prompts—generated per input image and fused with shared prompts through PCA—improves parameter-efficient vision tuning, reporting new state-of-the-art…

desk verdict The method idea is plausible and the ablation is clean, but the paper's own tables give three different VTAB-1k averages—the headline SOTA claim does not hold. read the letter →

arxiv 2507.07796 v1 pith:YCYD7KAK submitted 2025-07-10 cs.CV cs.AI

Xi Xiao , Yunbei Zhang , Xingjian Li , Tianyang Wang , Xiao Wang , Yuxiang Wei , Jihun Hamm , Min Xu This is my paper
classification cs.CVcs.AI
keywords prompttuningvisiontransformerinstance-awarepromptsprincipalcomponentanalysisparameter-efficientfine-tuningfine-grainedclassificationVTAB-1kvisual
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Visual Prompt Tuning normally learns one set of prompt tokens for the whole dataset and freezes the backbone. The paper argues that this leaves accuracy on the table because images within a task vary enough that a single static prompt cannot serve all of them. It proposes ViaPT, which generates a small number of prompt tokens for each input image, sampling them from a Gaussian whose mean and standard deviation are predicted by a lightweight encoder, and combines these with the usual dataset-level prompts. Principal Component Analysis compresses the fused prompts so the token width stays fixed, and the discarded dimensions are re-learned at every layer. The reported result is consistent accuracy gains over strong baselines on FGVC, HTA, and VTAB-1k while tuning only about 0.66% of the backbone parameters.

What carries the argument

The carrying object is the instance prompt generator paired with PCA-based prompt propagation. The generator $g$ maps image tokens $E_0$ to $(\mu, \sigma) \in \mathbb{R}^{2d}$; per-instance tokens are formed by sampling $z \sim \mathcal{N}(0,I)$, then applying $p = z \cdot \sigma + \mu$, which keeps instance-aware prompts in the same scale as the dataset-level prompt initialization. At each layer after the first, the previous layer's prompt outputs are projected by PCA from dimension $d$ to $m$, then padded with $d-m$ freshly initialized learnable parameters before being fed into the transformer. This $m$ is the dial that interpolates between VPT-Deep ($m=0$, nothing carried forward) and VPT-Shallow ($m=d=768$, everything carried forward); the paper finds a moderate value, typically $m=128$, gives the best accuracy.

What would settle it

Rerun the Table 1 methods on VTAB-1k with five or more seeds and report per-seed accuracies; if the gap between ViaPT and the strongest baseline falls within one standard deviation, the claimed consistent advantage is not established. A second falsifier replaces the PCA projection with a random projection of the same dimension $m$ and checks whether the reported 76.36% mean accuracy survives the swap.

Watch

Extended reading notes

Core claim

The central claim is that dataset-level prompts are inherently suboptimal and that per-instance prompt generation removes the bottleneck. In ViaPT, a two-layer convolutional encoder reads the image tokens and outputs a mean and standard deviation per dimension; prompt tokens are then drawn as $z_i \cdot \sigma + \mu$ with $z_i \sim \mathcal{N}(0, I)$, regularized by a KL term. Those instance-aware tokens are concatenated with dataset-level tokens, and PCA projects the concatenated prompt stream down to $m$ dimensions, after which new learnable parameters fill the remaining $d-m$ dimensions so each token keeps its required width. With $m=0$ the scheme reduces to VPT-Deep and with $m=768$ it reduces to VPT-Shallow, so the two classic methods are corner cases of a single interpolation rather than separate designs. Experiments across 34 datasets report the best mean accuracies among the listed methods on FGVC (91.40%), HTA (92.20%), and VTAB-1k (76.36%) with ViT-Base/16.

Load-bearing premise

The load-bearing premise is that the reported baseline numbers are directly comparable to the paper's own runs; because the main table gives no standard deviations even though Appendix D says VTAB-1k results are averaged over three seeds, a gap of roughly half a point to SA2VP and VFPT could be run-to-run noise rather than a real improvement.

Editorial extensions

If this is right

  • On FGVC, HTA, and VTAB-1k, ViaPT reports the best mean accuracy among the compared methods while tuning roughly 0.66% of ViT-Base/16 parameters.
  • VPT-Shallow and VPT-Deep become two endpoints of one prompt-propagation formula, so the choice between them is replaced by a single dimension-reduction knob $m$.
  • Ablations attribute most of the gain to the combination: removing instance-aware prompts costs 1.33 points and removing PCA costs 2.01 points on VTAB-1k.
  • The same design carries over to Swin-Base and to MAE- and MoCo v3-pretrained ViTs, with gains in all VTAB-1k categories.
  • Inference variance from the sampling is handled by fixed sampling or multi-round averaging, so the deployed model gives stable predictions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the $m$-spectrum is as clean as claimed, one testable extension is to tune $m$ per task or per layer; the paper's observation that $m=128$ works across benchmarks suggests there may be a universal default.
  • The small win of probabilistic sampling over a fixed-token generator (89.1% vs 88.7% on CUB-200) hints that the sampling noise itself acts as a regularizer, but the comparison does not fully isolate parameter count, so a matched-budget study would settle that.
  • Because gains concentrate on fine-grained and domain-shifted tasks, one would expect the benefit to grow with within-class visual variance; a per-dataset meta-analysis could test this prediction without any new method.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes Visual Instance-aware Prompt Tuning (ViaPT), a parameter-efficient fine-tuning method that generates instance-aware prompts from input statistics via a probabilistic encoder and fuses them with dataset-level prompts, using PCA-based dimension reduction to propagate information across transformer layers. The authors also propose a conceptual unification of VPT-Shallow, VPT-Deep, and their method as corner cases of the PCA dimension parameter m. Experiments are reported on FGVC, HTA, and VTAB-1k using ViT-Base/16 and Swin-Base, along with ablations and qualitative analyses. The paper claims consistent state-of-the-art results across all benchmarks with a small parameter footprint.

Significance. The core idea—moving from static dataset-level prompts to instance-conditioned prompts while keeping a dataset-level component—is a reasonable and potentially useful direction for visual prompt tuning. The ablation structure in Table 4 is clean in principle: removing either component hurts accuracy, and the reported full model improves over both ablated variants. The probabilistic generator with KL regularization is a sensible way to limit parameter overhead. However, the quantitative claims as written are not reliable: multiple headline numbers are arithmetically inconsistent with the paper's own tables, and no standard deviations are reported despite Appendix D stating that three seeds are averaged. Because the central claim is that ViaPT achieves new state-of-the-art results, these inconsistencies are load-bearing. The method may still have value after correction, but the current manuscript does not support its main conclusion.

major comments (5)
  1. [Section 4.2, Table 1, Table 6, Appendix A.3] The VTAB-1k headline mean of 76.36% is inconsistent with the paper's own data. The category means reported in Table 1 (Natural 82.62, Specialized 85.22, Structured 61.25) give a weighted average of (82.62×7 + 85.22×4 + 61.25×8)/19 = 74.17%, not 76.36%. Averaging the per-dataset numbers in Table 6 gives 73.71% and a Natural subset mean of 81.34% rather than the reported 82.62%. Appendix A.3 states 81.91% for the same benchmark. Three different values appear for the same mean, and the two values consistent with the paper's tables place ViaPT below SA2VP (75.83) and VFPT (75.49), reversing the paper's headline claim. This must be resolved before the central result can be assessed.
  2. [Table 7 and Table 8] The reported FGVC and HTA means for Ours are not supported by the per-dataset rows in the same tables. In Table 7, the Ours row sums to 89.4+85.5+99.2+92.6+83.9 = 450.6, giving a mean of 90.12%, not 91.40%. In Table 8, the Ours row sums to 916.8, giving a mean of 91.68%, not 92.20%. These discrepancies affect the abstract and Section 4.2 claims of outperforming all baselines, and they also change the margin over SA2VP on HTA from 0.70% to about 0.18%.
  3. [Section 4.1, Section 4.2, Appendix D, Table 1] The reported trainable-parameter ratios are mutually inconsistent. Section 4.1 states 0.25%, Section 4.2 states 0.74%, Table 1 lists 0.66%, Appendix D and Table 16 state 0.31%, Table 11 at m=128 gives 371.2K/86.57M = 0.43%, and Table 12 gives 441.6K (0.51%). The parameter-efficiency claim is central to the paper, so the authors must specify exactly which components are counted (prompt tokens, generator, head, PCA/learnable complement parameters) and report one consistent number across the paper.
  4. [Section 3.3, Eqs. (10)-(11)] The PCA transformation Φ is not defined precisely enough for reproducibility. It is unclear whether the PCA projection is computed from global training-set statistics, per-batch statistics, or is a learnable linear layer, and how gradients flow through the PCA step during end-to-end training. Standard PCA via eigendecomposition has no learnable parameters and is not differentiable in the usual sense; if a fixed projection is estimated offline, the timing and procedure must be described. Without this, the claim that the whole pipeline is trained end-to-end is unverifiable.
  5. [Section 3.3, Figure 1] The statement that m=0 reduces Vi aPT to VPT-Deep is only a conceptual mapping, not an exact equivalence, because the full ViaPT still includes the instance-aware prompt generator at the first layer and the PCA propagation module only governs subsequent layers. This is acknowledged informally, but the text should state explicitly that the reduction applies to the propagation module alone, not to the full method, to avoid overclaiming.
minor comments (6)
  1. [Appendix D] Appendix D states that results are averaged over 3 runs with different seeds, but no standard deviations or per-seed results are reported anywhere. Given the small margins over some baselines (e.g., HTA 92.20 vs. 91.50 in Table 1, and even smaller after correcting the arithmetic), the absence of variance estimates makes the stated advantages difficult to distinguish from run-to-run noise.
  2. [Table 4] The row 'w/ Random Prompt' is not described in the text. Clarify whether this replaces the instance-aware prompt generator with random per-instance prompts, replaces PCA with a random projection, or something else.
  3. [Figure 1] Figure 1 lacks axis labels and a description of the experimental setting (dataset, number of prompt tokens, PCA dimension values). The 'random dimension reduction' curve referenced in the abstract and introduction is not defined or discussed in the main text.
  4. [Conclusion] There is a typo in the Conclusion: 'Based our proposed conceptual understanding' should read 'Based on our proposed conceptual understanding.'
  5. [References] References [11] and [20] appear to refer to the same E2VPT work with different author lists and formatting; please consolidate and correct. Also, the arXiv number for [76] is incomplete.
  6. [Table 6] The Natural subset mean in Table 6 is arithmetically incorrect: the listed seven values average to 81.34%, not 82.62%. This is part of the broader VTAB-1k inconsistency noted above.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ViaPT's empirical claims are tested against external benchmarks, and its conceptual VPT-corner-case mapping is not used to derive the reported gains.

full rationale

The paper's central claims are empirical: ViaPT is trained end-to-end on external benchmarks (FGVC, HTA, VTAB-1k) and compared with published baselines using held-out test splits. No fitted parameter is renamed as a prediction; the instance-aware generator and PCA propagation are ablated in Table 4, and their contributions are measured rather than assumed. The 'VPT-Deep/VPT-Shallow as corner cases' statement (Section 3.3, Eqs. 10-12) is a conceptual mapping (m=0 and m=d) and, although the m=d equivalence is approximate because a full-rank PCA rotation is not identical to propagating prompts unchanged, this mapping is not load-bearing for the reported accuracy numbers and does not reduce any result to its inputs. Self-citations (e.g., refs. [34], [41], [67], [68], [81], [82]) appear in related-work or reference lists and are not used as an external authority to force the method or to justify the main result. The internal inconsistency in the VTAB-1k averages (Table 6 per-dataset entries sum to about 73.71% for the 19 tasks, Table 1 reports 76.36%, and Appendix A.3 reports 81.91%) is a data-reporting/correctness problem; it is not a circular derivation, so per the review rules it does not raise the circularity score.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or conceptual entities; it is a neural architecture design. The main assumptions are standard PEFT assumptions plus a dataset-level PCA prior. The free parameters are all standard hyperparameters (m, beta, lambda, R) fitted on validation data. The paper is transparent about most of them, except the prompt token split which is only clear from Appendix A.6.

free parameters (4)
  • m (PCA reduced dimension) = 128
    The PCA dimension m controls how much previous-layer prompt information is carried forward. The paper reports that m=128 consistently gives the best performance across benchmarks (Figure 1, Section 3.3), and the ablation shows a peak at m=128, so this is a tuned hyperparameter fitted on validation data.
  • beta (KL weight) = 0.01
    The KL divergence weight in the loss (Eq. 9) is set to 0.01 in Appendix D. It controls the strength of the Gaussian prior and is a hyperparameter chosen by the authors.
  • lambda (number of instance-aware prompt tokens) = 25
    The paper splits the 50 prompt tokens into 25 instance-aware and 25 dataset-level tokens (Appendix A.6 and Table 10). The specific split is a design choice not derived from first principles.
  • R (number of inference rounds) = 5
    Multi-round inference averages R forward passes, set to 5 in Section 3.4. The paper reports this balances stability and efficiency; the value is chosen empirically.
assumptions (4)
  • domain assumption Pretrained ViT-Base/16 features are partially frozen and provide a sufficient representational base for prompt tuning.
    The entire method assumes the frozen backbone contains transferable features and that tuning prompts plus a small generator is sufficient for strong downstream accuracy. This is the standard PEFT assumption of the cited literature.
  • domain assumption The image tokens E0 are a sufficient input for the prompt generator.
    Section 3.2 states the generator consumes image tokens rather than raw images to reduce overhead, assuming E0 retains enough instance-specific information to produce useful mean/std statistics.
  • domain assumption PCA computed on the training set prompt features captures the dimensions most useful for downstream classification.
    Section 3.3 applies PCA to select the highest-variance prompt dimensions, assuming variance equals task-relevant information. The paper itself notes that this can suppress subtle but useful dimensions at extreme compression.
  • domain assumption The baseline numbers in tables are directly comparable.
    The paper compares to numbers published in prior papers, assuming the same experimental protocol and splits. No code or re-run of baselines is provided to guarantee comparability.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Visual Instance-aware Prompt Tuning." pith.science (2026). https://pith.science/paper/YCYD7KAK

@misc{pith2026250707796,
  author       = {Pith},
  title        = {Pith review of: Visual Instance-aware Prompt Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YCYD7KAK}},
  note         = {Machine review of arXiv:2507.07796}
}
read the original abstract

Visual Prompt Tuning (VPT) has emerged as a parameter-efficient fine-tuning paradigm for vision transformers, with conventional approaches utilizing dataset-level prompts that remain the same across all input instances. We observe that this strategy results in sub-optimal performance due to high variance in downstream datasets. To address this challenge, we propose Visual Instance-aware Prompt Tuning (ViaPT), which generates instance-aware prompts based on each individual input and fuses them with dataset-level prompts, leveraging Principal Component Analysis (PCA) to retain important prompting information. Moreover, we reveal that VPT-Deep and VPT-Shallow represent two corner cases based on a conceptual understanding, in which they fail to effectively capture instance-specific information, while random dimension reduction on prompts only yields performance between the two extremes. Instead, ViaPT overcomes these limitations by balancing dataset-level and instance-level knowledge, while reducing the amount of learnable parameters compared to VPT-Deep. Extensive experiments across 34 diverse datasets demonstrate that our method consistently outperforms state-of-the-art baselines, establishing a new paradigm for analyzing and optimizing visual prompts for vision transformers.

Figures

Figures reproduced from arXiv: 2507.07796 by the authors.

Figure 1
Figure 1. Balancing information flow and instance aware [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed ViaPT framework. Left: Conventional VPT uses dataset-level learnable prompts that remain [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Grad-CAM heatmaps on CUB-200 samples. Com [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: t-SNE visualization on CUB-200. The proposed Vi [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Prompt token similarity heatmaps. Compared with [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Mutual information across transformer layers. Our model consistently increases label correlation ( [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Per-class accuracy comparison on CUB-200. We compare the top-10 classes between VPT-Deep and our method. Across [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

91 extracted references · 57 canonical work pages

  1. [1]

    Reduan Achtibat, Maximilian Dreyer, Ilona Eisenbraun, Sebastian Bosse, Thomas Wiegand, Wojciech Samek, and Sebastian Lapuschkin. 2023. From attribution maps to human-understandable explanations through concept relevance propa- gation.Nature Machine Intelligence5, 9 (2023), 1006–1019

    2023

  2. [2]

    Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normaliza- tion.arXiv preprint arXiv:1607.06450(2016)

    arXiv 2016

  3. [3]

    Hyojin Bahng, Ali Jahanian, Swami Sankaranarayanan, and Phillip Isola. 2022. Exploring visual prompts for adapting large-scale models

    2022

  4. [4]

    Hyojin Bahng, Ali Jahanian, Swami Sankaranarayanan, and Phillip Isola. 2022. Visual prompting: Modifying pixel space to adapt pre-trained models.arXiv preprint arXiv:2203.17274(2022)

    arXiv 2022

  5. [5]

    Emmanuel Bengio, Pierre-Luc Bacon, Joelle Pineau, and Doina Precup. 2015. Conditional computation in neural networks for faster models.arXiv preprint arXiv:1511.06297(2015)

    arXiv 2015

  6. [6]

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. 2014. Food-101–mining discriminative components with random forests. InECCV

    2014

  7. [7]

    Han Cai, Chuang Gan, Ligeng Zhu, and Song Han. 2020. Tinytl: Reduce memory, not parameters for efficient on-device learning

    2020

  8. [8]

    Shoufa Chen, Chuang Ge, Zhiqiang Tong, Jianzhuang Wang, Yang Song, Jianmin Wang, and Ping Luo. 2022. Adaptformer: Adapting vision transformers for scalable visual recognition.arXiv preprint arXiv:2205.13535(2022)

    arXiv 2022

Show all 91 references
  1. [9]

    Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. 2020. Improved baselines with momentum contrastive learning

    2020

  2. [10]

    Xinlei Chen, Saining Xie, and Kaiming He. 2021. An empirical study of training self-supervised vision transformers

    2021

  3. [11]

    Han Cheng, Wang Qifan, Cui Yiming, Cao Zhiwen, Wang Wenguan, Qi Siyuan, and Liu Dongfang. 2023. E2VPT: An Effective and Efficient Approach for Visual Prompt Tuning. InInternational Conference on Computer Vision (ICCV)

    2023

  4. [12]

    Campolongo, Daniel Rubenstein, Charles V

    Arpita Chowdhury, Dipanjyoti Paul, Zheda Mai, Jianyang Gu, Ziheng Zhang, Kazi Sajeed Mehrab, Elizabeth G. Campolongo, Daniel Rubenstein, Charles V. Stewart, Anuj Karpatne, Tanya Berger-Wolf, Yu Su, and Wei-Lun Chao. 2025. Prompt-CAM: Making Vision Transformers Interpretable fo...

    2025

  5. [13]

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and An- drea Vedaldi. 2014. Describing textures in the wild. InCVPR

    2014

  6. [14]

    Rahul Das, Yahel Dukler, Avinash Ravichandran, and Ajay Swaminathan. 2023. Learning Expressive Prompting With Residuals for Vision Transformers. In CVPR

    2023

  7. [15]

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. InCVPR

    2009

  8. [16]

    Shaohua Dong, Yunhe Feng, Qing Yang, Yan Huang, Dongfang Liu, and Heng Fan

  9. [17]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprin...

    2020 arXiv

  10. [18]

    Timnit Gebru, Jonathan Krause, Yilun Wang, Duyun Chen, Jia Deng, Erez Lieber- man Aiden, and Li Fei-Fei. 2017. Fine-grained car detection for visual census estimation. InAAAI Conference on Artificial Intelligence

    2017

  11. [19]

    Koustava Goswami, Srikrishna Karanam, Prateksha Udhayanan, KJ Joseph, and Balaji Vasan Srinivasan. 2024. Copl: Contextual prompt learning for vision- language understanding. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 18090–18098

    2024

  12. [20]

    Cheng Han, Qifan Wang, Yiming Cui, Zhiwen Cao, Wenguan Wang, Siyuan Qi, and Dongfang Liu. 2023. E2vpt: An effective and efficient approach for visual prompt tuning

    2023

  13. [21]

    Chunyuan Han, Qi Wang, Yuxin Cui, Zhiqian Cao, Wenqing Wang, Shiyu Qi, and Ding Liu. 2023. E^ 2VPT: An Effective and Efficient Approach for Visual Prompt Tuning.arXiv preprint arXiv:2307.13770(2023)

    2023 arXiv

  14. [22]

    Cheng Han, Qifan Wang, Yiming Cui, Wenguan Wang, Lifu Huang, and Dongfang Liu. 2024. Facing the Elephant in the Room: Visual Prompt Tuning or Full Finetuning?. InInternational Conference on Learning Representations (ICLR)

    2024

  15. [23]

    Dongchen Han, Xuran Pan, Yizeng Han, Shiji Song, and Gao Huang. 2023. Flatten transformer: Vision transformer using focused linear attention. InProceedings of the IEEE/CVF international conference on computer vision. 5961–5971

    2023

  16. [24]

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick

  17. [25]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. InCVPR

    2016

  18. [26]

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. 2019. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification.IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing12, 7 (2019), 2217–2226

    2019

  19. [27]

    Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. 2017. beta-vae: Learning basic visual concepts with a constrained variational framework. In International conference on learning representations

    2017

  20. [28]

    Neil Houlsby et al. 2019. Parameter-efficient transfer learning for NLP. InICML

    2019

  21. [29]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. InICLR

    2022

  22. [30]

    Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger

  23. [31]

    Mingzhen Huang, Jingru Zhang, Xiaodan Liang, and Hao Wang. 2023. DAM-VP: Adaptive Meta-Learning for Visual Prompt Tuning in Domain-Adaptive Vision Transformers. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    2023

  24. [32]

    Qingyao Huang, Xingchen Dong, Dantong Chen, Weiwei Zhang, Fuzhen Wang, Gang Hua, and Nenghai Yu. 2023. Diversity-Aware Meta Visual Prompting. In CVPR. ACM MM ’25, October 27–31, 2025, Dublin, Ireland Xiao et al

    2023

  25. [33]

    Eugenia Iofinova, Alexandra Peste, Mark Kurtz, and Dan Alistarh. 2022. How well do sparse imagenet models transfer?. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12266–12276

    2022

  26. [34]

    Yingrui Ji, Xi Xiao, Gaofei Chen, Hao Xu, Chenrui Ma, Lijing Zhu, Aokun Liang, and Jiansheng Chen. 2025. CIBR: Cross-modal Information Bottle- neck Regularization for Robust CLIP Generalization. arXiv:2503.24182 [cs.CV] https://arxiv.org/abs/2503.24182

    2025 arXiv

  27. [35]

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. 2022. Visual prompt tuning. InECCV

    2022

  28. [36]

    Can Jin, Ying Li, Mingyu Zhao, Shiyu Zhao, Zhenting Wang, Xiaoxiao He, Ligong Han, Tong Che, and Dimitris N Metaxas. [n. d.]. LoR-VP: Low-Rank Visual Prompting for Efficient Vision Model Adaptation. InThe Thirteenth International Conference on Learning Representations

  29. [37]

    Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Li Fei-Fei. 2011. Novel dataset for fine-grained image categorization. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 109–116

    2011

  30. [38]

    Diederik P Kingma and Max Welling. 2013. Auto-Encoding Variational Bayes. arXiv preprint arXiv:1312.6114(2013)

    2013 arXiv

  31. [39]

    Alex Krizhevsky and Geoffrey Hinton. 2009. Learning multiple layers of features from tiny images. InTechnical report, University of Toronto

    2009

  32. [40]

    Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning.arXiv preprint arXiv:2104.08691(2021)

    2021 arXiv

  33. [41]

    Hengjia Li, Lifan Jiang, Xi Xiao, Tianyang Wang, Hongwei Yi, Boxi Wu, and Deng Cai. 2025. MagicID: Hybrid Preference Optimization for ID-Consistent and Dynamic-Preserved Video Customization.arXiv preprint arXiv:2503.12689 (2025)

    2025 arXiv

  34. [42]

    Hengjia Li, Yang Liu, Yibo Zhao, Haoran Cheng, Yang Yang, Linxuan Xia, Zekai Luo, Qibo Qiu, Boxi Wu, Tu Zheng, et al. 2024. GCA-3D: Towards Generalized and Consistent Domain Adaptation of 3D Generators.arXiv preprint arXiv:2412.15491 (2024)

    2024 arXiv

  35. [43]

    Wanhua Li, Xiaoke Huang, Jiwen Lu, Jianjiang Feng, and Jie Zhou. 2021. Learning probabilistic ordinal embeddings for uncertainty-aware regression. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 13896– 13905

    2021

  36. [44]

    Lanlan Liu and Jia Deng. 2018. Dynamic deep neural networks: Optimizing accuracy-efficiency trade-offs by selective execution. InProceedings of the AAAI conference on artificial intelligence, Vol. 32

    2018

  37. [45]

    Ze Liu, Yutong Lin, Yutong Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin transformer: Hierarchical vision transformer using shifted windows. InICCV

    2021

  38. [46]

    Zichen Liu, Yuxin Peng, and Jiahuan Zhou. 2024. InsVP: Efficient Instance Visual Prompting from Image Itself. InProceedings of the 32nd ACM International Conference on Multimedia. 6443–6452

    2024

  39. [47]

    Zheda Mai, Ping Zhang, Cheng-Hao Tu, Hong-You Chen, Quang-Huy Nguyen, Li Zhang, and Wei-Lun Chao. 2025. Lessons and Insights from a Unifying Study of Parameter-Efficient Fine-Tuning (PEFT) in Visual Recognition. InProceedings of the Computer Vision and Pattern Recognition Conf...

    2025

  40. [48]

    Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and An- drew Y Ng. 2011. Reading digits in natural images with unsupervised feature learning

    2011

  41. [49]

    Maria-Elena Nilsback and Andrew Zisserman. 2008. Automated flower classifica- tion over a large number of classes. InProceedings of the Indian Conference on Computer Vision, Graphics & Image Processing. 722–729

    2008

  42. [50]

    Changdae Oh, Hyeji Hwang, Hee-young Lee, YongTaek Lim, Geunyoung Jung, Jiyoung Jung, Hosik Choi, and Kyungwoo Song. 2023. BlackVIP: Black-Box Visual Prompting for Robust Transfer Learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVP...

    2023

  43. [51]

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Des- maison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, L...

    2019

  44. [52]

    Xiang Pei et al. 2024. SA2VP: Spatially Aware Visual Prompting for Fine-Grained Image Recognition.arXiv preprint arXiv:2401.01567(2024)

    2024 arXiv

  45. [53]

    Silviu Pitis, Michael R Zhang, Andrew Wang, and Jimmy Ba. 2023. Boosted prompt ensembles for large language models.arXiv preprint arXiv:2304.05970 (2023)

    2023 arXiv

  46. [54]

    Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. 2017. Learning multi- ple visual domains with residual adapters. InNeurIPS, Vol. 30

    2017

  47. [55]

    Li Ren, Chen Chen, Liqiang Wang, and Kien Hua. 2025. DA-VPT: Semantic- Guided Visual Prompt Tuning for Vision Transformers. InProceedings of the Computer Vision and Pattern Recognition Conference. 4353–4363

    2025

  48. [56]

    Chunqing Ruan and Hongjian Wang. 2023. Dynamic visual prompt tuning for parameter efficient transfer learning. InChinese Conference on Pattern Recognition and Computer Vision (PRCV). Springer, 293–303

    2023

  49. [57]

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedan- tam, Devi Parikh, and Dhruv Batra. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization. InICCV

    2017

  50. [58]

    Guohao Chen Pengcheng Wu Peilin Zhao Shuaicheng Niu, Chunyan Miao. 2024. Test-Time Model Adaptation with Only Forward Passes. InThe International Conference on Machine Learning

    2024

  51. [59]

    Johannes Stallkamp, Marc Schlipsing, Jan Salmen, and Christian Igel. 2012. Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural networks32, 323–332

    2012

  52. [60]

    Cheng-Hao Tu, Zheda Mai, and Wei-Lun Chao. 2023. Visual Query Tuning: Towards Effective Usage of Intermediate Representations for Parameter and Memory Efficient Transfer Learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 7725–7735

    2023

  53. [61]

    Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. JMLR9, 11

    2008

  54. [62]

    Grant Van Horn, Steve Branson, Ryan Farrell, Scott Haber, Jessie Barry, Panos Ipeirotis, Pietro Perona, and Serge Belongie. 2015. Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection. InProceedings o...

    2015

  55. [63]

    Bastiaan S Veeling, Jasper Linmans, Jim Winkens, Taco Cohen, and Max Welling. 2018. Rotation Equivariant CNNs for Digital Pathology. (June 2018). arXiv:1806.03962 [cs.CV]

    2018 arXiv

  56. [64]

    Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie

  57. [65]

    Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Be- longie. 2011. The Caltech-UCSD Birds-200-2011 Dataset. InCalifornia Institute of Technology

    2011

  58. [66]

    Taowen Wang, Yiyang Liu, James Chenhao Liang, junhan zhao, Yiming Cui, Yuning Mao, Shaoliang Nie, Jiahao Liu, Fuli Feng, Zenglin Xu, Cheng Han, Lifu Huang, Qifan Wang, and Dongfang Liu. 2024. M2PT: Multimodal Prompt Tuning for Zero-shot Instruction Learning. arXiv:2409.15657 [...

    2024 arXiv

  59. [67]

    Xi Xiao, Wentao Wang, Jiacheng Xie, Lijing Zhu, Gaofei Chen, Zhengji Li, Tianyang Wang, and Min Xu. 2025. Hgtdp-dta: Hybrid graph-transformer with dynamic prompt for drug-target binding affinity prediction. InInternational Conference on Neural Information Processing. Springer, 340–354

    2025

  60. [68]

    Xi Xiao, Yunbei Zhang, Yanshuh Li, Xingjian Li, Tianyang Wang, Jihun Hamm, Xiao Wang, and Min Xu. 2025. Visual Variational Autoencoder Prompt Tuning. arXiv:2503.17650 [cs.CV] https://arxiv.org/abs/2503.17650

    2025 arXiv

  61. [69]

    Zehao Xiao and Cees G. M. Snoek. 2024. Beyond Model Adaptation at Test Time: A Survey. arXiv:2411.03687 [cs.LG] https://arxiv.org/abs/2411.03687

    2024 arXiv

  62. [70]

    Seungryong Yoo, Eunji Kim, Dahuin Jung, Jungbeom Lee, and Sungroh Yoon

  63. [71]

    Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. 2014. How transferable are features in deep neural networks?

    2014

  64. [72]

    Tianshu Yu, Min Yang, and Xiaoyan Zhao. 2022. Dependency-aware Prototype Learning for Few-shot Relation Classification. InProceedings of the 29th Inter- national Conference on Computational Linguistics, Nicoletta Calzolari, Chu-Ren Huang, Hansaem Kim, James Pustejovsky, Leo Wa...

    2022

  65. [73]

    Xinlei Yu, Ahmed Elazab, Ruiquan Ge, Jichao Zhu, Lingyan Zhang, Gangyong Jia, Qing Wu, Xiang Wan, Lihua Li, and Changmiao Wang. 2025. ICH-PRNet: a cross-modal intracerebral haemorrhage prognostic prediction method using joint-attention interaction mechanism.Neural Networks184 ...

    2025

  66. [74]

    Xinlei Yu, Chanmiao Wang, Hui Jin, Ahmed Elazab, Gangyong Jia, Xiang Wan, Changqing Zou, and Ruiquan Ge. 2025. CRISP-SAM2: SAM2 with Cross-Modal In- teraction and Semantic Prompting for Multi-Organ Segmentation.arXiv preprint arXiv:2506.23121(2025)

    2025 arXiv

  67. [75]

    Improving visual prompt tuning for self-supervised vision transformers

  68. [76]

    Runjia Zeng, Cheng Han, Qifan Wang, Chunshu Wu, Tong Geng, Lifu Huangg, Ying Nian Wu, and Dongfang Liu. 2024. Visual fourier prompt tuning.Advances in Neural Information Processing Systems37 (2024), 5552–5585

    2024

  69. [77]

    Wang Zeng, Sheng Jin, Wentao Liu, Chen Qian, Ping Luo, Wanli Ouyang, and Xiaogang Wang. 2022. Not all tokens are equal: Human-centric visual analysis via token clustering transformer. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 11101–11111

    2022

  70. [79]

    Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov, Pierre Ruyssen, Carlos Riquelme, Mario Lucic, Josip Djolonga, Alexandre Sablayrolles Pinto, Mario Neu- mann, Alexey Dosovitskiy, et al. 2019. A large-scale study of representation learn- ing with the visual task adaptation b...

    2019 arXiv

  71. [80]

    Yuhang Zang, Wei Li, Kaiyang Zhou, Chen Huang, and Chen Change Loy. 2022. Unified vision and language prompt learning

    2022

  72. [81]

    Yunbei Zhang, Akshay Mehra, and Jihun Hamm. 2025. OT-VP: Optimal Transport- Guided Visual Prompting for Test-Time Adaptation. InProceedings of the Winter Conference on Applications of Computer Vision (W ACV). 1122–1132

    2025

  73. [82]

    Yunbei Zhang, Akshay Mehra, Shuaicheng Niu, and Jihun Hamm. 2025. DPCore: Dynamic Prompt Coreset for Continual Test-Time Adaptation. InForty-second International Conference on Machine Learning. https://openreview.net/forum? id=A6zDim0rQf

    2025

  74. [83]

    Zangwei Zheng, Xiangyu Yue, Kai Wang, and Yang You. 2022. Prompt Vision Transformer for Domain Generalization. arXiv:2208.08914 [cs.CV] https://arxiv. org/abs/2208.08914

    2022 arXiv

  75. [84]

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022. Conditional prompt learning for vision-language models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 16816–16825

    2022

  76. [85]

    Jeffrey O Zhang, Alexander Sax, Amir Zamir, Leonidas Guibas, and Jitendra Malik

  77. [86]

    Beier Zhu, Yulei Niu, Yucheng Han, Yue Wu, and Hanwang Zhang. 2023. Prompt- aligned gradient for prompt tuning. InProceedings of the IEEE/CVF international conference on computer vision. 15659–15669. ACM MM ’25, October 27–31, 2025, Dublin, Ireland Xiao et al. A Appendix Datas...

    2023

  78. [91]

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022. Condi- tional Prompt Learning for Vision-Language Models. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2022

  79. [2011]

    The caltech-ucsd birds-200-2011 dataset

    2011

  80. [2017]

    InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    Densely connected convolutional networks. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 4700–4708

  81. [2020]

    Side-tuning: a baseline for network adaptation via additive side networks

  82. [2022]

    Masked autoencoders are scalable vision learners

  83. [2023]

    Efficient Multimodal Semantic Segmentation via Dual-Prompt Learning

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.