Pith. sign in

REVIEW 2 major objections 10 minor 73 references

Differentiable Prompt Learning for Vision Language Models

T0 review · 2 major / 10 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Automatic search over per-layer prompt lengths beats fixed prompt designs

desk verdict The search idea is real and the experiments are careful, but the missing cross-attention ablation means the 2.60% gain cannot yet be attributed to the automatic search. read the letter →

arxiv 2501.00457 v1 pith:BNTAS5CB submitted 2024-12-31 cs.LG cs.AIcs.CLcs.CV

Zhenhan Huang , Tejaswini Pedapati , Pin-Yu Chen , Jianxi Gao This is my paper · ORCID
classification cs.LGcs.AIcs.CLcs.CV MSC 68T0768T45
keywords differentiablepromptlearningcontinuouspromptsvision-languagemodelsCLIPconfigurationsearchfew-shotbileveloptimizationneuralarchitecture
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether the hand-chosen settings of deep prompt learning, one fixed context length applied to every layer, are actually optimal, and it answers no. The authors propose differentiable prompt learning (DPL), a search procedure that automatically picks a context length for the continuous prompts inserted at each transformer layer of CLIP, in the text branch and image branch separately. Using only 16-shot data for the search, DPL finds configurations that vary by dataset and by layer, and these automatic configurations raise average test accuracy by 2.60% on 11 benchmark datasets over strong prompt-learning baselines. This matters because prompt learning is the main parameter-efficient way to adapt frozen foundation models, and its main hyperparameters have so far been set by hand.

What carries the argument

The load-bearing mechanism is a differentiable relaxation of the discrete choice of context length per layer, borrowed from differentiable neural architecture search: each layer has candidate prompt lengths (0, 2, 4, or 6), a trainable alpha weight per candidate is softmaxed into a mixing weight, and the layer output is a convex combination of the outputs under each candidate. To make the candidates mixable despite different lengths, DPL replaces self-attention with cross-attention in the transformer blocks, using the prompt tokens as keys and values while the original tokens provide the queries. The search stage is bilevel, updating alpha on the validation loss and the prompt embeddings on the training loss; after convergence the argmax defines the subprompt used in a standard training stage, optionally with knowledge distillation toward the zero-shot CLIP predictions.

What would settle it

Train DPL's final-stage model with a fixed context length on every layer (for example, the average searched length) while keeping the cross-attention blocks and all training hyperparameters identical to DPL; if the fixed-length cross-attention model matches DPL's reported 81.71% average accuracy on the 11 datasets, then the automatic search contributed nothing beyond the attention change.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the optimal continuous-prompt configuration is not uniform: the context length that should be added differs from layer to layer, and the best configuration for the text branch differs from that for the image branch. DPL makes this choice differentiable by relaxing it with a softmax over candidate context lengths, solving a bilevel problem where the prompt vectors minimize training loss and the length-choice parameters minimize validation loss, then committing to the argmax per layer and fine-tuning prompts under that configuration. The paper reports that this automatic design outperforms fixed-length deep-prompt baselines, with the largest gains on datasets with large distribution shift such as EuroSAT and FGVCAircraft, and that the searched configurations are dataset-dependent, supporting the claim that manual homogeneous designs leave performance on the table.

Load-bearing premise

The paper credits the accuracy gain to the automatic choice of prompt lengths, but every compared baseline uses self-attention while DPL uses cross-attention in the transformer blocks, and no experiment isolates the attention change from the search, so the search alone is not yet proven to be the source of the improvement.

Editorial extensions

If this is right

  • If DPL is right, fixed context length and depth, the two main hyperparameters of deep prompt tuning, need not be chosen by hand; a cheap search over per-layer lengths can replace them.
  • The dataset-dependence of the found configurations implies that transfer under distribution shift benefits from letting prompt depth adapt to the gap between pretraining and target data, rather than using one global depth.
  • Because DPL only changes the prompt configuration, it can be combined with other prompt-learning refinements, such as optimal-transport alignment or gradient-aligned distillation, and still contribute its gains.
  • The asymmetric text and image branch configurations suggest that the two modalities need different amounts of adaptation, a degree of freedom that one-prompt-depth-fits-all methods cannot express.
  • At inference time the method adds no cost relative to other prompt-tuning methods, since only the chosen prompts remain in the final model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The search operates per layer, but nothing restricts it to the four candidate lengths {0, 2, 4, 6}; enlarging the candidate set or allowing per-head prompt lengths could push the same mechanism further, though search cost grows with the space.
  • The cross-attention modification is itself a design change; the paper does not ablate it against a fixed-length cross-attention baseline, so the relative contribution of the search versus the attention change is an open question a follow-up could settle.
  • The same differentiable configuration search could be applied to other frozen-foundation adaptation methods, such as visual prompt tuning for detection or prompt tuning for large language models, wherever a discrete hyperparameter controls per-layer inserted vectors.
  • Dataset-dependent prompt configurations, if they generalize across training seeds, could serve as cheap fingerprints of distribution shift: the per-layer length profile might predict how far a downstream task is from the pretraining distribution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 10 minor

Summary. The paper proposes DPL, a two-stage differentiable method for automatically selecting per-layer context lengths of deep continuous prompts for vision-language models like CLIP. In the search stage, a supernet-style model with softmax-weighted context-length options is trained via alternating updates of continuous prompts and architecture parameters alpha, producing an alpha matrix; the argmax per layer yields a discrete prompt configuration. In the training stage, that configuration is trained from scratch with cross-entropy loss and an optional KL distillation term. Experiments on 11 datasets with ViT-B/16 CLIP report an average accuracy of 81.71%, exceeding CoCoOp, PLOT, ProGrad, MaPLe, zero-shot CLIP, and linear probe. The paper also analyzes alpha-matrix convergence, few-shot behavior, and computational complexity.

Significance. If the empirical gains are truly due to the automatic configuration search, DPL is a useful contribution that removes manual per-layer prompt-length and depth design and is conceptually orthogonal to other prompt designs, making it a plausible building block for future methods. The manuscript has several strengths: the problem is well motivated, the differentiable relaxation is a natural extension of DARTS, the benchmarks cover a standard suite of 11 datasets, and the alpha-matrix evolution analysis is thoughtful. However, the headline claim rests on a confounded comparison: DPL also changes the internal attention mechanism from self-attention to cross-attention (Eqs. 4-5), and the only ablation does not control for this change or for the value of the search itself. Two targeted controlled experiments would substantially raise confidence in the central claim.

major comments (2)
  1. [§3.1, Eqs. (4)-(5); Table 1] The reported 2.60% average accuracy gain may be attributable to the change from self-attention to cross-attention rather than to the automatic context-length search. All baselines (CoCoOp, PLOT, ProGrad, MaPLe) use standard self-attention blocks, while DPL uses cross-attention in every transformer block during both the search and training stages. No experiment isolates this change. The authors should add a fixed-length deep prompt baseline with the same cross-attention mechanism (e.g., context length 4 at all layers, no alpha search) and report its test accuracy. If such a baseline matches DPL, the automatic search contributes nothing; if it does not, the contribution of the search is supported. This comparison is load-bearing for the paper's central claim.
  2. [§4.4, Table 2] The only ablation compares DPL to a shallow (depth-1, context-length 16) cross-attention prompt. This does not control for prompt depth or for the homogeneity or heterogeneity of the configuration, so it cannot demonstrate that the automatic heterogeneous configuration is superior to a manually designed deep prompt with the same attention mechanism. The authors should additionally report (i) a fixed-length deep prompt with cross-attention (e.g., length 4 at every layer) and (ii) a randomly selected heterogeneous configuration with matched total number of prompt tokens. Without these controls, the claim that the searched configuration is better than manually designed configurations is not empirically grounded.
minor comments (10)
  1. [Figure 2 caption] The caption says the row dimension is related to context length and the column to model depth, but Section 4.2 states the opposite. Please correct the inconsistency.
  2. [Abstract; §4.3] "Boosts the average test accuracy by 2.60%" is ambiguous. Please specify the comparison baseline (apparently ProGrad at 79.14%, not the average of all baselines) and state whether the value is in percentage points.
  3. [Section 6] The stated search-space size 2.81×10^15 does not match the actual candidate set. With t=4 options per layer, ℓ=12 layers per branch, and two independent branches, the total number of configurations is (4^12)^2 ≈ 2.8×10^14; if branches are counted separately, the number is 4^12 ≈ 1.7×10^7. Please correct the numerical value.
  4. [§3.1, Algorithm 1; Eqs. (8)-(9)] The optimization is described as bilevel, but Algorithm 1 uses single-loop alternating updates of alpha and E rather than a nested optimization. DARTS-style approximations are common, but the paper should explicitly acknowledge that this is an approximation to the bilevel optimum, not an exact solution.
  5. [§3.2] "Hypereparameter" is a typo for "hyperparameter" near Eq. (13). Also, the KL-divergence choice is given with no motivation; a sentence explaining why distillation is added would help.
  6. [§4.3] "Zero-sho CLIP" is a typo for "zero-shot CLIP."
  7. [Appendix A.5, Figures 4 and 7] The captions contain typos: "dataets" and "exibit" should be "datasets" and "exhibit."
  8. [§4.2, Definition 4.1; Appendix A.2] The "single-dominant" definition uses "≫" without a quantitative threshold; the later threshold T in the appendix is described in words but not given a value. A precise definition would improve reproducibility.
  9. [Table 1] DPL+KD has nearly identical average accuracy to DPL (81.74 vs 81.71) and is worse on several datasets (e.g., DTD, Pets, UCF, SUN397). Consider removing the KD variant from the headline results or providing an analysis of when it helps.
  10. [§2.1, Eq. (2)] The notation [x(l), E(l)] is overloaded; the authors should use explicit concatenation symbols, since the same bracket style is used for both concatenation and indexing elsewhere.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the searched prompt configuration is retrained from scratch before test evaluation, so the reported gains are not forced by the search objective; the cross-attention confound is a missing control, not a circular reduction.

full rationale

The paper's derivation chain is an empirical benchmark comparison rather than a fitted derivation. The DPL search stage (Eqs. 8-9) optimizes alpha on the validation loss, then the selected context lengths are used to retrain continuous prompts from scratch in Section 3.2, and test accuracy is reported on held-out splits in Table 1. No fitted parameter is renamed as a prediction: the alpha matrices select an architecture, and the final model is trained independently. The claim that the optimal configuration is dataset-dependent is supported by the converged alpha matrices in Figure 4 and Appendix A.1; although a per-dataset search can of course produce per-dataset configurations, the paper does not present this as a derivation from the search equations, and different datasets could in principle have converged to the same configuration. The one self-citation (Chen et al. 2023, with co-author Pin-Yu Chen) appears in the introduction as background on visual prompting and is not load-bearing for any of DPL's claims. The skeptic's strongest concern, that the cross-attention mechanism (Eqs. 4-5) rather than the automatic length search may explain the 2.60% gain, is a valid benchmarking and ablation gap (no fixed-length deep-prompt baseline with cross-attention is reported), but a missing control is not circular reasoning: it does not make the reported numbers equivalent to the method's inputs by construction. The KD claim is contradicted by the paper's own Table 1 (DPL 81.71 vs DPL+KD 81.74, within noise), which is an overclaim but again not a circular step. Overall, no equation-level reduction, fitted-input renaming, or load-bearing self-citation chain exists in the manuscript.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

No new physical or model entities are introduced; the supprompt, subprompt, and alpha matrices are optimization constructs, not free-floating entities.

free parameters (3)
  • Candidate context lengths = {0, 2, 4, 6}
    The search only chooses among these four hand-picked lengths per layer; the reported "optimal" configuration is relative to this set (Section 3.1, Section 6).
  • KD weight lambda = not reported
    Controls the KL distillation term in Eq. 13; the DPL+KD results in Table 1 depend on it and the paper does not state its value.
  • Search/training epochs and batch size = 60 search, 40 train, batch 4, LR 3.5e-3
    Fixed hyperparameters for all datasets (Section 4.1); results could shift with different settings.
assumptions (5)
  • domain assumption The frozen CLIP encoder provides a good feature space, so adapting only prompt vectors is sufficient for downstream tasks.
    The method updates only prompts and alpha while CLIP weights are frozen (Section 3.1); this is standard for prompt learning but unproven for the modified cross-attention block.
  • ad hoc to paper Cross-attention with Q from x and K/V from [E, x] preserves the pretrained transformer's function well enough for fair comparison with self-attention prompt methods.
    This architectural choice is introduced in Eqs. 4-5 and used throughout, but no baseline or ablation isolates its effect.
  • ad hoc to paper Single-loop alternating updates of alpha and E in Algorithm 1 approximate the bilevel optimum in Eqs. 8-9.
    The paper states the bilevel problem but does not analyze whether the alternating gradient descent converges to its solution.
  • domain assumption Argmax selection of the converged alpha matrix yields a prompt configuration that is near-optimal after retraining from scratch.
    Standard DARTS assumption; some alpha rows are not single-dominant (Section 4.2), so the selected configuration may be unstable.
  • ad hoc to paper The candidate set {0, 2, 4, 6} is expressive enough to contain near-optimal prompt lengths.
    No sensitivity analysis over alternative candidate sets is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Differentiable Prompt Learning for Vision Language Models." pith.science (2026). https://pith.science/paper/BNTAS5CB

@misc{pith2026250100457,
  author       = {Pith},
  title        = {Pith review of: Differentiable Prompt Learning for Vision Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BNTAS5CB}},
  note         = {Machine review of arXiv:2501.00457}
}
read the original abstract

Prompt learning is an effective way to exploit the potential of large-scale pre-trained foundational models. Continuous prompts parameterize context tokens in prompts by turning them into differentiable vectors. Deep continuous prompts insert prompts not only in the input but also in the intermediate hidden representations. Manually designed deep continuous prompts exhibit a remarkable improvement compared to the zero-shot pre-trained model on downstream tasks. How to automate the continuous prompt design is an underexplored area, and a fundamental question arises, is manually designed deep prompt strategy optimal? To answer this question, we propose a method dubbed differentiable prompt learning (DPL). The DPL method is formulated as an optimization problem to automatically determine the optimal context length of the prompt to be added to each layer, where the objective is to maximize the performance. We test the DPL method on the pre-trained CLIP. We empirically find that by using only limited data, our DPL method can find deep continuous prompt configuration with high confidence. The performance on the downstream tasks exhibits the superiority of the automatic design: our method boosts the average test accuracy by 2.60% on 11 datasets compared to baseline methods. Besides, our method focuses only on the prompt configuration (i.e. context length for each layer), which means that our method is compatible with the baseline methods that have sophisticated designs to boost the performance. The DPL method can be deployed to large language models or computer vision models at no cost.

Figures

Figures reproduced from arXiv: 2501.00457 by the authors.

Figure 1
Figure 1. (a) In the searching stage, continuous prompts with dif [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. (a) α matrices for the text branch. (b) α matrices for the image branch. α matrices are obtained at the epoch of 60. The row dimension is related to the context length of added continuous prompts. The column dimension is related to model depth, i.e. the number of transformer blocks. (c) The evolution of the α difference and the number of dominants. As the number of training epochs increases, the α matrix gradually c… view at source ↗
Figure 3
Figure 3. Original image and Grad-CAM visualization [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Evolution of α matrices using various datasets in the searching stage. Although α matrices have the same random initialization for the same text branch or image branch, the converged matrices are different for different datasets. It indicates that the prompt learning m…
Figure 5
Figure 5. Figure 5: The computational complexity of the optimal prompt con [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: The variation of the α difference in the searching process. The number of shots is 16, 8, 4, 2 and 1. When using fewer shots in the searching stage, the searching algorithm becomes less confident. There is a pronounced drop in α difference using 1-shot learning. 12 4 8…
Figure 7
Figure 7. Figure 7: Performance of few-shot learning on 11 datasets for various prompting methods. The DPL method requires a fair amount of data [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

73 extracted references · 39 canonical work pages

  1. [1]

    Hierarchical optimization: An introduc- tion

    [Anandalingam and Friesz, 1992] Gnana Anandalingam and Terry L Friesz. Hierarchical optimization: An introduc- tion. Annals of Operations Research, 34:1–11,

    work page 1992

  2. [3]

    , 2018; Xu et al

    Supprompt is analogous to Supernet while Subprompt is analogous to Sub- prompt in differentiable NAS [Liu et al. , 2018; Xu et al. , 2019; Dong and Yang, 2019; Liang et al., 2019; Zela et al., 2019; Chu et al., 2020; Yan et al., 2021]. Similarly, we use search space to group prompt configurations in the search- ing process. The computational cost of the s...

    work page 2018

  3. [5]

    Food-101–mining discriminative com- ponents with random forests

    [Bossard et al., 2014] Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101–mining discriminative com- ponents with random forests. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part VI 13 , pages 446–461. Springer,

    work page 2014

  4. [7]

    However, the DPL method be- comes less competitive when the number of shots is very limited (e.g

    When the number of shots is large, there is a significant performance boost for the DPL method. However, the DPL method be- comes less competitive when the number of shots is very limited (e.g. 1 and 2 shots). When the number of shots de- creases, all prompting methods exhibit a conspicuous perfor- mance drop. Most prompting methods assume at least 16- sh...

    work page 2024

  5. [9]

    Understanding and improving visual prompting: A label-mapping perspec- tive

    [Chen et al., 2023] Aochuan Chen, Yuguang Yao, Pin-Yu Chen, Yihua Zhang, and Sijia Liu. Understanding and improving visual prompting: A label-mapping perspec- tive. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 19133– 19143,

    work page 2023

  6. [11]

    Describing textures in the wild

    [Cimpoi et al., 2014] Mircea Cimpoi, Subhransu Maji, Ia- sonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pages 3606–3613,

    work page 2014

  7. [12]

    An overview of bilevel optimization

    [Colson et al., 2007] Benoˆıt Colson, Patrice Marcotte, and Gilles Savard. An overview of bilevel optimization. An- nals of operations research, 153:235–256,

    work page 2007

  8. [14]

    Imagenet: A large-scale hierarchical image database

    [Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pages 248–255. Ieee,

    2009

Show all 73 references
  1. [16]

    An image is worth 16x16 words: Transformers for image recognition at scale

    [Dosovitskiy et al., 2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Min- derer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition a...

    2020 arXiv

  2. [17]

    Learning generative visual models from few train- ing examples: An incremental bayesian approach tested on 101 object categories

    [Fei-Fei et al., 2004] Li Fei-Fei, Rob Fergus, and Pietro Per- ona. Learning generative visual models from few train- ing examples: An incremental bayesian approach tested on 101 object categories. In 2004 conference on computer vision and pattern recognition workshop, pages 1...

    2004

  3. [19]

    Bertese: Learning to speak to bert

    [Haviv et al., 2021] Adi Haviv, Jonathan Berant, and Amir Globerson. Bertese: Learning to speak to bert. arXiv preprint arXiv:2103.05327,

    2021 arXiv

  4. [22]

    Benchmarking neural network ro- bustness to common corruptions and perturbations

    [Hendrycks and Dietterich, 2019] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network ro- bustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261,

    2019 arXiv

  5. [23]

    Distilling the knowledge in a neural network

    [Hinton et al., 2015] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531,

    2015 arXiv

  6. [26]

    Visual prompt tuning

    [Jia et al., 2022] Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. In European Conference on Computer Vision, pages 709–727. Springer,

    2022

  7. [27]

    How can we know what language models know? Transactions of the Association for Com- putational Linguistics, 8:423–438,

    [Jiang et al., 2020] Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. How can we know what language models know? Transactions of the Association for Com- putational Linguistics, 8:423–438,

    2020

  8. [28]

    Less-forgetting learning in deep neural networks

    [Jung et al., 2016] Heechul Jung, Jeongwoo Ju, Minju Jung, and Junmo Kim. Less-forgetting learning in deep neural networks. arXiv preprint arXiv:1607.00122,

    2016 arXiv

  9. [29]

    Maple: Multi-modal prompt learn- ing

    [Khattak et al., 2023] Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fa- had Shahbaz Khan. Maple: Multi-modal prompt learn- ing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 19113– 19122,

    2023

  10. [30]

    Wilds: A benchmark of in-the-wild distribution shifts

    [Koh et al., 2021] Pang Wei Koh, Shiori Sagawa, Hen- rik Marklund, Sang Michael Xie, Marvin Zhang, Ak- shay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: A benchmark of in-the-wild distribution shifts. In Interna- tional conferen...

    2021

  11. [31]

    3d object representations for fine- grained categorization

    [Krause et al., 2013] Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine- grained categorization. In Proceedings of the IEEE inter- national conference on computer vision workshops, pages 554–561,

    2013

  12. [33]

    The power of scale for parameter-efficient prompt tuning

    [Lester et al., 2021] Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691,

    2021 arXiv

  13. [34]

    Prefix- tuning: Optimizing continuous prompts for generation

    [Li and Liang, 2021] Xiang Lisa Li and Percy Liang. Prefix- tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190,

    2021 arXiv

  14. [35]

    Darts+: Improved differentiable archi- tecture search with early stopping

    [Liang et al., 2019] Hanwen Liang, Shifeng Zhang, Jiacheng Sun, Xingqiu He, Weiran Huang, Kechen Zhuang, and Zhenguo Li. Darts+: Improved differentiable archi- tecture search with early stopping. arXiv preprint arXiv:1909.06035,

    2019 arXiv

  15. [36]

    Darts: Differentiable architecture search

    [Liu et al., 2018] Hanxiao Liu, Karen Simonyan, and Yim- ing Yang. Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09055,

    2018 arXiv

  16. [37]

    P-tuning v2: Prompt tuning can be comparable to fine- tuning universally across scales and tasks

    [Liu et al., 2021] Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. P-tuning v2: Prompt tuning can be comparable to fine- tuning universally across scales and tasks. arXiv preprint arXiv:2110.07602,

    2021 arXiv

  17. [38]

    Gpt understands, too

    [Liu et al., 2023] Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. Gpt understands, too. AI Open,

    2023

  18. [39]

    On surgical fine-tuning for language encoders

    [Lodha et al., 2023] Abhilasha Lodha, Gayatri Belapurkar, Saloni Chalkapurkar, Yuanming Tao, Reshmi Ghosh, Samyadeep Basu, Dmitrii Petrov, and Soundararajan Srinivasan. On surgical fine-tuning for language encoders. arXiv preprint arXiv:2310.17041,

    2023 arXiv

  19. [40]

    Fine- grained visual classification of aircraft

    [Maji et al., 2013] Subhransu Maji, Esa Rahtu, Juho Kan- nala, Matthew Blaschko, and Andrea Vedaldi. Fine- grained visual classification of aircraft. arXiv preprint arXiv:1306.5151,

    2013 arXiv

  20. [42]

    Automated flower classification over a large number of classes

    [Nilsback and Zisserman, 2008] Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing , pages 722–729. IEEE,

    2008

  21. [44]

    Task-specific skill localization in fine-tuned language models

    [Panigrahi et al., 2023] Abhishek Panigrahi, Nikunj Saun- shi, Haoyu Zhao, and Sanjeev Arora. Task-specific skill localization in fine-tuned language models. In Interna- tional Conference on Machine Learning , pages 27011– 27033. PMLR,

    2023

  22. [45]

    Cats and dogs

    [Parkhi et al., 2012] Omkar M Parkhi, Andrea Vedaldi, An- drew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recogni- tion, pages 3498–3505. IEEE,

    2012

  23. [47]

    Efficient neural architecture search via parameters sharing

    [Pham et al., 2018] Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean. Efficient neural architecture search via parameters sharing. In International conference on machine learning, pages 4095–4104. PMLR,

    2018

  24. [48]

    Learning transferable visual models from nat- ural language supervision

    [Radford et al., 2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from nat- ural language supervision. In International confere...

    2021

  25. [49]

    Do imagenet classi- fiers generalize to imagenet? In International conference on machine learning, pages 5389–5400

    [Recht et al., 2019] Benjamin Recht, Rebecca Roelofs, Lud- wig Schmidt, and Vaishaal Shankar. Do imagenet classi- fiers generalize to imagenet? In International conference on machine learning, pages 5389–5400. PMLR,

    2019

  26. [50]

    Au- tolr: Layer-wise pruning and auto-tuning of learning rates in fine-tuning of deep networks

    [Ro and Choi, 2021] Youngmin Ro and Jin Young Choi. Au- tolr: Layer-wise pruning and auto-tuning of learning rates in fine-tuning of deep networks. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 2486–2494,

    2021

  27. [51]

    Targeted transfer learning to improve performance in small medical physics datasets

    [Romero et al., 2020] Miguel Romero, Yannet Interian, Timothy Solberg, and Gilmer Valdes. Targeted transfer learning to improve performance in small medical physics datasets. Medical physics, 47(12):6246–6256,

    2020

  28. [52]

    The earth mover’s distance as a met- ric for image retrieval

    [Rubner et al., 2000] Yossi Rubner, Carlo Tomasi, and Leonidas J Guibas. The earth mover’s distance as a met- ric for image retrieval. International journal of computer vision, 40:99–121,

    2000

  29. [54]

    Exploiting cloze questions for few shot text classification and natural language inference

    [Schick and Sch¨utze, 2020a] Timo Schick and Hinrich Sch¨utze. Exploiting cloze questions for few shot text classification and natural language inference. arXiv preprint arXiv:2001.07676,

    2001 arXiv

  30. [55]

    It’s not just size that matters: Small language models are also few-shot learners

    [Schick and Sch¨utze, 2020b] Timo Schick and Hinrich Sch¨utze. It’s not just size that matters: Small language models are also few-shot learners. arXiv preprint arXiv:2009.07118,

    2009 arXiv

  31. [56]

    Few-shot text generation with natural language instructions

    [Schick and Sch¨utze, 2021] Timo Schick and Hinrich Sch¨utze. Few-shot text generation with natural language instructions. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages 390–402,

    2021

  32. [57]

    Grad-cam: Visual explanations from deep networks via gradient-based localization

    [Selvaraju et al., 2017] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on com- p...

    2017

  33. [59]

    Au- toprompt: Eliciting knowledge from language models with automatically generated prompts

    [Shin et al., 2020] Taylor Shin, Yasaman Razeghi, Robert L Logan IV , Eric Wallace, and Sameer Singh. Au- toprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980,

    2020 arXiv

  34. [60]

    What does clip know about a red circle? visual prompt engineering for vlms

    [Shtedritski et al., 2023] Aleksandar Shtedritski, Christian Rupprecht, and Andrea Vedaldi. What does clip know about a red circle? visual prompt engineering for vlms. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11987–11997,

    2023

  35. [61]

    Ucf101: A dataset of 101 human actions classes from videos in the wild

    [Soomro et al., 2012] Khurram Soomro, Amir Roshan Za- mir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402,

    2012 arXiv

  36. [62]

    An empirical study of exam- ple forgetting during deep neural network learning

    [Tonevaet al., 2018] Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Ben- gio, and Geoffrey J Gordon. An empirical study of exam- ple forgetting during deep neural network learning. arXiv preprint arXiv:1812.05159,

    2018 arXiv

  37. [63]

    Convolutional visual prompt for robust visual perception

    [Tsai et al., 2024] Yun-Yun Tsai, Chengzhi Mao, and Jun- feng Yang. Convolutional visual prompt for robust visual perception. Advances in Neural Information Processing Systems, 36,

    2024

  38. [64]

    Attention is all you need

    [Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30,

    2017

  39. [65]

    Advances and challenges in meta- learning: A technical review

    [Vettoruzzo et al., 2024] Anna Vettoruzzo, Mohamed-Rafik Bouguelia, Joaquin Vanschoren, Thorsteinn Rognvalds- son, and KC Santosh. Advances and challenges in meta- learning: A technical review. IEEE Transactions on Pat- tern Analysis and Machine Intelligence,

    2024

  40. [66]

    Sun database: Large-scale scene recognition from abbey to zoo

    [Xiao et al., 2010] Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE computer society conference on computer vision and pattern recognition, pages 3485–3492. IEEE,

    2010

  41. [68]

    Zeronas: Differentiable generative adversarial networks search for zero-shot learning

    [Yan et al., 2021] Caixia Yan, Xiaojun Chang, Zhihui Li, Weili Guan, Zongyuan Ge, Lei Zhu, and Qinghua Zheng. Zeronas: Differentiable generative adversarial networks search for zero-shot learning. IEEE transactions on pattern analysis and machine intelligence , 44(12):9733– 9740,

    2021

  42. [69]

    Improving vi- sual prompt tuning for self-supervised vision transformers

    [Yoo et al., 2023] Seungryong Yoo, Eunji Kim, Dahuin Jung, Jungbeom Lee, and Sungroh Yoon. Improving vi- sual prompt tuning for self-supervised vision transformers. In International Conference on Machine Learning , pages 40075–40092. PMLR,

    2023

  43. [70]

    Unified vi- sion and language prompt learning

    [Zang et al., 2022] Yuhang Zang, Wei Li, Kaiyang Zhou, Chen Huang, and Chen Change Loy. Unified vi- sion and language prompt learning. arXiv preprint arXiv:2210.07225,

    2022 arXiv

  44. [71]

    Understanding and robustifying differentiable archi- tecture search

    [Zela et al., 2019] Arber Zela, Thomas Elsken, Tonmoy Saikia, Yassine Marrakchi, Thomas Brox, and Frank Hut- ter. Understanding and robustifying differentiable archi- tecture search. arXiv preprint arXiv:1909.09656,

    2019 arXiv

  45. [1781]

    Distilling bert into simple neural networks with unlabeled transfer data

    [Mukherjee and Awadallah, 2019] Subhabrata Mukherjee and Ahmed Hassan Awadallah. Distilling bert into simple neural networks with unlabeled transfer data. arXiv preprint arXiv:1910.01769,

    2019 arXiv

  46. [1992]

    Visual prompt- ing via image inpainting

    [Bar et al., 2022] Amir Bar, Yossi Gandelsman, Trevor Dar- rell, Amir Globerson, and Alexei Efros. Visual prompt- ing via image inpainting. Advances in Neural Information Processing Systems, 35:25005–25017,

    2022

  47. [2000]

    Convolutional neural fabrics

    [Saxena and Verbeek, 2016] Shreyas Saxena and Jakob Ver- beek. Convolutional neural fabrics. Advances in neural information processing systems, 29,

    2016

  48. [2004]

    Making pre-trained language models better few- shot learners

    [Gao et al., 2020] Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better few- shot learners. arXiv preprint arXiv:2012.15723,

    2020 arXiv

  49. [2007]

    Probing representation forgetting in supervised and un- supervised continual learning

    [Davari et al., 2022] MohammadReza Davari, Nader Asadi, Sudhir Mudur, Rahaf Aljundi, and Eugene Belilovsky. Probing representation forgetting in supervised and un- supervised continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogniti...

    2022

  50. [2008]

    Efficient test-time model adaptation without forgetting

    [Niu et al., 2022] Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. In International conference on machine learn- ing, pages 16888–16905. PMLR,

    2022

  51. [2009]

    Search- ing for a robust neural architecture in four gpu hours

    [Dong and Yang, 2019] Xuanyi Dong and Yi Yang. Search- ing for a robust neural architecture in four gpu hours. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 1761–1770,

    2019

  52. [2010]

    Pc- darts: Partial channel connections for memory-efficient ar- chitecture search

    [Xu et al., 2019] Yuhui Xu, Lingxi Xie, Xiaopeng Zhang, Xin Chen, Guo-Jun Qi, Qi Tian, and Hongkai Xiong. Pc- darts: Partial channel connections for memory-efficient ar- chitecture search. arXiv preprint arXiv:1907.05737,

    2019 arXiv

  53. [2012]

    Com- putational optimal transport: With applications to data sci- ence

    [Peyr´e et al., 2019] Gabriel Peyr´e, Marco Cuturi, et al. Com- putational optimal transport: With applications to data sci- ence. Foundations and Trends® in Machine Learning , 11(5-6):355–607,

    2019

  54. [2013]

    Surgical fine-tuning improves adaptation to distri- bution shifts

    [Lee et al., 2022] Yoonho Lee, Annie S Chen, Fahim Tajwar, Ananya Kumar, Huaxiu Yao, Percy Liang, and Chelsea Finn. Surgical fine-tuning improves adaptation to distri- bution shifts. arXiv preprint arXiv:2210.11466,

    2022 arXiv

  55. [2014]

    Language models are few-shot learners

    [Brown et al., 2020] Tom Brown, Benjamin Mann, Nick Ry- der, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing sys- tems,...

    2020

  56. [2015]

    Prompt learning with one-shot setting based feature space analy- sis in vision-and-language models

    [Hirohashi et al., 2024] Yuki Hirohashi, Tsubasa Hirakawa, Takayoshi Yamashita, and Hironobu Fujiyoshi. Prompt learning with one-shot setting based feature space analy- sis in vision-and-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern ...

    2024

  57. [2016]

    Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification

    [Helber et al., 2019] Patrick Helber, Benjamin Bischke, An- dreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Top- ics in Applied Earth Observations and Remote Sensing , 12(...

    2019

  58. [2017]

    Partial is better than all: Revisiting fine-tuning strategy for few-shot learn- ing

    [Shen et al., 2021] Zhiqiang Shen, Zechun Liu, Jie Qin, Marios Savvides, and Kwang-Ting Cheng. Partial is better than all: Revisiting fine-tuning strategy for few-shot learn- ing. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 9594–9602,

    2021

  59. [2018]

    Transfer without forgetting

    [Boschini et al., 2022] Matteo Boschini, Lorenzo Bonicelli, Angelo Porrello, Giovanni Bellitto, Matteo Pennisi, Si- mone Palazzo, Concetto Spampinato, and Simone Calder- ara. Transfer without forgetting. In European Conference on Computer Vision, pages 692–709. Springer,

    2022

  60. [2019]

    Plot: Prompt learning with optimal transport for vision- language models

    [Chen et al., 2022] Guangyi Chen, Weiran Yao, Xiangchen Song, Xinyue Li, Yongming Rao, and Kun Zhang. Plot: Prompt learning with optimal transport for vision- language models. arXiv preprint arXiv:2210.01253,

    2022 arXiv

  61. [2020]

    Catastrophic for- getting meets negative transfer: Batch spectral shrinkage for safe transfer learning

    [Chen et al., 2019] Xinyang Chen, Sinan Wang, Bo Fu, Mingsheng Long, and Jianmin Wang. Catastrophic for- getting meets negative transfer: Batch spectral shrinkage for safe transfer learning. Advances in Neural Information Processing Systems, 32,

    2019

  62. [2021]

    Deep residual learning for image recog- nition

    [He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778,

    2016

  63. [2022]

    Un- derstanding and simplifying one-shot architecture search

    [Bender et al., 2018] Gabriel Bender, Pieter-Jan Kinder- mans, Barret Zoph, Vijay Vasudevan, and Quoc Le. Un- derstanding and simplifying one-shot architecture search. In International conference on machine learning , pages 550–559. PMLR,

    2018

  64. [2023]

    Fair darts: Eliminating unfair advantages in differentiable architecture search

    [Chu et al., 2020] Xiangxiang Chu, Tianbao Zhou, Bo Zhang, and Jixiang Li. Fair darts: Eliminating unfair advantages in differentiable architecture search. In European conference on computer vision, pages 465–480. Springer,

    2020

  65. [2024]

    Universal language model fine-tuning for text clas- sification

    [Howard and Ruder, 2018] Jeremy Howard and Sebastian Ruder. Universal language model fine-tuning for text clas- sification. arXiv preprint arXiv:1801.06146,

    2018 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.