REVIEW 4 major objections 5 minor 1 cited by
Efficient Adaptation of Pre-trained Vision Transformer underpinned by Approximately Orthogonal Fine-Tuning Strategy
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read This paper proposes Approximately Orthogonal Fine-Tuning (AOFT), which generates the down- and up-projection matrices of LoRA and Adapter from a single learnable vector so their columns are approximately orthogonal, and argues this lowers…
desk verdict A genuinely cheaper way to parameterize LoRA/Adapter projections that matches accuracy, wrapped in a generalization story that does not survive contact with scaling. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the AOFT generator: one learnable vector $\vec{q} = (q_0, q_1, \ldots, q_N)^\top$ is completed into an $(N+1)\times(N+1)$ orthogonal matrix $Q$ (columns pairwise perpendicular, unit length) by Eq. (5), and $\mathrm{AO}(\vec{q}) = Q[:, 0:d]$ takes the first $d$ columns to serve as a down- or up-projection matrix. By deliberately not enforcing $\sum_i |q_i|^2 = 1$, the resulting columns are only approximately orthogonal, which is exactly the property claimed for the backbone weights. This single object does two jobs: it transfers the angle distribution of the backbone to the adapters, and it makes the parameter count independent of bottleneck width, since any $d$ costs the same one vector. It plugs into LoRA as $W + \mathrm{AO}(\vec{q}_{\mathrm{down}})\mathrm{AO}(\vec{q}_{\mathrm{up}})^\top$, into Adapter in the same product form, and into VPT as a prompt matrix, with an optional learned scaling vector defining the AOFT* variant.
What would settle it
Take a trained AOFT adapter pair $(A, B)$ and replace it with $(c^{-1}A, cB)$ for a constant $c > 1$; the product $BA$ is unchanged, so the network's outputs are identical, but the $\ell^2$ norms of the two matrices change by $c$ and $1/c$ and the paper's Eq. (9) bound changes accordingly. Because the outputs are identically the same, test accuracy is provably unchanged while the bound moves, which would show that the norm-based generalization argument is not the mechanism behind AOFT's results; a reader could check this directly on the released checkpoints.
Extended reading notes
Core claim
The paper's central claim is that enforcing approximate orthogonality on the down/up-projection matrices of LoRA and Adapter aligns those matrices with the pre-trained backbone and improves generalization. The observed fact is that weight matrices in a pre-trained ViT are nearly orthogonal in the sense that angles between column vectors concentrate near 90 degrees, while the trained LoRA and Adapter projections are not; AOFT restores that property by generating $Q$ from one vector $\vec{q}$ and taking $d$ of its columns as the projection. The paper reports that this makes the projection matrices' $\ell^2$ norms much smaller, and uses Eq. (9) to conclude that the upper bound on generalization error is therefore lower for AOFT than for LoRA and Adapter. Experimentally the claim is supported on FGVC and VTAB-1k: for example, LoRA+AOFT reaches a VTAB-1k mean of $72.9$ with $0.08$M parameters against LoRA's $72.3$ with $0.29$M, and Adapter+AOFT* reaches $72.7$ with $0.06$M. The same recipe also improves VPT prompts, larger ViT backbones, and Swin.
Load-bearing premise
The load-bearing premise is that the numerical size of the adapter matrices, measured by their own $\ell^2$ norms, controls the whole fine-tuned network's generalization: the bound the paper writes is for a single linear layer and ignores the frozen backbone, every other layer, and the classifier, and rescaling the adapter together with the head changes the norms without changing the predictions.
Editorial extensions
If this is right
- If the lower-bound argument is correct, any PEFT scheme whose adapter matrices are replaced by AOFT projections inherits a smaller generalization-error upper bound, so the benefit is a structural property rather than a per-dataset tuning effect.
- Because one vector generates projections of any width $d$, downstream tasks can use wider bottlenecks at no extra parameter cost, which is a regime LoRA and Adapter cannot enter.
- The same construction applies to VPT prompts, and the reported prompt norms drop from the 2-7 range to roughly 1.0 across all layers, extending the claimed mechanism beyond bottleneck adapters.
- Results on ViT-L, ViT-H, and Swin-B show the recipe transfers across model scale and architecture family, so the alignment-with-backbone rationale is not tied to ViT-B.
Reading between the lines
- The paper does not isolate orthogonality from the smaller calibrated norms it induces; a control that uses the same single-vector parameterization but relaxes the near-orthogonality constraint would separate the two, and would be a direct test of the stated mechanism.
- A natural extension the authors leave implicit is to language-model LoRA and multimodal adapters: if near-orthogonality is a general property of large pre-trained transformers, the same one-vector construction should transfer outside image classification.
- Since the generating vector's length is independent of adapter width, AOFT points toward very wide or even full-rank adapters at constant parameter cost, which standard LoRA cannot represent; this is a testable regime rather than a claim the paper makes.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Approximately Orthogonal Fine-Tuning (AOFT), a PEFT strategy that replaces the down/up-projection matrices of LoRA and Adapter (and the prompt matrix of VPT) with submatrices of an orthogonal matrix generated from a single learnable vector. The authors report competitive accuracy on FGVC and VTAB-1k with substantially fewer trainable parameters, and they argue that the approximate orthogonality of the generated matrices reduces an upper bound on generalization error, thereby explaining the improved performance.
Significance. If the empirical results are taken at face value, AOFT is a simple and parameter-efficient adapter variant: for example, LoRA+AOFT reaches a VTAB-1k mean of 72.9 with 0.08M parameters versus LoRA's 72.3 with 0.29M, and the method is tested on ViT-B, ViT-L, ViT-H, and Swin backbones. The paper also provides a code link and reports extensive comparisons on 24 datasets. However, the paper's central explanatory claim—that approximately orthogonal projection matrices yield a provably lower generalization-error bound—is not supported by the presented theory or experiments. The empirical accuracy improvements are modest and may stem from the changed parameterization rather than from orthogonality, so the theoretical mechanism as stated should be substantially revised or removed.
major comments (4)
- [Section 3.3, Eq. (9)] The bound in Eq. (9) is a Rademacher-complexity estimate for a single linear layer whose weight matrix has L2 norm at most gamma, and the paper does not derive a corresponding bound for the full fine-tuned ViT. A full-network generalization bound would need to compose Lipschitz constants across the frozen backbone, the classification head, and all adapter layers; the frozen backbone's norm would dominate and the adapter norm differences in Fig. 5 would become negligible. Consequently, the claim that 'the upper bound of the generalization error for AOFT is much lower than that of the LoRA and Adapter methods' does not follow from Eq. (9).
- [Section 3.3, Fig. 5] The L2-norm comparison in Fig. 5 is scale-dependent and therefore does not establish a property of the learned function. In LoRA the update is scaled as (alpha/r)BA, while in AOFT the down/up matrices are built directly from a vector q; if one rescales a learned down/up pair by c and 1/c, the product BA and the model function remain identical, yet the individual norms change by factors c and 1/c. Thus the smaller AOFT norms are a parameterization convention, not evidence of lower generalization error, and the claimed 'much lower upper bound' is an artifact of that convention.
- [Section 3.2, Eq. (5) and Fig. 4] The approximate orthogonality of the AOFT matrices is guaranteed by construction: Eq. (5) defines a matrix whose columns are exactly orthogonal when the normalization condition is enforced, and approximately orthogonal when it is relaxed. Therefore the angle distributions in Fig. 4, which show AOFT matrices concentrated near 90 degrees, are a direct consequence of the construction and cannot serve as independent empirical confirmation of the orthogonality mechanism. The paper should provide a control experiment that varies orthogonality while keeping the parameterization and parameter count fixed.
- [Sections 3.2–3.3] The paper never isolates the orthogonality constraint from the change in low-rank parameterization. AOFT restricts the update to a product of matrices generated from a single vector, which imposes a strong inductive bias (near-unit singular values, shared generating vector, reduced degrees of freedom) independent of orthogonality. The observed accuracy gains could therefore be due to this inductive bias or to the smaller number of trainable parameters, rather than to the generalization bound. The authors should compare AOFT against non-orthogonal parameterizations with matched parameter count and matched norms, and against randomly generated or fixed orthogonal matrices, to identify the actual source of any improvement.
minor comments (5)
- [Throughout] There are numerous typographical and grammatical errors, including 'Lora' instead of 'LoRA' in Section 3.3, 'Structed' in the VTAB-1k tables, and incomplete sentences such as 'we evaluate the AOFT integrate state-of-the-art PEFT approaches' in Section 4.2; the manuscript would benefit from careful proofreading.
- [Table 6] The caption of Table 6 mentions 'with or without incorporating eigenvalues,' but the table and surrounding text do not contain any eigenvalue analysis; either the caption should be corrected or the analysis should be added.
- [Section 4.1] The description of the AOFT* variant says 'AOFT* is defined as (W_down ⊙ λ^T)W_up,' but the notation is not fully specified: it is unclear whether λ is applied per column, per layer, or shared across layers, and how this interacts with the orthogonality of W_down.
- [Appendix D] The derivation of the orthogonal matrix in Appendix D is referenced to a quantum-chemistry textbook [22]; a self-contained statement of the construction would improve accessibility and avoid reliance on an unusual source for a linear-algebra fact.
- [Figures 1–4 and 7–9] The histograms in several figures are difficult to read because the legends are provided only in Appendix A, and the figure captions do not state the number of bins or the normalization; adding these details would improve interpretability.
Circularity Check
Mechanistic argument is partly tautological and scale-dependent: AOFT matrices are approximately orthogonal by construction, and the norm-based generalization claim is a post-hoc, rescaling-dependent statistic; the accuracy comparisons themselves remain independent evidence.
-
self definitional
[Section 3.2, Eq. (4)-(5), and Fig. 4]
"In [22] a method to construct the orthogonal matrix is proposed... Following this construction, we use a learnable vector ⃗q = (q0, q1, · · ·, qN )⊤ ∈ RN to generate an orthogonal matrix Q ∈ RN ×N , then, we replace down- or up-projection matrix with the orthogonal matrix Q. ... AO(⃗q) = Q[:, 0 : d] ... the column vectors of Q are strictly orthogonal to each other when using the normalization PN i=1 |qi|2 = 1. To enhance the flexibility of the model's capacity, we do not strictly adhere to this normalization, allowing the column vectors of the matrix Q to be approximately orthogonal."
The down/up-projection matrices are defined, in Eq. (4), as submatrices of Q, and Q is constructed in Eq. (5) to have strictly orthogonal columns when normalized and approximately orthogonal columns otherwise. Therefore Fig. 4, which shows that trained AOFT matrices have pairwise angles near 90 degrees, is not an empirical confirmation of the orthogonality hypothesis; it is a restatement of the construction. The paper uses this figure as evidence that AOFT aligns the adaptation matrices with the backbone, but the property is baked into the parameterization, so the confirmation is self-definitional.
-
fitted input called prediction
[Section 3.3, Eq. (9), Fig. 5]
"In our method, the L2-norms ∥Wdown∥ and ∥Wup∥ of AOFT are significantly smaller than that of the LoRA and Adapter methods, as illustrated in Fig. 5. Consequently, the upper bound of the generalization error for AOFT is much lower than that of the Lora and Adapter methods, demonstrating that AOFT possesses superior generalization capability."
The norms plotted in Fig. 5 are measured post hoc on trained AOFT models, not predicted by the construction. Eq. (9) bounds the Rademacher complexity of a single linear layer by γ = ∥W∥ and contains no orthogonality term, so it does not establish a bound for the entire frozen-backbone ViT. Moreover, the norm comparison is scale-dependent: for any learned pair (A, B), replacing it by (cA, c^{-1}B) leaves the product BA and the model function unchanged while changing both norms. The claimed 'much lower upper bound' is therefore an artifact of the chosen parameterization and training scale, not a consequence of the AOFT mechanism, and the measured norms are fitted statistics used as a stand-in for a prediction of generalization error.
full rationale
The paper's empirical contribution — replacing LoRA/Adapter down/up projections with matrices generated from a single vector and measuring accuracy on FGVC and VTAB-1k — is self-contained and compares against external baselines, so it is not circular. The claimed mechanism, however, is circular in two places. First, the approximate orthogonality of AOFT matrices is guaranteed by construction (Eqs. 4-5), so Fig. 4 merely reflects the definition rather than validating the hypothesis. Second, the theoretical bridge in Section 3.3 uses a single-layer Rademacher bound that depends only on weight norms, and the reported norm reductions are measured on the trained models; because a reciprocal rescaling of the down/up pair preserves the function while changing the norms, the 'much lower upper bound' conclusion is a scaling artifact rather than a derived prediction. No load-bearing self-citation chain was found: references [2] and [3] share authors but serve as baselines, not as support for the core premise. The accuracy gains and parameter savings remain independent empirical results, which caps the circularity score at a moderate level.
Assumptions & free parameters
free parameters (4)
- per-column scaling vector λ used in AOFT*/AOFT† =
learned per dataset, not reported numerically
- bottleneck dimension d per task (AOFT†) =
d ∈ {4, 8, 16} in Fig. 6; dim=64 (VPT-Shallow), dim=768 (VPT-Deep)
- norm of the generating vector q =
unconstrained (unit-norm condition explicitly dropped)
- learning rate, weight decay, dropout, batch size =
grid-searched per task (Appendix E)
assumptions (4)
- domain assumption Pre-trained ViT backbone weight matrices (Wq, Wk, Wv, Wo, WFC1, WFC2) have approximately orthogonal row/column vectors.
- domain assumption A reduced L2 norm γ of the adaptation matrices implies reduced generalization error of the full fine-tuned ViT.
- standard math The Householder-type matrix in Eq. (5) has strictly orthogonal columns when Σ|qi|²=1.
- domain assumption Approximate orthogonality and the smaller L2 norms of AOFT matrices persist after training despite the dropped normalization.
Cite this review
Pith. "Pith review of Efficient Adaptation of Pre-trained Vision Transformer underpinned by Approximately Orthogonal Fine-Tuning Strategy." pith.science (2026). https://pith.science/paper/UIYT4TYY
@misc{pith2026250713260,
author = {Pith},
title = {Pith review of: Efficient Adaptation of Pre-trained Vision Transformer underpinned by Approximately Orthogonal Fine-Tuning Strategy},
year = {2026},
howpublished = {\url{https://pith.science/paper/UIYT4TYY}},
note = {Machine review of arXiv:2507.13260}
}
read the original abstract
A prevalent approach in Parameter-Efficient Fine-Tuning (PEFT) of pre-trained Vision Transformers (ViT) involves freezing the majority of the backbone parameters and solely learning low-rank adaptation weight matrices to accommodate downstream tasks. These low-rank matrices are commonly derived through the multiplication structure of down-projection and up-projection matrices, exemplified by methods such as LoRA and Adapter. In this work, we observe an approximate orthogonality among any two row or column vectors within any weight matrix of the backbone parameters; however, this property is absent in the vectors of the down/up-projection matrices. Approximate orthogonality implies a reduction in the upper bound of the model's generalization error, signifying that the model possesses enhanced generalization capability. If the fine-tuned down/up-projection matrices were to exhibit this same property as the pre-trained backbone matrices, could the generalization capability of fine-tuned ViTs be further augmented? To address this question, we propose an Approximately Orthogonal Fine-Tuning (AOFT) strategy for representing the low-rank weight matrices. This strategy employs a single learnable vector to generate a set of approximately orthogonal vectors, which form the down/up-projection matrices, thereby aligning the properties of these matrices with those of the backbone. Extensive experimental results demonstrate that our method achieves competitive performance across a range of downstream image classification tasks, confirming the efficacy of the enhanced generalization capability embedded in the down/up-projection matrices.
Figures
Figures from the paper (5 more)
Forward citations
Cited by 1 Pith paper
-
Understanding and Enforcing Weight Disentanglement in Task Arithmetic
Task-Feature Specialization explains weight disentanglement in task arithmetic and leads to orthogonality, which OrthoReg enforces to enhance performance of model composition methods.
Reference graph
Works this paper leans on
-
[1]
Qlora: Efficient finetuning of quantized llms
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. Advances in neural information processing systems , 36: 10088–10115, 2023. 1
work page 2023
-
[2]
Effi- cient adaptation of large vision transformer via adapter re- composing
Wei Dong, Dawei Yan, Zhijun Lin, and Peng Wang. Effi- cient adaptation of large vision transformer via adapter re- composing. In Thirty-seventh Conference on Neural Infor- mation Processing Systems, 2023. 3, 5, 7
work page 2023
-
[3]
Low-rank rescaled vision transformer fine-tuning: A residual design approach
Wei Dong, Xing Zhang, Bihui Chen, Dawei Yan, Zhijun Lin, Qingsen Yan, Peng Wang, and Yang Yang. Low-rank rescaled vision transformer fine-tuning: A residual design approach. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 16101– 16110, 2024. 3, 7
work page 2024
-
[4]
An image is worth 16x16 words: Trans- formers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. InInternational Con- ference on Learning Representations, 2020. 1, 3, 6
work page 2020
-
[5]
Fine-grained car detection for vi- sual census estimation
Timnit Gebru, Jonathan Krause, Yilun Wang, Duyun Chen, Jia Deng, and Li Fei-Fei. Fine-grained car detection for vi- sual census estimation. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, 2017. 5, 2
work page 2017
-
[6]
Foundations of ma- chine learning
Vishal Goar and Nagendra Singh Yadav. Foundations of ma- chine learning. In Intelligent Optimization Techniques for Business Analytics, pages 25–48. IGI Global, 2024. 4
work page 2024
-
[7]
Neural network methods in natural lan- guage processing
Yoav Goldberg. Neural network methods in natural lan- guage processing. Morgan & Claypool Publishers, 2017. 7
work page 2017
-
[8]
Gene H Golub and Charles F Van Loan. Matrix computa- tions. JHU press, 2013. 2
work page 2013
Show all 34 references
-
[9]
Parameter-efficient transfer learning for nlp
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In International Conference on Machine Learning, pages 2790–2799. PMLR, 2019. 1, 2, 6, 7, 3
2019
-
[10]
Lora: Low- rank adaptation of large language models
Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low- rank adaptation of large language models. In International Conference on Learning Representations, 2021. 1, 2, 6, 7, 3
2021
-
[11]
Vi- sual prompt tuning
Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Vi- sual prompt tuning. In European Conference on Computer Vision, pages 709–727. Springer, 2022. 1, 3, 5, 6, 7, 8, 2
2022
-
[12]
Fact: Factor-tuning for lightweight adaptation on vision transformer
Shibo Jie and Zhi-Hong Deng. Fact: Factor-tuning for lightweight adaptation on vision transformer. In Proceed- ings of the AAAI Conference on Artificial Intelligence, pages 1060–1068, 2023. 5
2023
-
[13]
Novel dataset for fine-grained image categorization: Stanford dogs
Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Fei-Fei Li. Novel dataset for fine-grained image categorization: Stanford dogs. In Proc. CVPR workshop on fine-grained visual categorization (FGVC) . Citeseer, 2011. 5, 2
2011
-
[14]
Vera: Vector-based random matrix adaptation.arXiv preprint arXiv:2310.11454, 2023
Dawid J Kopiczko, Tijmen Blankevoort, and Yuki M Asano. Vera: Vector-based random matrix adaptation.arXiv preprint arXiv:2310.11454, 2023. 1, 2, 7
2023 arXiv
-
[15]
Scaling & shifting your features: A new baseline for efficient model tuning
Dongze Lian, Daquan Zhou, Jiashi Feng, and Xinchao Wang. Scaling & shifting your features: A new baseline for efficient model tuning. Advances in Neural Information Processing Systems, 35:109–123, 2022. 5
2022
-
[16]
Dora: Weight-decomposed low-rank adaptation
Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. Dora: Weight-decomposed low-rank adaptation. In Forty-first International Conference on Ma- chine Learning, 2024. 7
2024
-
[17]
Parameter-efficient orthogo- nal finetuning via butterfly factorization
Weiyang Liu, Zeju Qiu, Yao Feng, Yuliang Xiu, Yux- uan Xue, Longhui Yu, Haiwen Feng, Zhen Liu, Juyeon Heo, Songyou Peng, et al. Parameter-efficient orthogo- nal finetuning via butterfly factorization. arXiv preprint arXiv:2311.06243, 2023. 3
2023 arXiv
-
[18]
Swin transformer: Hierarchical vision transformer using shifted windows
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 6
2021
-
[19]
Towards efficient visual adaption via structural re-parameterization
Gen Luo, Minglang Huang, Yiyi Zhou, Xiaoshuai Sun, Guannan Jiang, Zhiyu Wang, and Rongrong Ji. Towards efficient visual adaption via structural re-parameterization. arXiv preprint arXiv:2302.08106, 2023. 7
2023 arXiv
-
[20]
Moil: Momentum imita- tion learning for efficient vision-language adaptation
Gen Luo, Yiyi Zhou, Minglang Huang, Tianhe Ren, Xi- aoshuai Sun, and Rongrong Ji. Moil: Momentum imita- tion learning for efficient vision-language adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence,
-
[21]
Parameter efficient quasi-orthogonal fine- tuning via givens rotation
Xinyu Ma, Xu Chu, Zhibang Yang, Yang Lin, Xin Gao, and Junfeng Zhao. Parameter efficient quasi-orthogonal fine- tuning via givens rotation. arXiv preprint arXiv:2404.04316,
-
[22]
Simple theorems, proofs, and derivations in quantum chemistry
Istv ´an Mayer. Simple theorems, proofs, and derivations in quantum chemistry . Springer Science & Business Media,
-
[23]
Automated flower classification over a large number of classes
Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & im- age processing, pages 722–729. IEEE, 2008. 5, 2
2008
-
[24]
Controlling text-to-image diffusion by orthogo- nal finetuning
Zeju Qiu, Weiyang Liu, Haiwen Feng, Yuxuan Xue, Yao Feng, Zhen Liu, Dan Zhang, Adrian Weller, and Bernhard Sch¨olkopf. Controlling text-to-image diffusion by orthogo- nal finetuning. Advances in Neural Information Processing Systems, 36:79320–79362, 2023. 3, 7
2023
-
[25]
Deep learning in neural networks: An overview
J ¨urgen Schmidhuber. Deep learning in neural networks: An overview. Neural networks, 61:85–117, 2015. 7
2015
-
[26]
An adaptive strategy selection method with reinforcement learning for robotic soccer games
Haobin Shi, Zhiqiang Lin, Kao-Shing Hwang, Shike Yang, and Jialin Chen. An adaptive strategy selection method with reinforcement learning for robotic soccer games. IEEE Ac- cess, 6:8376–8386, 2018. 7
2018
-
[27]
Adapters strike back
Jan-Martin O Steitz and Stefan Roth. Adapters strike back. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23449–23459, 2024. 7
2024
-
[28]
Numerical linear alge- bra
Lloyd N Trefethen and David Bau. Numerical linear alge- bra. SIAM, 2022. 2
2022
-
[29]
Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection
Grant Van Horn, Steve Branson, Ryan Farrell, Scott Haber, Jessie Barry, Panos Ipeirotis, Pietro Perona, and Serge Be- longie. Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection. In Proceedings of t...
-
[30]
Deep neural networks for im- age classification
A Vasuki and S Govindaraju. Deep neural networks for im- age classification. In Deep Learning for Image Processing Applications, pages 27–49. IOS Press, 2017. 7
2017
-
[31]
The caltech-ucsd birds-200-2011 dataset
Catherine Wah, Steve Branson, Peter Welinder, Pietro Per- ona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. 2011. 5, 2
2011
-
[32]
Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models
Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel. Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. In Proceedings of the 60th An- nual Meeting of the Association for Computational Linguis- tics (Volume 2: Short Papers), pages 1–9, ...
2022
-
[33]
A large-scale study of representation learning with the visual task adaptation benchmark
Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov, Pierre Ruyssen, Carlos Riquelme, Mario Lucic, Josip Djo- longa, Andre Susano Pinto, Maxim Neumann, Alexey Doso- vitskiy, et al. A large-scale study of representation learning with the visual task adaptation benchmark. arXiv ...
1910 arXiv
-
[34]
The visual task adaptation benchmark
Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov, Pierre Ruyssen, Carlos Riquelme, Mario Lucic, Josip Djo- longa, Andre Susano Pinto, Maxim Neumann, Alexey Doso- vitskiy, et al. The visual task adaptation benchmark. 2019. 6 Efficient Adaptation of Pre-trained Vision Transfo...
2019
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.