Pith. sign in

REVIEW 3 major objections 5 minor 80 references

Complementary Subspace Low-Rank Adaptation of Vision-Language Models for Few-Shot Classification

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tune CLIP in the subspace perpendicular to its principal singular directions to preserve zero-shot ability while learning few-shot tasks.

desk verdict A plausible but lightly-verified twist on LoRA for few-shot CLIP: gains are small but real, math has dimensional errors, and the mechanism is untested. read the letter →

arxiv 2501.15040 v1 pith:UI3VP7GG submitted 2025-01-25 cs.CV

Zhongqi Wang , Jia Dai , Kai Li , Xu Li , Yanmeng Guo , Maosheng Xiang This is my paper · ORCID
classification cs.CV
keywords few-shotclassificationvision-languagemodelsCLIPlow-rankadaptationcatastrophicforgettingcomplementarysubspacesingularvaluedecompositionparameter-efficientfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes Comp-LoRA, a low-rank adaptation method for fine-tuning CLIP on few-shot classification tasks while avoiding catastrophic forgetting. The authors argue that standard LoRA updates interfere with the most influential directions of the pretrained weight matrices, eroding the vision-language alignment knowledge that CLIP needs to generalize from very few examples. Their fix is to optimize the low-rank update only inside the complementary subspace, the part of the weight space spanned by the singular vectors that are left after removing the top principal directions. In experiments across 11 image classification datasets, Comp-LoRA beats the CLIP-LoRA baseline by about +1.0% average Top-1 accuracy, and it retains about +1.3% more zero-shot accuracy after fine-tuning on ImageNet. If the result holds, it gives a parameter-efficient path to adapting large vision-language models with scarce labels.

What carries the argument

The central object is the singular value decomposition of each linear weight matrix in CLIP's attention layers, written W = U_p Σ_p V_p^T + U_c Σ_c V_c^T. The top-p singular vectors (U_p, V_p) define the principal subspace that the paper treats as carrying the model's general vision-language knowledge; the remaining vectors define the complementary subspace R^c. Comp-LoRA computes fixed projection matrices U_c and (V_c)^T once, then places the learnable low-rank pair (A, B) inside that subspace, so the forward pass is W x + η (V_c)^T B A U_c x. Because SVD yields orthonormal bases, the update direction is orthogonal to the principal subspace, which is what is supposed to eliminate interference with pretrained knowledge. The scalar singular values are intentionally discarded since only the unitary projection is needed.

What would settle it

Ablate the subspace choice: replace the top-p singular vectors with p random orthogonal directions of the same dimension. If zero-shot retention stays equally high, the ranking of directions is irrelevant and the method's benefit is generic dimensionality reduction, not the preservation of principal knowledge.

Watch

Extended reading notes

Core claim

The paper's claim is that catastrophic forgetting in few-shot LoRA fine-tuning is caused by interfering with the principal singular directions of each linear weight matrix, and that constraining the low-rank matrices A and B to act on the complementary subspace preserves the model's general alignment ability while still learning the new task. Concretely, the weight update becomes h = W x + η (V_c)^T B A U_c x, with U_c and V_c fixed from a one-time SVD and only A and B trainable. The authors report that this suppresses forgetting: after fine-tuning on an ImageNet support set, Comp-LoRA achieves 66.80% average Top-1 accuracy across ten other zero-shot tasks versus 65.30% for direct CLIP-LoRA, and on the main few-shot benchmark it outperforms the baseline at 1, 4, and 16 shots (73.85% vs 72.5% at 1-shot, 83.45% vs 83.0% at 16-shot). They also show that the complementary subspace dimension is a meaningful hyper-parameter, with best results around 496 of 512 dimensions and degrading performance when too many directions are removed.

Load-bearing premise

The argument depends on believing that the few most important directions of a pretrained weight matrix contain the general vision–language knowledge, so leaving them untouched protects that knowledge without preventing the new task from being learned.

Editorial extensions

If this is right

  • On the 11-dataset few-shot benchmark, Comp-LoRA improves average Top-1 accuracy over the CLIP-LoRA baseline by roughly +1.0% at 1-shot, +0.2% at 4-shot, and +0.45% at 16-shot.
  • Fine-tuning CLIP on ImageNet with Comp-LoRA preserves more zero-shot accuracy on ten other datasets than baseline LoRA does, with the clearest gains on EuroSAT, Cars, and DTD.
  • Because the projection matrices are computed once and only A and B are trained, the method adds no extra inference cost compared with standard LoRA and uses slightly fewer parameters.
  • The method is designed to be combined with other few-shot regularization techniques, such as entropy regularization or margin maximization.
  • The complementary subspace dimension is a practical hyper-parameter; the paper's univariate sweep shows performance peaks near 496 dimensions and degrades outside the 384–511 range.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension would be to ablate which singular directions are removed: if deleting random directions yields the same forgetting suppression, the benefit would be generic subspace regularization rather than protection of principal knowledge.
  • The same complementary-subspace construction could be applied to other PEFT methods, such as prompt tuning or adapter layers, and the paper's dimension sweep suggests the optimal subspace size will be dataset-dependent.
  • The observed gains might partly come from the smaller effective parameter count rather than the orthogonality; comparing against a standard LoRA with the same number of trainable parameters would disentangle these.
  • If zero-shot knowledge is actually distributed across all singular directions, the preset projection could discard useful signal on some datasets; the paper's EuroSAT dimension sweep, which oscillates without a clear trend, hints that this may already be happening.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Comp-LoRA, a variant of low-rank adaptation for few-shot fine-tuning of CLIP. The method computes the SVD of each linear layer's weight matrix, discards the top-p principal singular directions, and constrains the LoRA update to the complementary subspace spanned by the remaining singular vectors. The authors claim this preserves the vision-language alignment ability of the pretrained model and suppresses catastrophic forgetting, reporting about +1.0% Top-1 accuracy over CLIP-LoRA on few-shot classification and +1.3% preservation of zero-shot accuracy on a cross-dataset forgetting benchmark. The paper includes comparisons with prompt-tuning and adapter methods on 11 datasets at 1/2/4/8/16 shots, a univariate study of the complementary dimension c, and supplementary forgetting experiments with each dataset as the fine-tuning source.

Significance. If the empirical results hold, Comp-LoRA is a simple, parameter-efficient regularizer that could be applied on top of existing few-shot CLIP methods, and the cross-dataset forgetting experiments are a useful addition to the LoRA-for-VLM literature. The paper's strengths are its standard benchmark covering 11 datasets, 5-seed averaging in the main tables, and a large supplementary suite of forgetting experiments (Tables 3-13). The main weakness is that the mathematical formulation in Section 3 is not self-consistent, and the central mechanism — that the top-p singular directions carry the zero-shot knowledge — is asserted rather than tested. The conclusion itself concedes that a solid explanation for the superior performance is still missing.

major comments (3)
  1. [Sec. 3.3, Eq. (5); Sec. 3.4, Eqs. (15)-(16)] The update rule h = Wx + eta (V_c)^T B A U_c x + b is dimensionally inconsistent with the definitions in Eq. (3), where U_c and V_c are d x c matrices. If x is d-dimensional, U_c x is undefined, and (V_c)^T B A U_c cannot be added to Wx as a d x d update. In Eqs. (15)-(16), the objective ||(V_c)^T B A U_c - Delta W||_F^2 compares a c x c matrix with a d x d matrix, and the claimed equivalence to ||BA - (V_c) Delta W (U_c)^T||_F^2 is not a valid algebraic step unless the projection matrices are properly inverted. Please correct the orientation of the projections (e.g., using U_c^T for the input projection and V_c for the output pull-back, with matching dimensions) and re-derive the optimization objective accordingly.
  2. [Sec. 3.2 and Sec. 5] The load-bearing premise that 'these singular vectors guide the most changeable directions' and that removing the top-p directions preserves vision-language alignment is asserted but never tested. The paper does not compare Comp-LoRA with an equally constrained LoRA update in a random complementary subspace or in the complement of the bottom-p directions; without such a control, the observed suppression of catastrophic forgetting could be a generic effect of reducing the update subspace rather than a specific benefit of protecting the principal directions. The conclusion's statement that 'we still need a solid explanation for the superior performance' acknowledges this gap, but the main text should either provide a direct test of the mechanism or substantially temper the causal claim.
  3. [Sec. 4.4 and Tables 1-2] The complementary dimension c = 496 is selected by inspecting the ImageNet accuracy curve in Fig. 5, and the same ImageNet results then appear in the reported Tables 1 and 2. This selection-on-the-test-set introduces bias in the ImageNet rows and in the averages, and the 5-seed averages are reported without standard deviations, so the magnitude of the claimed improvements (+1.0%, +1.3%) cannot be assessed against run-to-run variability. Please provide a validation-based selection procedure, report per-seed variance, and show results for the full range of c values across datasets rather than only the chosen value.
minor comments (5)
  1. [Sec. 2.2] The word 'catestrophy' should be 'catastrophic', and in Sec. 4.1 'conveninence' should be 'convenience'.
  2. [Fig. 1 and Sec. 3.3] The caption of Fig. 1 says the input is projected with 'the pre-computed matrix U_c' and the output is pulled back with 'matrix V_c', which is inconsistent with the notation in Eq. (5) where (V_c)^T appears on the left. Please align the caption, the equations, and the definitions of U_c and V_c.
  3. [Fig. 6] Figure 6 is described as showing singular values of 'all linear weight matrices' but it is unclear which layers are included and how the curves are aggregated; please clarify the caption and the text.
  4. [Eq. (2)] The decomposition W = U · Σ · V should be W = U Σ V^T to match standard SVD notation; the transpose is used inconsistently in Eq. (3).
  5. [Sec. 5] The sentence 'Regarding theory, we still need a solid explanation for the superior performance' is placed in the conclusion but is not discussed in the main body; please expand on what is missing or remove the sentence if the empirical evidence is considered sufficient.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Comp-LoRA's claims rest on external benchmark measurements; the subspace-preservation premise is an untested assumption but not a by-construction reduction.

full rationale

The paper's central claim is that optimizing a LoRA adapter in the SVD-complement of the top-p singular directions preserves CLIP's zero-shot ability while learning few-shot tasks. The claimed mechanism is an empirical hypothesis tested on external benchmarks (Tables 1-2): the forward pass of Eq. 5 is not a surrogate for Top-1 accuracy, so no reported number is forced by construction. The statement in Sec. 3.2 that 'these singular vectors guide the most changeable directions' and that the complementary space 'does not interfere with the principal directions that hold the zero-shot ability of VLM' is an asserted premise rather than a derived theorem; the paper itself concedes in the Conclusion that 'we still need a solid explanation for the superior performance.' That is a correctness or evidence gap, not a circular reduction. The choice c=496 is made from the ImageNet univariate sweep (Sec. 4.4) and then reused for the ImageNet rows in Tables 1-2, which is a mild selection-leakage concern, but this single hyperparameter does not by construction determine the reported gains over CLIP-LoRA. No load-bearing self-citations, no imported uniqueness theorems, and no renaming of known results were found.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method depends on three assumptions: the principal directions carry the knowledge to preserve (domain assumption), the new task fits in the complementary subspace (domain assumption), and SVD orthogonality preserves the LoRA optimization behavior (standard math). The only hand-tuned scalar is the complementary subspace dimension c=496, chosen on ImageNet; rank r=2 is inherited from the baseline. No new physical or conceptual entities are introduced.

free parameters (3)
  • complementary subspace dimension c = 496 for d=512
    Chosen after inspecting the singular value distribution of CLIP weights and univariate accuracy curves on ImageNet (Section 4.4); applied to all datasets. This is a hand-tuned hyperparameter, and its value affects results.
  • LoRA rank r = 2
    Follows the baseline CLIP-LoRA [66], with no ablation shown for other ranks.
  • learning rate = ambiguous (2^-4 or 2e-4)
    The text in Section 4.1 reads 'initial learning rate is set to 2−4', which is ambiguous and not clarified.
assumptions (3)
  • domain assumption The top-p singular vectors of a weight matrix capture the directions most relevant to the model's general knowledge.
    Stated in Section 3.2: 'these singular vectors guide the most changeable directions'; used to justify eliminating them. Not directly tested.
  • domain assumption The adaptation needed for a new few-shot task can be expressed in the complementary subspace with the same low rank r.
    Assumed implicitly when constraining BA to act only in the reduced subspace (Eq. 12-16).
  • standard math Orthogonal projection matrices from SVD provide a valid parameterization of LoRA without losing optimization efficacy.
    Relies on the orthogonality of SVD, a standard fact, but its sufficiency for preserving LoRA's optimization behavior is asserted in Section 3.4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Complementary Subspace Low-Rank Adaptation of Vision-Language Models for Few-Shot Classification." pith.science (2026). https://pith.science/paper/UI3VP7GG

@misc{pith2026250115040,
  author       = {Pith},
  title        = {Pith review of: Complementary Subspace Low-Rank Adaptation of Vision-Language Models for Few-Shot Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UI3VP7GG}},
  note         = {Machine review of arXiv:2501.15040}
}
read the original abstract

Vision language model (VLM) has been designed for large scale image-text alignment as a pretrained foundation model. For downstream few shot classification tasks, parameter efficient fine-tuning (PEFT) VLM has gained much popularity in the computer vision community. PEFT methods like prompt tuning and linear adapter have been studied for fine-tuning VLM while low rank adaptation (LoRA) algorithm has rarely been considered for few shot fine-tuning VLM. The main obstacle to use LoRA for few shot fine-tuning is the catastrophic forgetting problem. Because the visual language alignment knowledge is important for the generality in few shot learning, whereas low rank adaptation interferes with the most informative direction of the pretrained weight matrix. We propose the complementary subspace low rank adaptation (Comp-LoRA) method to regularize the catastrophic forgetting problem in few shot VLM finetuning. In detail, we optimize the low rank matrix in the complementary subspace, thus preserving the general vision language alignment ability of VLM when learning the novel few shot information. We conduct comparison experiments of the proposed Comp-LoRA method and other PEFT methods on fine-tuning VLM for few shot classification. And we also present the suppression on the catastrophic forgetting problem of our proposed method against directly applying LoRA to VLM. The results show that the proposed method surpasses the baseline method by about +1.0\% Top-1 accuracy and preserves the VLM zero-shot performance over the baseline method by about +1.3\% Top-1 accuracy.

Figures

Figures reproduced from arXiv: 2501.15040 by the authors.

Figure 1
Figure 1. The architecture of Comp-LoRA with comparison to original LoRA. We first project the input [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. SVD decomposition for weights in linear layers. We [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The diagram of complementary subspace. We optimize [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: The comparison experiments of different methods on [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: The effect of complementary subspace dimension on [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 8
Figure 8. Figure 8: The effect of complementary subspace dimension on [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: The effect of complementary subspace dimension on [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 7
Figure 7. Figure 7: The effect of complementary subspace dimension on [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 11
Figure 11. Figure 11: The effect of complementary subspace dimension on [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 15
Figure 15. Figure 15: The effect of complementary subspace dimension on [PITH_FULL_IMAGE:figures/full_fig_p014_15.png]
Figure 16
Figure 16. Figure 16: The effect of complementary subspace dimension on [PITH_FULL_IMAGE:figures/full_fig_p014_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

80 extracted references · 63 canonical work pages

  1. [1]

    In- trinsic dimensionality explains the effectiveness of language model fine-tuning

    Armen Aghajanyan, Sonal Gupta, and Luke Zettlemoyer. In- trinsic dimensionality explains the effectiveness of language model fine-tuning. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , pages 7319–7328,

    work page

  2. [2]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Men- sch, Katherine Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob L Menick, Sebastian Borgeaud, Andy Brock, Aida Nematzadeh, Sa- hand Sharifzadeh, Mikoł aj B...

    work page 2022

  3. [3]

    Food-101–mining discriminative components with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101–mining discriminative components with random forests. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VI 13, pages 446–461. Springer, 2014. 6

    work page 2014

  4. [4]

    Convex Optimiza- tion

    Stephen Boyd and Lieven Vandenberghe. Convex Optimiza- tion. Cambridge University Press, 2004. 5

    work page 2004

  5. [5]

    Olora: Orthonormal low-rank adapta- tion of large language models, 2024

    Kerim B ¨uy¨ukaky¨uz. Olora: Orthonormal low-rank adapta- tion of large language models, 2024. 3

    work page 2024

  6. [6]

    Domain prompt learning with quaternion networks

    Qinglong Cao, Zhengqin Xu, Yuntian Chen, Chao Ma, and Xiaokang Yang. Domain prompt learning with quaternion networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 26637–26646, 2024. 2

    work page 2024

  7. [7]

    Plot: Prompt learning with optimal transport for vision-language models

    Guangyi Chen, Weiran Yao, Xiangchen Song, Xinyue Li, Yongming Rao, and Kun Zhang. Plot: Prompt learning with optimal transport for vision-language models. In The Eleventh International Conference on Learning Representa- tions, 2022. 1, 2, 6

    work page 2022

  8. [8]

    Unleashing the power of meta-tuning for few-shot generalization through sparse interpolated experts

    Shengzhuang Chen, Jihoon Tack, Yunqiao Yang, Yee Whye Teh, Jonathan Richard Schwarz, and Ying Wei. Unleashing the power of meta-tuning for few-shot generalization through sparse interpolated experts. In Forty-first International Con- ference on Machine Learning, 2024. 2

    work page 2024

Show all 80 references
  1. [9]

    Describing textures in the wild

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3606–3613, 2014. 6

    2014

  2. [10]

    Har- monizing generalization and personalization in federated prompt learning

    Tianyu Cui, Hongxia Li, Jingya Wang, and Ye Shi. Har- monizing generalization and personalization in federated prompt learning. In Proceedings of the 41st International Conference on Machine Learning, pages 9646–9661. PMLR,

  3. [11]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009. 6

    2009

  4. [12]

    Qlora: Efficient finetuning of quantized llms

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. arXiv preprint arXiv:2305.14314, 2023. 3

    2023 arXiv

  5. [13]

    A baseline for few-shot image clas- sification

    Guneet Singh Dhillon, Pratik Chaudhari, Avinash Ravichan- dran, and Stefano Soatto. A baseline for few-shot image clas- sification. In International Conference on Learning Repre- sentations, 2020. 2

    2020

  6. [14]

    Domain-agnostic mutual prompting for unsuper- vised domain adaptation

    Zhekai Du, Xinyao Li, Fengling Li, Ke Lu, Lei Zhu, and Jingjing Li. Domain-agnostic mutual prompting for unsuper- vised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 23375–23384, 2024. 2

    2024

  7. [15]

    Learning gener- ative visual models from few training examples: An incre- mental bayesian approach tested on 101 object categories

    Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning gener- ative visual models from few training examples: An incre- mental bayesian approach tested on 101 object categories. In 2004 conference on computer vision and pattern recognition workshop, pages 178–178. IEEE, 2004. 6

    2004

  8. [16]

    Instance-based max-margin for practical few-shot recognition

    Minghao Fu and Ke Zhu. Instance-based max-margin for practical few-shot recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 28674–28683, 2024. 2

    2024

  9. [17]

    ROSA: Random orthogonal subspace adaptation

    Marawan Gamal and Guillaume Rabusseau. ROSA: Random orthogonal subspace adaptation. In Workshop on Efficient Systems for Foundation Models @ ICML2023, 2023. 3

    2023

  10. [18]

    Clip-adapter: Better vision-language models with feature adapters

    Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with feature adapters. International Journal of Computer Vision , pages 1–15, 2023. 1, 2, 6

    2023

  11. [19]

    Parameter-efficient fine-tuning with discrete fourier transform

    Ziqi Gao, Qichao Wang, Aochuan Chen, Zijing Liu, Bingzhe Wu, Liang Chen, and Jia Li. Parameter-efficient fine-tuning with discrete fourier transform. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Aus- tria, July 21-27, 2024, 2024. 3

    2024

  12. [20]

    Hassan Gharoun, Fereshteh Momenifar, Fang Chen, and Amir H. Gandomi. Meta-learning approaches for few-shot 9 learning: A survey of recent advances. ACM Comput. Surv., 56(12), 2024. 2

    2024

  13. [21]

    Exploring the frontier of vision- language models: A survey of current methodologies and future directions, 2024

    Akash Ghosh, Arkadeep Acharya, Sriparna Saha, Vinija Jain, and Aman Chadha. Exploring the frontier of vision- language models: A survey of current methodologies and future directions, 2024. 1

    2024

  14. [22]

    Anchor- based robust finetuning of vision-language models

    Jinwei Han, Zhiwen Lin, Zhongyisun Sun, Yingguo Gao, Ke Yan, Shouhong Ding, Yuan Gao, and Gui-Song Xia. Anchor- based robust finetuning of vision-language models. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 26919–26928, 2024. 2

    2024

  15. [23]

    LoRA+: Effi- cient low rank adaptation of large models

    Soufiane Hayou, Nikhil Ghosh, and Bin Yu. LoRA+: Effi- cient low rank adaptation of large models. In Proceedings of the 41st International Conference on Machine Learning , pages 17783–17806. PMLR, 2024. 3

    2024

  16. [24]

    Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217–2226, 2019. 6

    2019

  17. [25]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. InIn- ternational Conference on Learning Representations , 2022. 1, 3

    2022

  18. [26]

    van Rijn

    Mike Huisman, Aske Plaat, and Jan N. van Rijn. Subspace adaptation prior for few-shot learning, 2023. 2, 3

    2023

  19. [27]

    Scaling up visual and vision-language representa- tion learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representa- tion learning with noisy text supervision. In International conference on machine learning, pages 4904–4916. PMLR,

  20. [28]

    Singular Value Decomposition , pages 1174–1177

    Kenichi Kanatani. Singular Value Decomposition , pages 1174–1177. Springer International Publishing, Cham, 2021. 3, 4

    2021

  21. [29]

    Combining weakly and webly supervised learning for classifying food images, 2017

    Parneet Kaur, Karan Sikka, and Ajay Divakaran. Combining weakly and webly supervised learning for classifying food images, 2017. 6

    2017

  22. [30]

    Maple: Multi-modal prompt learning

    Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. Maple: Multi-modal prompt learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19113–19122, 2023. 1, 2, 6

    2023

  23. [31]

    Self-regulating prompts: Foundational model adaptation without forgetting

    Muhammad Uzair Khattak, Syed Talal Wasim, Muzam- mal Naseer, Salman Khan, Ming-Hsuan Yang, and Fa- had Shahbaz Khan. Self-regulating prompts: Foundational model adaptation without forgetting. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pa...

    2023

  24. [32]

    Memory-efficient fine-tuning of compressed large language models via sub-4-bit integer quantization, 2023

    Jeonghoon Kim, Jung Hyun Lee, Sungdong Kim, Joonsuk Park, Kang Min Yoo, Se Jung Kwon, and Dongsoo Lee. Memory-efficient fine-tuning of compressed large language models via sub-4-bit integer quantization, 2023. 3

    2023

  25. [33]

    NOLA: Compressing lora using linear combination of random ba- sis

    Soroush Abbasi Koohpayegani, Navaneet K L, Parsa Noora- linejad, Soheil Kolouri, and Hamed Pirsiavash. NOLA: Compressing lora using linear combination of random ba- sis. In The Twelfth International Conference on Learning Representations, 2024. 3

    2024

  26. [34]

    VeRA: Vector-based random matrix adaptation

    Dawid Jan Kopiczko, Tijmen Blankevoort, and Yuki M Asano. VeRA: Vector-based random matrix adaptation. In The Twelfth International Conference on Learning Represen- tations, 2024. 3

    2024

  27. [35]

    3d object representations for fine-grained categorization

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In Proceedings of the IEEE international conference on com- puter vision workshops, pages 554–561, 2013. 6

    2013

  28. [36]

    Beyond max-margin: Class margin equi- librium for few-shot object detection

    Bohao Li, Boyu Yang, Chang Liu, Feng Liu, Rongrong Ji, and Qixiang Ye. Beyond max-margin: Class margin equi- librium for few-shot object detection. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7359–7368, 2021. 2

    2021

  29. [37]

    Measuring the intrinsic dimension of objective landscapes

    Chunyuan Li, Heerad Farkhoor, Rosanne Liu, and Jason Yosinski. Measuring the intrinsic dimension of objective landscapes. In International Conference on Learning Rep- resentations, 2018. 2

    2018

  30. [38]

    Inflora: Interference-free low-rank adaptation for continual learning

    Yan-Shuo Liang and Wu-Jun Li. Inflora: Interference-free low-rank adaptation for continual learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23638–23647, 2024. 2, 3

    2024

  31. [39]

    Svft: Parameter-efficient fine-tuning with singular vectors,

    Vijay Lingam, Atula Tejaswi, Aditya Vavre, Aneesh Shetty, Gautham Krishna Gudur, Joydeep Ghosh, Alex Dimakis, Eunsol Choi, Aleksandar Bojchevski, and Sujay Sanghavi. Svft: Parameter-efficient fine-tuning with singular vectors,

  32. [40]

    Fine-grained visual classi- fication of aircraft

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained visual classi- fication of aircraft. arXiv preprint arXiv:1306.5151 , 2013. 6

    2013 arXiv

  33. [41]

    Transductive zero-shot and few-shot clip

    S ´egol`ene Martin, Yunshi Huang, Fereshteh Shakeri, Jean- Christophe Pesquet, and Ismail Ben Ayed. Transductive zero-shot and few-shot clip. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 28816–28826, 2024. 2

    2024

  34. [42]

    Pissa: Prin- cipal singular values and singular vectors adaptation of large language models

    Fanxu Meng, Zhaohui Wang, and Muhan Zhang. Pissa: Prin- cipal singular values and singular vectors adaptation of large language models. arXiv preprint arXiv:2404.02948 , 2024. 3, 4

    2024 arXiv

  35. [43]

    Automated flower classification over a large number of classes

    Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & im- age processing, pages 722–729. IEEE, 2008. 6

    2008

  36. [44]

    Black box few-shot adaptation for vision- language models

    Yassine Ouali, Adrian Bulat, Brais Matinez, and Georgios Tzimiropoulos. Black box few-shot adaptation for vision- language models. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 15534–15546,

  37. [45]

    Jinyoung Park, Juyeon Ko, and Hyunwoo J. Kim. Prompt learning via meta-regularization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 26940–26950, 2024. 2

    2024

  38. [46]

    Cats and dogs

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE conference on 10 computer vision and pattern recognition, pages 3498–3505. IEEE, 2012. 6

    2012

  39. [47]

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and C. V . Jawahar. Cats and dogs. In 2012 IEEE Conference on Computer Vision and Pattern Recognition , pages 3498– 3505, 2012. 6

    2012

  40. [48]

    Semi- orthogonal low-rank matrix factorization for deep neural net- works

    Daniel Povey, Gaofeng Cheng, Yiming Wang, Ke Li, Hainan Xu, Mahsa Yarmohammadi, and Sanjeev Khudanpur. Semi- orthogonal low-rank matrix factorization for deep neural net- works. In Interspeech, pages 3743–3747, 2018. 2

    2018

  41. [49]

    Learning transferable visual models from natural language supervision, 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. 1, 5

    2021

  42. [50]

    Low-rank matrix factorization for deep neural network training with high- dimensional output targets

    Tara N Sainath, Brian Kingsbury, Vikas Sindhwani, Ebru Arisoy, and Bhuvana Ramabhadran. Low-rank matrix factorization for deep neural network training with high- dimensional output targets. In 2013 IEEE international con- ference on acoustics, speech and signal processing , pa...

    2013

  43. [51]

    Model-agnostic graph regularization for few-shot learning, 2021

    Ethan Shen, Maria Brbic, Nicholas Monath, Jiaqi Zhai, Manzil Zaheer, and Jure Leskovec. Model-agnostic graph regularization for few-shot learning, 2021. 2

    2021

  44. [52]

    Ucf101: A dataset of 101 human actions classes from videos in the wild

    Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402, 2012. 6

    2012 arXiv

  45. [53]

    Block- mix: Meta regularization and self-calibrated inference for metric-based meta-learning

    Hao Tang, Zechao Li, Zhimao Peng, and Jinhui Tang. Block- mix: Meta regularization and self-calibrated inference for metric-based meta-learning. In Proceedings of the 28th ACM International Conference on Multimedia , pages 610–618, New York, NY , USA, 2020. Association for Com...

    2020

  46. [54]

    Argue: Attribute-guided prompt tuning for vision-language models

    Xinyu Tian, Shu Zou, Zhaoyuan Yang, and Jing Zhang. Argue: Attribute-guided prompt tuning for vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 28578–28587, 2024. 2

    2024

  47. [55]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 5

    2017

  48. [56]

    Lora-ga: Low-rank adaptation with gradient approximation, 2024

    Shaowen Wang, Linxi Yu, and Jian Li. Lora-ga: Low-rank adaptation with gradient approximation, 2024. 3, 4, 5

    2024

  49. [57]

    Or- thogonal subspace learning for language model continual learning

    Xiao Wang, Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, and Xuanjing Huang. Or- thogonal subspace learning for language model continual learning. arXiv preprint arXiv:2310.14152, 2023. 2, 3

    2023 arXiv

  50. [58]

    Lora-pro: Are low-rank adapters properly optimized? arXiv preprint arXiv:2407.18242, 2024

    Zhengbo Wang, Jian Liang, Ran He, Zilei Wang, and Tieniu Tan. Lora-pro: Are low-rank adapters properly optimized? arXiv preprint arXiv:2407.18242, 2024. 3

    2024 arXiv

  51. [59]

    Sun database: Large-scale scene recognition from abbey to zoo

    Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE computer so- ciety conference on computer vision and pattern recognition, pages 3485–3492. IEEE, 2010. 6

    2010

  52. [60]

    Zehao Xiao, Jiayi Shen, Mohammad Mahdi Derakhshani, Shengcai Liao, and Cees G. M. Snoek. Any-shift prompt- ing for generalization over distributions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13849–13860, 2024. 2

    2024

  53. [61]

    Mma: Multi-modal adapter for vision-language models

    Lingxiao Yang, Ru-Yuan Zhang, Yanchen Wang, and Xiao- hua Xie. Mma: Multi-modal adapter for vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 23826–23837, 2024. 2, 3, 5, 6

    2024

  54. [62]

    Visual- language prompt tuning with knowledge-guided context op- timization

    Hantao Yao, Rui Zhang, and Changsheng Xu. Visual- language prompt tuning with knowledge-guided context op- timization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6757– 6767, 2023. 1, 2, 6

    2023

  55. [63]

    Tcp:textual- based class-aware prompt tuning for visual-language model

    Hantao Yao, Rui Zhang, and Changsheng Xu. Tcp:textual- based class-aware prompt tuning for visual-language model. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 23438– 23448, 2024. 2

    2024

  56. [64]

    Lever- aging cross-modal neighbor representation for improved clip classification

    Chao Yi, Lu Ren, De-Chuan Zhan, and Han-Jia Ye. Lever- aging cross-modal neighbor representation for improved clip classification. In CVPR, 2024. 2

    2024

  57. [65]

    Task residual for tuning vision-language models

    Tao Yu, Zhihe Lu, Xin Jin, Zhibo Chen, and Xinchao Wang. Task residual for tuning vision-language models. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10899–10909, 2023. 2, 6

    2023

  58. [66]

    Low-rank few-shot adaptation of vision-language models

    Maxime Zanella and Ismail Ben Ayed. Low-rank few-shot adaptation of vision-language models. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1593–1603, 2024. 1, 3, 6

    2024

  59. [67]

    Sim- ple semantic-aided few-shot learning

    Hai Zhang, Junzhe Xu, Shanlin Jiang, and Zhenan He. Sim- ple semantic-aided few-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 28588–28597, 2024. 2

    2024

  60. [68]

    Dept: Decoupled prompt tuning

    Ji Zhang, Shihan Wu, Lianli Gao, Heng Tao Shen, and Jingkuan Song. Dept: Decoupled prompt tuning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12924–12933, 2024. 2

    2024

  61. [69]

    Adaptive budget allocation for parameter-efficient fine- tuning

    Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. Adaptive budget allocation for parameter-efficient fine- tuning. In The Eleventh International Conference on Learn- ing Representations, 2023. 3

    2023

  62. [70]

    Tip- adapter: Training-free adaption of clip for few-shot classifi- cation

    Renrui Zhang, Wei Zhang, Rongyao Fang, Peng Gao, Kun- chang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li. Tip- adapter: Training-free adaption of clip for few-shot classifi- cation. In European Conference on Computer Vision, pages 493–510. Springer, 2022. 1, 2, 6

  63. [71]

    Ex- tracting deep neural network bottleneck features using low- rank matrix factorization

    Yu Zhang, Ekapol Chuangsuwanich, and James Glass. Ex- tracting deep neural network bottleneck features using low- rank matrix factorization. In 2014 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 185–189. IEEE, 2014. 2

    2014

  64. [72]

    Large language models are good prompt learners for low-shot image classification

    Zhaoheng Zheng, Jingmin Wei, Xuefeng Hu, Haidong Zhu, and Ram Nevatia. Large language models are good prompt learners for low-shot image classification. In CVPR, 2024. 3, 6 11

    2024

  65. [73]

    Conditional prompt learning for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Zi- wei Liu. Conditional prompt learning for vision-language models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 1, 2

    2022

  66. [74]

    Learning to prompt for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. Inter- national Journal of Computer Vision (IJCV), 2022. 1, 2, 6

    2022

  67. [75]

    Prompt-aligned gradient for prompt tuning

    Beier Zhu, Yulei Niu, Yucheng Han, Yue Wu, and Han- wang Zhang. Prompt-aligned gradient for prompt tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15659–15669, 2023. 2, 6

    2023

  68. [76]

    Not all features mat- ter: Enhancing few-shot clip with adaptive prior refinement

    Xiangyang Zhu, Renrui Zhang, Bowei He, Aojun Zhou, Dong Wang, Bin Zhao, and Peng Gao. Not all features mat- ter: Enhancing few-shot clip with adaptive prior refinement. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 2605–2615, 2023. 1, 2, 6

    2023

  69. [77]

    Laplacian regularized few-shot learning, 2021

    Imtiaz Masud Ziko, Jose Dolz, Eric Granger, and Ismail Ben Ayed. Laplacian regularized few-shot learning, 2021. 2 1 Complementary Subspace Low-Rank Adaptation of Vision-Language Models for Few-Shot Classification Supplementary Material

    2021

  70. [78]

    Complete Few Shot Experiments We present the complete experimental results of {1,2,4,8,16}-shots in Tab. 3. On most of these few shot datasets, our Comp-LoRA method achieves the highest or second highest performance

  71. [79]

    Extra Generality Experiments We conducted more designed experiments on catastrophic forgetting problem as shown in Tabs. 4 to 13. We fine-tuned on one few shot support set through the proposed and the baseline methods, and then tested their reserved zero shot classification ab...

  72. [80]

    7 to 16, the univariate experiment on other experiments presents similar trend to that of ImageNet in Fig

    Complementary Dimension Experiments As shown in Figs. 7 to 16, the univariate experiment on other experiments presents similar trend to that of ImageNet in Fig. 5. In general, the performance of the proposed Comp-LoRA method increases at the beginning and then decreases when t...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.