Pith. sign in

REVIEW 3 major objections 4 minor 74 references

Regularizing Subspace Redundancy of Low-Rank Adaptation

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Regularizing the overlap between LoRA's rank-1 subspaces boosts fine-tuning accuracy.

desk verdict A clean, plug-and-play subspace redundancy regularizer for LoRA whose main numbers compare 100-epoch baselines to 200-epoch runs; the causal claim needs matched-epoch controls beyond the single LoRA/VTAB case. read the letter →

arxiv 2507.20745 v1 pith:RZ3F6SND submitted 2025-07-28 cs.CV cs.AIcs.MM

Yue Zhu , Haiwen Diao , Shang Gao , Jiazuo Yu , Jiawen Zhu , Yunzhi Zhuge , Shuai Hao , Xu Jia
show 3 more authors
Lu Zhang Ying Zhang Huchuan Lu
This is my paper
classification cs.CVcs.AIcs.MM
keywords parameter-efficienttransferlearninglow-rankadaptationsubspaceredundancyregularizationvision-languageretrievalvisualclassificationfew-shotplug-and-play
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that low-rank adapters like LoRA learn updates whose rank-1 components are highly correlated, wasting representational capacity. It proposes ReSoRA, a training-only penalty that decomposes the low-rank update into equivalent subspaces and explicitly discourages redundancy among their output features. Added to existing parameter-efficient transfer learning methods, the penalty is reported to improve vision-language retrieval and visual classification accuracy consistently, with no additional inference cost. The paper also reports that redundancy persists even at rank 2, and that feature-level regularization outperforms weight-level regularization.

What carries the argument

The central object is the decomposition of the low-rank update into rank-1 subspaces, Δh = Σ_{i=1}^r B_i A_i^T x, where each Δh_i is the contribution of the i-th subspace. The default regularizer is the linear set-to-set term R_l, which measures the normalized Frobenius inner product between batch feature matrices of different subspaces, plus pairwise Euclidean and cosine variants. The paper derives the gradient of the cosine term to show that it actively pushes aligned subspace outputs apart, thereby encouraging orthogonality during training.

What would settle it

On VTAB-1K with ViT-B/16 and LoRA r=8, train plain LoRA for the full 200 epochs with the same schedule, weight decay, and seeds; if it matches the ~75.0 average accuracy of LoRA+ReSoRA, the central claim loses support. The reverse check is to run ReSoRA for only 100 epochs and compare it with the 100-epoch baseline.

Watch

Extended reading notes

Core claim

The paper's central claim is that penalizing overlap between the rank-1 subspaces of a low-rank adapter improves adaptation. Concretely, ReSoRA decomposes the LoRA update into r rank-1 components, Δh = Σ_{i=1}^r Δh_i, and adds a regularization term that suppresses similarity across the Δh_i feature projections. On VTAB-1K this raises the average accuracy of LoRA by 0.7 points, FacT by 0.6, MoSLoRA by 0.3, and DTL by 0.4, and it improves retrieval Rsum on Flickr30K and MSR-VTT by 2.6 and 3.3 points respectively. The regularizer is applied only during training and is designed to integrate with existing low-rank methods without changing inference-time weights or adding latency.

Load-bearing premise

The main tables compare a 100-epoch baseline against a 100-epoch regularized second stage, so the reported gains assume that the improvement comes from the regularizer rather than from the extra training epochs.

Editorial extensions

If this is right

  • Existing LoRA-style adapters can be upgraded by adding one penalty term while keeping the parameter count unchanged.
  • The reported gains span retrieval and classification tasks across multiple backbones and data regimes, including 1-shot to 16-shot few-shot settings.
  • Subspace redundancy is observed even at rank 2, suggesting that merely lowering the rank does not eliminate the problem.
  • Feature-level regularization outperforms weight-level regularization, indicating that shaping representations directly is more effective than constraining parameters implicitly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the reported gains should be re-checked with a matched training budget, since the main tables compare a 100-epoch baseline with a 100-epoch regularized second stage; Figure 4 suggests extra epochs alone hurt, but that control is not carried into the main tables.
  • Editorial extension: ReSoRA targets overlap rather than magnitude importance, so it could be combined with rank-allocation methods such as AdaLoRA or SoRA to address two distinct failure modes at once.
  • Editorial extension: the linear set-to-set term is essentially a normalized cross-covariance penalty, so a similar mechanism could transfer to other PETL forms or even full fine-tuning of smaller models.
  • Editorial extension: a testable prediction of the paper is that the average pairwise cosine similarity between subspace outputs should drop as ReSoRA trains, and the drop should correlate with the accuracy gain.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces ReSoRA, a training-time regularizer for low-rank adaptation methods such as LoRA, FacT, MoSLoRA, and DTL. The adaptive weight update is decomposed into rank-1 subspace contributions, and a regularizer penalizes pairwise or set-to-set similarity between the corresponding feature projections. Experiments on image-text retrieval (Flickr30K), video-text retrieval (MSR-VTT), VTAB-1K classification, and few-shot classification report consistent accuracy gains over the unregularized baselines across several backbones, with no additional inference cost.

Significance. If the reported gains are genuinely caused by the regularizer, ReSoRA is a practically valuable contribution: it is simple, plug-and-play, incurs no inference overhead, and shows positive results across multiple PETL methods and backbones. The paper includes a public code release, an ablation of regularizer variants, memory and training-time analysis, and an explicit attempt to control for the two-stage training protocol in Figure 4. The main weakness is that the central causal claim is not supported by matched training budgets in most of the reported comparisons.

major comments (3)
  1. [§4.2, Training Strategy; Tables 1-3] The comparison between baselines and "+ReSoRA" is not matched in training epochs. The text states that baselines are trained for 100 epochs in stage one and ReSoRA is then added for another 100 epochs, so every regularized result receives 200 total epochs while the baseline receives only 100. This confounds the effect of the regularizer with the effect of additional optimization. Figure 4 provides a useful control for LoRA on VTAB-1K, showing that continuing stage-one training without ReSoRA can degrade accuracy, but no such matched 200-epoch baseline is reported for FacT, MoSLoRA, DTL, the Swin/ConvNeXt experiments, the retrieval benchmarks in Table 1, or the few-shot curves in Figure 3. Since the reported deltas are often small (0.1-0.4%), the claim that ReSoRA "consistently facilitates" these methods requires matched-epoch baselines for every main result, or an equal-budget protocol in which the regularizer is present from the start of training.
  2. [§4.2, Tables 2-3; §4.3, Figure 3] The main VTAB-1K tables report single-run results without error bars, and several per-dataset deltas are 0.0 or negative even when the average is positive. For claims of consistent improvement, especially with average gains below 1%, a single run cannot distinguish a true regularizer effect from run-to-run variance. The few-shot section states that results are averaged over three seeds, but Figure 3 does not show the variance. Please provide mean and standard deviation over at least three seeds for the main comparisons, and ideally for the matched 200-epoch baselines requested above.
  3. [Eq. (1), Eq. (2), Eq. (7)] There is a dimensional inconsistency in the notation for the low-rank decomposition. If W0 is defined as d_in x d_out and x is d_in, then W0 x is undefined for the output h in R^{d_out}; the standard LoRA convention is W0 in R^{d_out x d_in}, B in R^{d_out x r}, A in R^{r x d_in}. Additionally, Eq. (7) defines the set-level feature matrix ΔH_i as being in R^{d_in x N}, whereas the vector features Δh_i in Eq. (4) and (6) live in R^{d_out}. This obscures which feature space is being regularized and should be corrected in a revision.
minor comments (4)
  1. [§3.3, Theoretical Justification] The theoretical justification is only a gradient derivation for the cosine term; it does not establish a formal relationship between reducing pairwise cosine similarity and improving generalization or reducing representational redundancy. I recommend softening the word "theoretically" in the abstract and Section 1, or adding a formal statement with assumptions.
  2. [Table 3] The delta notation "0.2↓0.4↑..." is difficult to parse. Use separate delta columns with explicit signs (e.g., -0.2, +0.4) and fill missing cells consistently.
  3. [Table 6] The text describes the overhead as "minimal" and "modest," but the table shows FLOPs increasing from 0.56 to 0.83 T for LoRA (about +48%) and training time increasing by up to 34%. Please either qualify this more precisely or discuss whether the accuracy gain is worth the added training compute relative to a matched-cost baseline.
  4. [Throughout] There are several typographical issues: "AdaLoR" in Table 2 should be "AdaLoRA," the VSE∞ name renders awkwardly, Table 1 contains an unexplained "*4" marker, and the symbol for rank is inconsistently given as r, n, and "𝑛" in Section 5.2. These should be cleaned up.

Circularity Check

1 steps flagged · score 2.0 of 10

Mild selection-circularity: the default ReSoRA variant is chosen on the same VTAB-1K benchmark later used as main evidence; otherwise the derivation is a heuristic training objective and not circular.

  1. other [Section 4.2, Implementation Details; Section 5.2, Table 5]
    "ReSoRA is applied with linear set-to-set regularization by default. | Among all the strategies evaluated, the linear set-to-set item stands out by delivering the most significant improvements..."

    The default variant that produces the headline VTAB-1K results is not fixed a priori; it is chosen after seeing the same benchmark's ablation. Section 5.2 ranks four regularizers on VTAB-1K and selects the linear set-to-set term as best, and Section 4.2 then makes that selected variant the default for the main comparisons. The reported 0.7% average gain is therefore the best of several candidate configurations evaluated on the same test benchmark, not an out-of-sample prediction of a pre-specified regularizer. This is a weak 'fitted input called prediction': the model choice is fitted to the target benchmark, and the same benchmark is then cited as evidence.

full rationale

The paper's derivation chain is mostly self-contained. Eq. (2) is an algebraic decomposition of BA into rank-1 terms; Eqs. (6)-(8) define new regularizers; Eq. (10) is a gradient computation for Eq. (9). None of these assume the target accuracy result. The claim that ReSoRA reduces subspace similarity is true by construction because the loss is that similarity, but it is not presented as an independent prediction; the load-bearing claim is the accuracy improvement, which is empirical and not forced by the equations. No self-citation is load-bearing: citations to the authors' prior work are related-work context, and the two-stage training protocol cites an external paper [26]. There is no imported uniqueness theorem and no ansatz smuggled via self-citation. The main experimental tables compare 100-epoch baselines to 200-epoch ReSoRA runs; this is a control/confound issue rather than a circularity, and Figure 4 partially addresses it for LoRA on VTAB. The one mild circular element is the selection of the linear set-to-set variant from an ablation on the same VTAB-1K benchmark used for the main tables, which makes the reported default a fitted choice rather than a pre-registered one. This raises the score to 2 but does not undermine the independent content of the regularizer.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

ReSoRA introduces no new physical or architectural entities; it is an auxiliary penalty on existing LoRA parameters. The main commitments are the domain assumption that orthogonality helps, and the protocol choice of selecting the best-performing regularizer variant from the evaluation benchmarks themselves.

free parameters (3)
  • beta in Eq. (5)
    Scaling factor in the Euclidean pairwise regularizer; value not reported in the paper.
  • sigma in Eq. (8)
    Gaussian kernel width in the nonlinear set-to-set regularizer; set to a fraction of median sample distances but the fraction is not specified.
  • choice of default regularizer = linear set-to-set
    Selected after comparing Re, Rc, Rn, and Rl on VTAB-1K; this is a model-selection choice informed by the same benchmarks used for the headline claims.
assumptions (5)
  • standard math The rank-1 decomposition of the LoRA update into subspaces is valid (Eq. 2).
    Follows from linear algebra, since BA equals the sum of outer products of corresponding columns and rows.
  • domain assumption Reducing inter-subspace similarity improves downstream transfer accuracy.
    This is the central inductive belief, supported only by ablations and not by a theorem or a prior fixed result.
  • domain assumption A mini-batch of N samples is representative of the feature distribution for computing redundancy.
    The regularizer uses empirical batch features; the paper does not analyze the effect of batch size or the sensitivity to N.
  • ad hoc to paper Two-stage training is a fair and necessary protocol for comparing regularizers.
    The paper trains baselines for 100 epochs in stage one and regularized models for another 100 epochs, without matching total epochs in the main tables.
  • ad hoc to paper The default linear set-to-set regularizer is better than alternatives.
    Chosen from the ablation results on the same benchmark used for the final evaluation, which risks overfitting the choice to the benchmark.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Regularizing Subspace Redundancy of Low-Rank Adaptation." pith.science (2026). https://pith.science/paper/RZ3F6SND

@misc{pith2026250720745,
  author       = {Pith},
  title        = {Pith review of: Regularizing Subspace Redundancy of Low-Rank Adaptation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RZ3F6SND}},
  note         = {Machine review of arXiv:2507.20745}
}
read the original abstract

Low-Rank Adaptation (LoRA) and its variants have delivered strong capability in Parameter-Efficient Transfer Learning (PETL) by minimizing trainable parameters and benefiting from reparameterization. However, their projection matrices remain unrestricted during training, causing high representation redundancy and diminishing the effectiveness of feature adaptation in the resulting subspaces. While existing methods mitigate this by manually adjusting the rank or implicitly applying channel-wise masks, they lack flexibility and generalize poorly across various datasets and architectures. Hence, we propose ReSoRA, a method that explicitly models redundancy between mapping subspaces and adaptively Regularizes Subspace redundancy of Low-Rank Adaptation. Specifically, it theoretically decomposes the low-rank submatrices into multiple equivalent subspaces and systematically applies de-redundancy constraints to the feature distributions across different projections. Extensive experiments validate that our proposed method consistently facilitates existing state-of-the-art PETL methods across various backbones and datasets in vision-language retrieval and standard visual classification benchmarks. Besides, as a training supervision, ReSoRA can be seamlessly integrated into existing approaches in a plug-and-play manner, with no additional inference costs. Code is publicly available at: https://github.com/Lucenova/ReSoRA.

Figures

Figures reproduced from arXiv: 2507.20745 by the authors.

Figure 1
Figure 1. Comparison between our ReSoRA and LoRA [ [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of our proposed ReSoRA. The original output space is decomposed into a combination of multiple subspaces. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Few-shot results on five fine-grained recognition datasets. Solid lines indicate PETL methods with added ReSoRA; [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Effective training by adding ReSoRA in the second stage. The curves show test accuracy across datasets during training. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

74 extracted references · 50 canonical work pages

  1. [1]

    Charles Beattie, Joel Z Leibo, Denis Teplyashin, Tom Ward, Marcus Wainwright, Heinrich Küttler, Andrew Lefrancq, Simon Green, Víctor Valdés, Amir Sadik, et al. 2016. Deepmind lab.arXiv:1612.03801(2016)

    arXiv 2016

  2. [2]

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. 2014. Food-101–mining discriminative components with random forests. InECCV. Springer, 446–461

    work page 2014

  3. [3]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners.Advances in neural information processing systems33 (2020), 1877–1901

    2020

  4. [4]

    Jiacheng Chen, Hexiang Hu, Hao Wu, Yuning Jiang, and Changhu Wang. 2021. Learning the Best Pooling Strategy for Visual Semantic Embedding. InCVPR. 15789–15798

    work page 2021

  5. [5]

    Shoufa Chen, Chongjian Ge, Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang, and Ping Luo. 2022. Adaptformer: Adapting vision transformers for scalable visual recognition.Advances in Neural Information Processing Systems35 (2022), 16664–16678

    2022

  6. [6]

    Xiangyu Chen, Jing Liu, Ye Wang, Matthew Brand, Guanghui Wang, Toshiaki Koike-Akino, et al. 2024. SuperLoRA: Parameter-Efficient Unified Adaptation of Multi-Layer Attention Modules.arXiv:2403.11887(2024)

    arXiv 2024

  7. [7]

    Zhe Chen, Yuchen Duan, Wenhai Wang, Junjun He, Tong Lu, Jifeng Dai, and Yu Qiao. 2023. Vision Transformer Adapter for Dense Predictions. InICLR

    work page 2023

  8. [8]

    Gong Cheng, Junwei Han, and Xiaoqiang Lu. 2017. Remote sensing image scene classification: Benchmark and state of the art.Proc. IEEE105, 10 (2017), 1865– 1883

    work page 2017

Show all 74 references
  1. [9]

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and An- drea Vedaldi. 2014. Describing textures in the wild. InCVPR. 3606–3613

    2014

  2. [10]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL. 4171–4186

    2019

  3. [11]

    Haiwen Diao, Yufeng Cui, Xiaotong Li, Yueze Wang, Huchuan Lu, and Xinlong Wang. 2024. Unveiling Encoder-Free Vision-Language Models. InNeurIPS

    2024

  4. [12]

    Haiwen Diao, Xiaotong Li, Yufeng Cui, Yueze Wang, Haoge Deng, Ting Pan, Wenxuan Wang, Huchuan Lu, and Xinlong Wang. 2025. EVEv2: Improved Base- lines for Encoder-Free Vision-Language Models.arXiv:2502.06788(2025)

    2025 arXiv

  5. [13]

    Haiwen Diao, Bo Wan, Xu Jia, Yunzhi Zhuge, Ying Zhang, Huchuan Lu, and Long Chen. 2025. SHERL: Synthesizing High Accuracy and Efficient Memory for Resource-Limited Transfer Learning. InECCV. 75–95

    2025

  6. [14]

    Haiwen Diao, Bo Wan, Ying Zhang, Xu Jia, Huchuan Lu, and Long Chen. 2024. UniPT: Universal Parallel Tuning for Transfer Learning with Efficient Parameter and Memory. InCVPR

    2024

  7. [15]

    Haiwen Diao, Ying Zhang, Shang Gao, Jiawen Zhu, Long Chen, and Huchuan Lu. 2024. GSSF: Generalized Structural Sparse Function for Deep Cross-Modal Metric Learning.TIP33 (2024), 6241–6252

    2024

  8. [16]

    Ning Ding, Xingtai Lv, Qiaosen Wang, Yulin Chen, Bowen Zhou, Zhiyuan Liu, and Maosong Sun. 2023. Sparse Low-rank Adaptation of Pre-trained Language Models. InEMNLP

    2023

  9. [17]

    Wei Dong, Yuan Sun, Yiting Yang, Xing Zhang, Zhijun Lin, Qingsen Yan, Haokui Zhang, Peng Wang, Yang Yang, and Hengtao Shen. 2024. Efficient Adaptation of Pre-trained Vision Transformer via Householder Transformation.arXiv: 2410.22952(2024)

    2024 arXiv

  10. [18]

    Wei Dong, Dawei Yan, Zhijun Lin, and Peng Wang. 2023. Efficient adaptation of large vision transformer via adapter re-composing.Advances in Neural Informa- tion Processing Systems36 (2023), 52548–52567

    2023

  11. [19]

    Wei Dong, Xing Zhang, Bihui Chen, Dawei Yan, Zhijun Lin, Qingsen Yan, Peng Wang, and Yang Yang. 2024. Low-rank rescaled vision transformer fine-tuning: A residual design approach. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 16101–16110

    2024

  12. [20]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recogn...

    2021

  13. [21]

    Li Fei-Fei, Robert Fergus, and Pietro Perona. 2006. One-shot learning of object categories.PAMI28, 4 (2006), 594–611

    2006

  14. [22]

    Minghao Fu, Ke Zhu, and Jianxin Wu. 2024. Dtl: Disentangled transfer learning for visual recognition. InAAAI, Vol. 38. 12082–12090

    2024

  15. [23]

    Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. 2021. CLIP-Adapter: Better Vision-Language Models with Feature Adapters.arXiv: 2110.04544(2021)

    2021 arXiv

  16. [24]

    A Geiger, P Lenz, C Stiller, and R Urtasun. 2013. Vision meets robotics: The KITTI dataset.The International Journal of Robotics Research(2013), 1–6

    2013

  17. [25]

    Rush, and Yoon Kim

    Demi Guo, Alexander M. Rush, and Yoon Kim. 2021. Parameter-Efficient Transfer Learning with Diff Pruning. InACL. 4884–4896

    2021

  18. [26]

    Haoyu He, Jianfei Cai, Jing Zhang, Dacheng Tao, and Bohan Zhuang. 2023. Sensitivity-aware visual parameter-efficient fine-tuning. InICCV. 11825–11835

    2023

  19. [27]

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. 2019. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification.IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing12, 7 (2019), 2217–2226

    2019

  20. [28]

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-Efficient Transfer Learning for NLP. InICML (Proceedings of Machine Learning Research, Vol. 97). 2790–2799

    2019

  21. [29]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. InICLR

    2022

  22. [30]

    Belongie, Bharath Hariharan, and Ser-Nam Lim

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge J. Belongie, Bharath Hariharan, and Ser-Nam Lim. 2022. Visual Prompt Tuning. InECCV, Vol. 13693. 709–727

    2022

  23. [31]

    Shibo Jie and Zhi-Hong Deng. 2023. FacT: Factor-Tuning for Lightweight Adap- tation on Vision Transformer. InAAAI

    2023

  24. [32]

    Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. 2017. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. InCVPR. 2901–2910

    2017

  25. [33]

    Chen Ju, Tengda Han, Kunhao Zheng, Ya Zhang, and Weidi Xie. 2022. Prompting Visual-Language Models for Efficient Video Understanding. InECCV, Vol. 13695. 105–124

    2022

  26. [34]

    EyePacs Kaggle. 2015. Kaggle Diabetic Retinopathy Detection. https://www. kaggle.com/c/diabetic-retinopathy-detection Accessed: Sep. 12, 2024

    2015

  27. [35]

    Konwoo Kim, Michael Laskin, Igor Mordatch, and Deepak Pathak. 2021. How to Adapt Your Large-Scale Vision-and-Language Model.openreview(2021)

    2021

  28. [36]

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 2013. 3d object repre- sentations for fine-grained categorization. InICCVW. 554–561

    2013

  29. [37]

    Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. (2009)

    2009

  30. [38]

    Yann LeCun, Fu Jie Huang, and Leon Bottou. 2004. Learning methods for generic object recognition with invariance to pose and lighting. InCVPR, Vol. 2. IEEE, II–104

    2004

  31. [39]

    Xiang Lisa Li and Percy Liang. 2021. Prefix-Tuning: Optimizing Continuous Prompts for Generation. InACL. 4582–4597

    2021

  32. [40]

    Dongze Lian, Daquan Zhou, Jiashi Feng, and Xinchao Wang. 2022. Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning. InNeurIPS

    2022

  33. [41]

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. InICCV. IEEE, 9992–10002

    2021

  34. [42]

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. 2022. A convnet for the 2020s. InCVPR. 11976–11986

    2022

  35. [43]

    Ilya Loshchilov, Frank Hutter, et al. 2017. Fixing weight decay regularization in adam.arXiv:1711.051015 (2017)

    2017 arXiv

  36. [44]

    Huaishao Luo, Lei Ji, Ming Zhong, Yang Chen, Wen Lei, Nan Duan, and Tianrui Li. 2021. CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval.arXiv: 2104.08860(2021)

    2021 arXiv

  37. [45]

    Rabeeh Karimi Mahabadi, James Henderson, and Sebastian Ruder. 2021. Com- pacter: Efficient Low-Rank Hypercomplex Adapter Layers. InNeurIPS. 1022–1035

    2021

  38. [46]

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi

  39. [47]

    Loic Matthey, Irina Higgins, Demis Hassabis, and Alexander Lerchner. 2017. dsprites: Disentanglement testing sprites dataset

    2017

  40. [48]

    Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, An- drew Y Ng, et al. 2011. Reading digits in natural images with unsupervised feature learning. InNIPS workshop on deep learning and unsupervised feature learning, Vol. 2011. Granada, 4

    2011

  41. [49]

    Nilsback and A

    M.-E. Nilsback and A. Zisserman. 2006. A Visual Vocabulary for Flower Classifi- cation. InCVPR, Vol. 2. 1447–1454. doi:10.1109/CVPR.2006.42

    2006 doi

  42. [50]

    Maria-Elena Nilsback and Andrew Zisserman. 2008. Automated flower classifica- tion over a large number of classes. In2008 Sixth Indian conference on computer vision, graphics & image processing. IEEE, 722–729

    2008

  43. [51]

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. 2012. Cats and dogs. InCVPR. IEEE, 3498–3505

    2012

  44. [52]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. InICML, Vol. 1...

    2021

  45. [53]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners.OpenAI blog 1, 8 (2019), 9

    2019

  46. [54]

    Girshick, and Jian Sun

    Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun. 2015. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. InNIPS. 91–99

    2015

  47. [55]

    Yi-Lin Sung, Jaemin Cho, and Mohit Bansal. 2022. LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning. InNeurIPS

    2022

  48. [56]

    Yi-Lin Sung, Jaemin Cho, and Mohit Bansal. 2022. VL-ADAPTER: Parameter- Efficient Transfer Learning for Vision-and-Language Tasks. InCVPR. 5217–5227

    2022

  49. [57]

    Bastiaan S Veeling, Jasper Linmans, Jim Winkens, Taco Cohen, and Max Welling

  50. [58]

    Junjie Wang, Guangjing Yang, Wentao Chen, Huahui Yi, Xiaohu Wu, and Qicheng Lao. 2024. MLAE: Masked LoRA Experts for Parameter-Efficient Fine-Tuning. arXiv:2405.18897(2024)

    2024 arXiv

  51. [59]

    Taiqiang Wu, Jiahao Wang, Zhe Zhao, and Ngai Wong. 2024. Mixture-of- Subspaces in Low-Rank Adaptation.arXiv:2406.11909(2024)

    2024 arXiv

  52. [60]

    Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba

  53. [61]

    Jun Xu, Tao Mei, Ting Yao, and Yong Rui. 2016. MSR-VTT: A Large Video Description Dataset for Bridging Video and Language. InCVPR. 5288–5296

    2016

  54. [62]

    Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions.TACL2 (2014), 67–78

    2014

  55. [63]

    Jiazuo Yu, Haomiao Xiong, Lu Zhang, Haiwen Diao, Yunzhi Zhuge, Lanqing Hong, Dong Wang, Huchuan Lu, You He, and Long Chen. 2024. LLMs Can Evolve Continually on Modality for X-Modal Reasoning. InNeurIPS

    2024

  56. [64]

    Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel. 2022. BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models. InACL. 1–9

    2022

  57. [65]

    Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov, Pierre Ruyssen, Carlos Riquelme, Mario Lucic, Josip Djolonga, Andre Susano Pinto, Maxim Neumann, Alexey Dosovitskiy, et al. 2019. A large-scale study of representation learning with the visual task adaptation benchmark.arXiv...

    2019 arXiv

  58. [66]

    Feiyu Zhang, Liangzhi Li, Junhao Chen, Zhouqiang Jiang, Bowen Wang, and Yiming Qian. 2023. Increlora: Incremental parameter allocation method for parameter-efficient fine-tuning.arXiv:2308.12043(2023)

    2023 arXiv

  59. [67]

    Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. 2023. Adaptive Budget Allocation for Parameter- Efficient Fine-Tuning. InICLR

    2023

  60. [68]

    Ruiyi Zhang, Rushi Qiang, Sai Ashish Somayajula, and Pengtao Xie. 2024. Au- toLoRA: Automatically Tuning Matrix Ranks in Low-Rank Adaptation Based on Meta Learning.arXiv:2403.09113(2024)

    2024 arXiv

  61. [69]

    Yuanhan Zhang, Kaiyang Zhou, and Ziwei Liu. 2022. Neural Prompt Search. arXiv: 2206.04673(2022)

    2022 arXiv

  62. [70]

    Yue Zhu, Haiwen Diao, Shang Gao, Long Chen, and Huchuan Lu. 2025. KARST: Multi-Kernel Kronecker Adaptation with Re-Scaling Transmission for Visual Classification. InICASSP. 1–5

    2025

  63. [71]

    Yun Zhu, Nevan Wichers, Chu-Cheng Lin, Xinyi Wang, Tianlong Chen, Lei Shu, Han Lu, Canoee Liu, Liangchen Luo, Jindong Chen, et al . 2023. Sira: Sparse mixture of low rank adaptation.arXiv:2311.09179(2023)

    2023 arXiv

  64. [2010]

    Sun database: Large-scale scene recognition from abbey to zoo. InCVPR. IEEE, 3485–3492

  65. [2013]

    Fine-grained visual classification of aircraft.arXiv:1306.5151(2013)

    2013 arXiv

  66. [2018]

    InMICCAI

    Rotation equivariant CNNs for digital pathology. InMICCAI. Springer, 210–218

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.