Pith. sign in

REVIEW 4 major objections 5 minor 68 references

MAP: Revisiting Weight Decomposition for Low-Rank Adaptation

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MAP reframes weight updates as normalized vectors and learns two scalars to split direction from magnitude.

desk verdict A simple LoRA variant with a real initialization bug and shaky empirical grounding; worth a revised look but not as written. read the letter →

arxiv 2505.23094 v1 pith:5DOAYA7A submitted 2025-05-29 cs.LG cs.CL

Chongjie Si , Zhiyi Shi , Yadao Wang , Xiaokang Yang , Susanto Rahardja , Wei Shen This is my paper
classification cs.LGcs.CL MSC 68T07
keywords parameter-efficientfine-tuninglow-rankadaptationLoRADoweightdecompositiondirectionandmagnitudeFrobeniusnormnormalizationmatrixvectorization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to establish that direction and magnitude in parameter-efficient fine-tuning are best defined globally, for the entire weight matrix treated as one vector, rather than column by column. To do this it proposes MAP, a layer that normalizes the frozen weight matrix and the low-rank update by their Frobenius norms and lets two learned scalars set the magnitude of each component independently. Wrapped around LoRA, AdaLoRA, or FLoRA, MAP is reported to improve accuracy on commonsense reasoning, GLUE, and image generation with essentially no added parameter cost. If the claim holds, MAP is a near-free drop-in step that could become a default component in later PEFT designs.

What carries the argument

The load-bearing identity is $\|W\|_F = \|\mathrm{vec}(W)\|_2$, which turns a weight matrix into a high-dimensional vector so that 'direction' and 'magnitude' are standard vector concepts. MAP then reparameterizes the adapted weight as $\alpha W/\|W\|_F + \beta AB/\|AB\|_F$, so the two scalar coefficients independently set the magnitudes of the normalized base and update components. This decoupling is what carries the argument: the low-rank term steers the direction while the scalars control scale, and the same layer can be attached to any existing low-rank update.

What would settle it

Re-run LoRA, DoRA, and LoMAP on LLaMA-7B over Commonsense170K at r = 16 with the same hyperparameters, seeds, and evaluation script used in the paper; if LoMAP's average does not exceed DoRA's 78.4, or if the gap vanishes across seeds, the claim of consistent superiority is not supported.

Watch

Extended reading notes

Core claim

MAP's central claim is that decoupling direction from magnitude at the level of the whole matrix, not per column, is the correct geometric reading of a low-rank update. The update is written as $W^* = \alpha \, W/\|W\|_F + \beta \, AB/\|AB\|_F$, where $W$ is the frozen pre-trained matrix, $AB$ is the LoRA-style update, and $\alpha,\beta$ are two learnable scalars per layer. Because $\|W\|_F$ equals the Euclidean norm of the flattened weight vector, the matrix formula is argued to be a true vector-space decomposition, giving direction a principled meaning that DoRA's column-wise normalization lacks. The paper further claims that this frame carries over to any method that models an update matrix, and that coupling MAP with LoRA, AdaLoRA, and FLoRA yields consistent gains.

Load-bearing premise

The argument assumes the baseline scores copied from earlier papers are a fair and directly comparable reference; if those runs used different settings, the reported LoMAP advantage may not reflect a real improvement.

Editorial extensions

If this is right

  • Wrapping a low-rank method with MAP adds only two trainable scalars per matrix, so the overhead over plain LoRA is negligible in parameter count and compute.
  • Because MAP is a drop-in modulation layer, future low-rank PEFT variants could inherit the decomposition without changing their internal update rule.
  • The initialization $\alpha = \|W\|_F, \beta = 1$ starts the adapted model exactly at the pre-trained weights, so the method is a minimal deviation from the frozen model.
  • If the empirical gains hold, MAP-type normalization could reduce the need to hand-tune rank or learning rate when moving a PEFT method to a new backbone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A cheap test of the mechanism is to apply MAP to non-LoRA update families, such as adapters or prefix modules, since the normalization only needs a base matrix and an update matrix.
  • The normalization makes the update scale-free, so part of the benefit may be an adaptive learning-rate effect; comparing LoMAP with LoRA under per-layer learning-rate tuning would isolate that.
  • Because the paper's baseline numbers are imported from earlier work, a multi-seed rerun of DoRA and LoRA under identical conditions on Commonsense170K would be the quickest way to verify the reported margins.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes MAP, a parameter-efficient fine-tuning framework that reinterprets weight matrices as high-dimensional vectors and decouples adaptation into direction and magnitude. It normalizes the pretrained weight W and the low-rank update AB by their Frobenius norms, then recombines them with two learnable scalars α and β per layer (Eq. 4). The method is presented as a drop-in enhancement for LoRA (LoMAP) and is also coupled with AdaLoRA and FLoRA. Experiments cover commonsense reasoning (LLaMA-7B/LLaMA3-8B), GLUE (DeBERTaV3), and subject-driven generation with SDXL, reporting improvements over several baselines.

Significance. If the empirical claims hold, MAP would be a simple, parameter-efficient plug-in for low-rank adaptation, adding only two scalars per layer and avoiding DoRA's per-column overhead. The modular evaluations with AdaLoRA and FLoRA in Table 4 are a genuine strength. However, the significance is currently not established because the method as written is not reproducible at initialization, the headline consistency claim is contradicted by the paper's own table, and the baseline comparisons rely on numbers from other papers without error bars or matched reruns. With those issues addressed, the contribution could be a useful incremental improvement, though not a foundational one.

major comments (4)
  1. [§3.2, Eq. (4); §2.2] The forward pass of MAP is undefined at initialization. Section 2.2 states that B is initialized to zeros, so AB = 0 and ||AB||_F = 0 at the first step; Eq. (4) then contains the term β·AB/||AB||_F = 0/0. The paper never specifies an epsilon, a special-case treatment, or a nonzero B initialization. Since every reported run starts from this configuration, the results in Tables 1–4 depend on an unstated implementation choice. The authors must state the exact formula used at initialization (and ideally provide code) to make the method reproducible.
  2. [§4.1.2; Table 1] The claim that “LoMAP consistently outperforms all competing PEFT baselines under similar parameter budgets” is contradicted by the paper's own Table 1. At r=16 on LLaMA-7B, DoRA achieves an average of 78.4 while LoMAP achieves 77.9. The text should be amended to acknowledge this exception, and the abstract's phrasing about “significantly improves performance” should be matched to the actual pattern of results rather than implying universal dominance.
  3. [Table 1 caption; §4.1.2] The baseline numbers in Table 1 are taken from other publications (Si et al., 2025a; Wu et al., 2024) and were not rerun under identical conditions. No error bars, standard deviations, or multiple-seed results are reported for LoMAP either. As a result, gains such as 85.8 vs. 85.2 on LLaMA3-8B (r=32) may be within seed variance. The authors should either rerun the most important baselines in their own pipeline, report variance across seeds, or substantially soften the consistency claims.
  4. [§4, paragraph after Table 6; Table 6] There is an inconsistency in the definition of α. The text states “The parameters are initialized with α = ∥W∥F and β = 1 for all tasks,” but Table 6 lists α values of 32 or 64 across all layers and ranks. For a linear layer in LLaMA-7B, ||W||_F is far larger than 32 or 64, so these cannot both be true. The paper should clarify whether α is initialized per-layer to the Frobenius norm or is a shared hyperparameter; this affects the actual update rule and the parameter count claim.
minor comments (5)
  1. [§2.3] The word “intergrade” should be “integrate” in the sentence “our approach can intergrade with any LoRA variants.”
  2. [§2.4 heading] The heading “Weigh Decomposed Low-rank Adaptation” appears to contain a typo; it should be “Weight Decomposed Low-rank Adaptation.”
  3. [Table 2 caption] The caption uses “Base’” where it should be “Base.”
  4. [§5.1, Table 3] The statement that joint and stepwise optimization show “no significant performance difference” is not supported by statistical testing; the averages are identical (85.8) but the per-task numbers differ. At minimum, report multiple seeds or state this is a single-run comparison.
  5. [§4.3.2, Figure 2] The subject-driven generation evaluation is qualitative only. Adding quantitative measures such as CLIP score or DINO-based subject fidelity would strengthen the claim that LoMAP improves subject fidelity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: MAP's decomposition is defined independently from its outputs, and the self-cited baseline provenance is a comparability concern, not a circularity.

full rationale

The paper's central construction is a reparameterization, not a derivation that reduces to its own inputs. Equation (3) and Equation (4) define the adapted weight as a learnable combination of the normalized pretrained weight and the normalized low-rank update; neither side of the equation is defined in terms of the empirical result being claimed. There is no fitted parameter that is later relabeled as a prediction, and no uniqueness or equivalence result is imported from the authors' own prior work to force the choice of MAP. The only self-referential element is the empirical baseline provenance: Table 1 states that baseline results are taken from (Si et al., 2025a; Wu et al., 2024), and Si et al. (2025a) is an authors' prior paper. This is a legitimate concern about comparability of experimental conditions, but it is not circular reasoning: the LoMAP numbers are newly reported, and the claimed improvement does not equal the baseline values by construction. The flagged issue that Equation (4) divides by ||AB||_F while LoRA initializes B to zero is a serious correctness and reproducibility gap, but it is a defect in the stated method, not a circularity between premises and conclusions. Under the requirement to exhibit a specific reduction, no circular step can be identified, so the score is 0.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on two learnable scalars per layer and on comparability of external baselines; no invented physical or mathematical entities are needed.

free parameters (1)
  • α and β scalars per layer = learned during fine-tuning; initialized to ||W||_F and 1
    The two learnable coefficients are the core of the MAP mechanism; their optimal values are not derived but optimized from data.
assumptions (4)
  • standard math Frobenius norm of W equals the ℓ2 norm of the flattened vector w (Eq. 5)
    Used to justify the equivalence between Eq. (3) and Eq. (4).
  • domain assumption A weight matrix's 'direction' is meaningfully represented by its globally normalized flattened vector
    This is the conceptual premise that motivates MAP over per-column normalization; it is asserted, not proven.
  • ad hoc to paper The low-rank update AB can be normalized by its Frobenius norm and combined with the normalized base via two scalars
    This design choice is introduced to make the adaptation interpretable as a directional combination; it is not derived from a theorem.
  • domain assumption Baseline results from (Si et al., 2025a; Wu et al., 2024) are directly comparable to the authors' runs
    The paper relies on this to claim improvements without rerunning baselines.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MAP: Revisiting Weight Decomposition for Low-Rank Adaptation." pith.science (2026). https://pith.science/paper/5DOAYA7A

@misc{pith2026250523094,
  author       = {Pith},
  title        = {Pith review of: MAP: Revisiting Weight Decomposition for Low-Rank Adaptation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5DOAYA7A}},
  note         = {Machine review of arXiv:2505.23094}
}
read the original abstract

The rapid development of large language models has revolutionized natural language processing, but their fine-tuning remains computationally expensive, hindering broad deployment. Parameter-efficient fine-tuning (PEFT) methods, such as LoRA, have emerged as solutions. Recent work like DoRA attempts to further decompose weight adaptation into direction and magnitude components. However, existing formulations often define direction heuristically at the column level, lacking a principled geometric foundation. In this paper, we propose MAP, a novel framework that reformulates weight matrices as high-dimensional vectors and decouples their adaptation into direction and magnitude in a rigorous manner. MAP normalizes the pre-trained weights, learns a directional update, and introduces two scalar coefficients to independently scale the magnitude of the base and update vectors. This design enables more interpretable and flexible adaptation, and can be seamlessly integrated into existing PEFT methods. Extensive experiments show that MAP significantly improves performance when coupling with existing methods, offering a simple yet powerful enhancement to existing PEFT methods. Given the universality and simplicity of MAP, we hope it can serve as a default setting for designing future PEFT methods.

Figures

Figures reproduced from arXiv: 2505.23094 by the authors.

Figure 1
Figure 1. Comparison of LoRA, DoRA, and our proposed MAP framework. DoRA normalizes the sum [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Comparison of generated images from LoRA and LoMAP on the subject-driven generation task. It [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

68 extracted references · 12 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Armen Aghajanyan, Luke Zettlemoyer, and Sonal Gupta. 2020. Intrinsic dimensionality explains the effectiveness of language model fine-tuning. arXiv preprint arXiv:2012.13255

    arXiv 2020

  4. [4]

    AI@Meta. 2024. https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md Llama 3 model card

    2024

  5. [5]

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862

    arXiv 2022

  6. [6]

    Roy Bar-Haim, Ido Dagan, Bill Dolan, Lisa Ferro, Danilo Giampiccolo, Bernardo Magnini, and Idan Szpektor. 2006. The second pascal recognising textual entailment challenge. In Proceedings of the second PASCAL challenges workshop on recognising textual entailment, volume 1. Citeseer

    2006

  7. [7]

    Luisa Bentivogli, Peter Clark, Ido Dagan, and Danilo Giampiccolo. 2009. The fifth pascal recognizing textual entailment challenge. TAC, 7(8):1

    2009

  8. [8]

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. 2020. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432--7439

    2020

Show all 68 references
  1. [9]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  2. [10]

    Daniel Cer, Mona Diab, Eneko Agirre, Inigo Lopez-Gazpio, and Lucia Specia. 2017. Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation. arXiv preprint arXiv:1708.00055

    2017 arXiv

  3. [11]

    Shoufa Chen, Chongjian Ge, Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang, and Ping Luo. 2022. Adaptformer: Adapting vision transformers for scalable visual recognition. Advances in Neural Information Processing Systems, 35:16664--16678

    2022

  4. [12]

    Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint arXiv:1905.10044

    2019 arXiv

  5. [13]

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457

    2018 arXiv

  6. [14]

    Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005. The pascal recognising textual entailment challenge. In Machine learning challenges workshop, pages 177--190. Springer

    2005

  7. [15]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

    2018 arXiv

  8. [16]

    Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, et al. 2023. Parameter-efficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence, 5(3):220--235

    2023

  9. [17]

    Bill Dolan and Chris Brockett. 2005. Automatically constructing a corpus of sentential paraphrases. In Third international workshop on paraphrasing (IWP2005)

    2005

  10. [18]

    Chengcheng Feng, Mu He, Qiuyu Tian, Haojie Yin, Xiaofang Zhao, Hongwei Tang, and Xingqiang Wei. 2024. Trilora: Integrating svd for advanced style personalization in text-to-image generation. arXiv preprint arXiv:2405.11236

    2024 arXiv

  11. [19]

    Marc Fischer, Alexander Bartler, and Bin Yang. 2024. Prompt tuning for parameter-efficient medical image segmentation. Medical Image Analysis, 91:103024

    2024

  12. [20]

    Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and William B Dolan. 2007. The third pascal recognizing textual entailment challenge. In Proceedings of the ACL-PASCAL workshop on textual entailment and paraphrasing, pages 1--9

    2007

  13. [21]

    Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig. 2021 a . Towards a unified view of parameter-efficient transfer learning. arXiv preprint arXiv:2110.04366

    2021 arXiv

  14. [22]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pages 1026--1034

    2015

  15. [23]

    Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2021 b . Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing. arXiv preprint arXiv:2111.09543

    2021 arXiv

  16. [24]

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In International conference on machine learning, pages 2790--2799. PMLR

    2019

  17. [25]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685

    2021 arXiv

  18. [26]

    Zhiqiang Hu, Lei Wang, Yihuai Lan, Wanyu Xu, Ee-Peng Lim, Lidong Bing, Xing Xu, Soujanya Poria, and Roy Ka-Wei Lee. 2023. Llm-adapters: An adapter family for parameter-efficient fine-tuning of large language models. arXiv preprint arXiv:2304.01933

    2023 arXiv

  19. [27]

    Nam Hyeon-Woo, Moon Ye-Bin, and Tae-Hyun Oh. 2021. Fedpara: Low-rank hadamard product for communication-efficient federated learning. arXiv preprint arXiv:2108.06098

    2021 arXiv

  20. [28]

    Dawid Jan Kopiczko, Tijmen Blankevoort, and Yuki Markus Asano. 2023. Vera: Vector-based random matrix adaptation. arXiv preprint arXiv:2310.11454

    2023 arXiv

  21. [29]

    Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691

    2021 arXiv

  22. [30]

    Chunyuan Li, Heerad Farkhoor, Rosanne Liu, and Jason Yosinski. 2018. Measuring the intrinsic dimension of objective landscapes. arXiv preprint arXiv:1804.08838

    2018 arXiv

  23. [31]

    Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190

    2021 arXiv

  24. [32]

    Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. 2024. Dora: Weight-decomposed low-rank adaptation. arXiv preprint arXiv:2402.09353

    2024 arXiv

  25. [33]

    Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. 2023. Gpt understands, too. AI Open

    2023

  26. [34]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692

    2019 arXiv

  27. [35]

    Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. 2023. Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. arXiv preprint arXiv:2308.09583

    2023 arXiv

  28. [36]

    Fanxu Meng, Zhaohui Wang, and Muhan Zhang. 2024. Pissa: Principal singular values and singular vectors adaptation of large language models. arXiv preprint arXiv:2404.02948

    2024 arXiv

  29. [37]

    Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789

    2018 arXiv

  30. [38]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 3...

    2022

  31. [39]

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32

    2019

  32. [40]

    Jonas Pfeiffer, Aishwarya Kamath, Andreas R \"u ckl \'e , Kyunghyun Cho, and Iryna Gurevych. 2020. Adapterfusion: Non-destructive task composition for transfer learning. arXiv preprint arXiv:2005.00247

    2020 arXiv

  33. [41]

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M \"u ller, Joe Penna, and Robin Rombach. 2023. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952

    2023 arXiv

  34. [42]

    Peijia Qin, Ruiyi Zhang, and Pengtao Xie. 2024. Bidora: Bi-level optimization-based weight-decomposed low-rank adaptation. arXiv preprint arXiv:2410.09758

    2024 arXiv

  35. [43]

    Xipeng Qiu, Tianxiang Sun, Yige Xu, Yunfan Shao, Ning Dai, and Xuanjing Huang. 2020. Pre-trained models for natural language processing: A survey. Science China Technological Sciences, 63(10):1872--1897

    2020

  36. [44]

    Zeju Qiu, Weiyang Liu, Haiwen Feng, Yuxuan Xue, Yao Feng, Zhen Liu, Dan Zhang, Adrian Weller, and Bernhard Sch \"o lkopf. 2023. Controlling text-to-image diffusion by orthogonal finetuning. Advances in Neural Information Processing Systems, 36:79320--79362

    2023

  37. [45]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36

    2024

  38. [46]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1--67

    2020

  39. [47]

    Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250

    2016 arXiv

  40. [48]

    Anastasia Razdaibiedina, Yuning Mao, Rui Hou, Madian Khabsa, Mike Lewis, Jimmy Ba, and Amjad Almahairi. 2023. Residual prompt tuning: Improving prompt tuning with residual reparameterization. arXiv preprint arXiv:2305.03937

    2023 arXiv

  41. [49]

    Adithya Renduchintala, Tugrul Konuk, and Oleksii Kuchaiev. 2023. Tied-lora: Enhacing parameter efficiency of lora with weight tying. arXiv preprint arXiv:2311.09578

    2023 arXiv

  42. [50]

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. 2023. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages...

    2023

  43. [51]

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99--106

    2021

  44. [52]

    Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. 2019. Socialiqa: Commonsense reasoning about social interactions. arXiv preprint arXiv:1904.09728

    2019 arXiv

  45. [53]

    Zhengxiang Shi and Aldo Lipani. 2023. Dept: Decomposed prompt tuning for parameter-efficient fine-tuning. arXiv preprint arXiv:2309.05173

    2023 arXiv

  46. [54]

    Chongjie Si, Zhiyi Shi, Shifan Zhang, Xiaokang Yang, and Wei Shen. 2025 a . https://openreview.net/forum?id=RYrJqz44p4 Unleashing the power of task-specific directions in parameter efficient fine-tuning . In The Thirteenth International Conference on Learning Representations

    2025

  47. [55]

    Chongjie Si, Xuehui Wang, Xue Yang, Zhengqin Xu, Qingyun Li, Jifeng Dai, Yu Qiao, Xiaokang Yang, and Wei Shen. 2025 b . https://openreview.net/forum?id=OALIb8oNfl Maintaining structural integrity in parameter spaces for parameter efficient fine-tuning . In The Thirteenth Inter...

    2025

  48. [56]

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language...

    2013

  49. [57]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  50. [58]

    Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2018. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461

    2018 arXiv

  51. [59]

    Hanqing Wang, Zeguan Xiao, Yixia Li, Shuo Wang, Guanhua Chen, and Yun Chen. 2024 a . Milora: Harnessing minor singular components for parameter-efficient llm finetuning. arXiv preprint arXiv:2406.09044

    2024 arXiv

  52. [60]

    Qiushi Wang, Yuchen Fan, Junwei Bao, Hongfei Jiang, and Yang Song. 2024 b . Bora: Bi-dimensional weight-decomposed low-rank adaptation. arXiv preprint arXiv:2412.06441

    2024 arXiv

  53. [61]

    Alex Warstadt, Amanpreet Singh, and Samuel R Bowman. 2019. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7:625--641

    2019

  54. [62]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  55. [63]

    Adina Williams, Nikita Nangia, and Samuel R Bowman. 2017. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426

    2017 arXiv

  56. [64]

    Taiqiang Wu, Jiahao Wang, Zhe Zhao, and Ngai Wong. 2024. Mixture-of-subspaces in low-rank adaptation. arXiv preprint arXiv:2406.11909

    2024 arXiv

  57. [65]

    SHIH-YING YEH, Yu-Guan Hsieh, Zhidong Gao, Bernard BW Yang, Giyeong Oh, and Yanmin Gong. 2023. Navigating text-to-image customization: From lycoris fine-tuning to model evaluation. In The Twelfth International Conference on Learning Representations

    2023

  58. [66]

    Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2023. Metamath: Bootstrap your own mathematical questions for large language models. arXiv preprint arXiv:2309.12284

    2023 arXiv

  59. [67]

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830

    2019 arXiv

  60. [68]

    Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. 2022. Adaptive budget allocation for parameter-efficient fine-tuning. In The Eleventh International Conference on Learning Representations

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.