Pith. sign in

REVIEW 3 major objections 6 minor 29 references

CORA: Per-Slice Coherent Orthogonal Rotation for SVD-based Low-Rank Adaptation

T0 review · 3 major / 6 minor · reviewed 2026-07-12 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Stable low-rank finetuning of neural weights can be reduced to one shared orthogonal rotation per row slice of the rank-r SVD, plus a small diagonal spectrum scale, using about one-quarter the parameters of LoRA.

desk verdict Clean per-slice orthogonal PEFT that really does cut parameters ~4–8× and wins on commonsense/code; the optimality story is imported motivation, not what the tables test. read the letter →

arxiv 2607.02576 v1 pith:UI4ZFGR7 submitted 2026-06-30 stat.ML cs.LG

Pengcheng Wang , Ziran Liu , Wei Wang , Wei Jiang This is my paper
classification stat.MLcs.LG MSC 68T0515A1865F15
keywords parameter-efficientfine-tuninglow-rankadaptationsingularvaluedecompositioncoherentrotationorthogonalfinetuningPEFTLoRACayleytransform
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the most stable way to adapt a pretrained linear weight is not an arbitrary low-rank additive patch, but a coherent rotation of its singular bases: the same orthogonal matrix must act on both the left and right singular vectors, together with a diagonal shift of the singular values. It proves that this structure still holds when the weight is broken into row slices of height equal to the target rank, so each slice can be adapted by a single shared rotation and an optional shared scale. The resulting adapter, CORA, stores only the free parameters of those rotations (½ m (r−1) numbers per layer) and a tiny per-layer scale vector. Empirically this yields higher accuracy than LoRA-style and other SVD-based adapters on commonsense reasoning and code generation while using roughly four to eight times fewer trainable parameters. A sympathetic reader cares because the construction both explains why certain existing adapters work and supplies a concrete, cheaper parameterization that respects the geometry of the pretrained model.

What carries the argument

The per-slice coherent rotation form (Theorem 2) together with the algebraic reparameterization Q_i = U_i^⊤ R_i U_i. The identity converts an unconstrained learnable orthogonal R_i into a rotation that automatically satisfies Q_U,i = Q_V,i, so the coherent condition never needs to be enforced by a penalty or retraction.

What would settle it

Train identical models on the same data with CORA versus an otherwise identical adapter that deliberately breaks left-right rotation coherence (or that rotates only the minor singular subspace instead of the top-r principal subspace) and check whether the accuracy and parameter-efficiency advantages disappear on the reported commonsense and HumanEval suites.

Watch

Extended reading notes

Core claim

Under a Frobenius minimum-perturbation objective and mild regularity on each row slice, every slice of a pretrained weight admits the coherent-rotation form W★_i = U_i Q_i (Σ_i + ΔΣ_i) Q_i^⊤ V_i^⊤, where a single orthogonal Q_i acts on both singular bases. Realizing this form on the rank-r truncation of the weight, via a Cayley-parameterized rotation per slice and a shared diagonal scale, produces an adapter whose trainable count is independent of the column dimension and is about one-quarter that of LoRA at matched rank, while matching or exceeding several strong PEFT baselines.

Load-bearing premise

Every row slice of the chosen source matrix must keep a non-degenerate spectrum, a bounded condition number, and a small stable perturbation so that the local minimum-perturbation problem remains well-posed and solved by a unitary similarity.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes CORA, a PEFT method that adapts each row slice of the rank-r SVD truncation W_lr of a pretrained weight by a single orthogonal rotation Q_i acting coherently on both singular bases, plus an optional shared diagonal spectrum scale δ. Motivated by a global minimum-perturbation coherent-rotation form from concurrent work [8], the authors prove a per-slice analogue (Theorem 2) under slice-level regularity, reparameterize Q_i via Q_i = U_i^⊤ R_i U_i with R_i obtained from a closed-form Cayley map, and obtain ½m(r−1) trainable parameters per layer. Empirically, CORA at r=16 (6.6–10.3 M parameters) exceeds LoRA, DoRA, PiSSA, and MiLoRA on LLaMA-2/3 commonsense averages and HumanEval Pass@1 at roughly 4–8× fewer parameters, with weaker results on MATH.

Significance. If the claims hold, CORA is a useful and unusually compact SVD-aware adapter that unifies ideas from additive SVD-PEFT and orthogonal finetuning under an explicit coherent-rotation parameterization, with a clean algebraic identity (Lemma 1 / Proposition 1) and a transparent parameter count independent of the column dimension k. The empirical efficiency gains on commonsense and code are substantial and well tabulated. The algebraic construction and Cayley implementation are strengths: the coherent form holds identically once the reparameterization is chosen, and orthogonality is exact without penalties. The main scientific value is therefore a well-engineered, theory-motivated PEFT design with strong parameter–accuracy trade-offs on several standard LLM adaptation benchmarks; the optimality interpretation is secondary and less firmly established.

major comments (3)
  1. [§4–5, Theorem 2, Lemma 1, Tables 1–3, App. E–F] The paper’s central scientific framing is that CORA realizes the per-slice minimum-perturbation form of Theorem 2 (the slice-local analogue of [8, Prop. 9.4]), yet Section 5 evaluates only downstream accuracy at fixed budgets (Tables 1–3). Lemma 1 / Proposition 1 show that the coherent form holds identically for any orthogonal R_i; they do not show that the learned update is minimum-perturbation, nor that the accuracy gains come from that property rather than from being a well-regularized, low-parameter orthogonal adapter on W_lr. Appendices E–F report high per-slice coherence for Full FT, PiSSA, and MiLoRA as well, so coherence alone does not separate CORA. A load-bearing addition would be a controlled diagnostic or ablation that isolates the coherent two-sided structure (beyond the U-only ablation in Table 7) against matched-budget orthogonal or SVD baselines and links it to the claime
  2. [§4.1 Assumption 1, §5.3 Table 3, §5.4 Table 6] Assumption 1 (non-degenerate spectrum, bounded condition number, small stable perturbation for every row slice) is stated as holding “generically” via interlacing, but is never checked on the actual W_lr slices of LLaMA/Mistral layers. Table 3 and §5.3 already show that when task structure appears to lie outside the top-r principal subspace (MATH: CORA r=64 at 12.7 vs MiLoRA 17.8), the W_lr design and the claimed form lose their justification; the W_0 variant (Table 6 / Table 3) only partially closes the gap and at much higher cost. Either verify Assumption 1 on real slices (condition numbers, σ_min, perturbation scale) and qualify the regime where W_lr is appropriate, or treat W_lr as an engineering regularizer rather than a consequence of the min-perturbation theory.
  3. [§3.2 Theorem 1, §4.1 Theorem 2, Appendix A] Theorem 1 is imported from concurrent work [8] with overlapping co-authors and is only restated. The per-slice lift (Theorem 2) is the paper’s theoretical contribution, but the proof sketch reduces to Frobenius row-additivity plus a classical Schur–Horn / von Neumann unitary-similarity argument under Assumption 1. That is fine if self-contained, but the manuscript currently leans on [8] for the global optimality story while the experiments never test the global or slice-level Frobenius objective. Please make the per-slice argument fully self-contained in the main text (what is assumed, what is proved, what is not claimed about task loss), and separate more sharply the algebraic adapter contribution from the optimality claim of [8].
minor comments (6)
  1. [Figure 1, §4.3, §5.3] Figure 1 is described in the caption but the manuscript text does not fully specify how s = m/r interacts with non-divisible m or with grouped-query layouts (flagged informally for Mistral in §5.3). A short implementation note would help.
  2. [§4.3, Eq. (4)] Parameter accounting (Eqs. 4–5) assumes the default full-block Cayley setting b = r; the block-diagonal option is mentioned but not ablated. A one-line note on when b < r is used would avoid ambiguity.
  3. [Table 1, §5.1] Table 1 mixes published baseline numbers and reproductions; the caption notes sources, but a column or footnote marking which rows are reproduced under the authors’ trainer would improve reproducibility.
  4. [Table 4, Appendix G] The shared scale δ is regularized with L2 = 10^−3 (Table 10) but this is not discussed in the main ablations beyond Table 4. Briefly state sensitivity.
  5. [§4] Minor notation: W★_i vs fW_i vs ey are used for adapted quantities; a single consistent notation for the adapted weight would reduce friction.
  6. [§6] Limitations (§6) correctly note offline SVD cost and max 8B scale; consider also stating that HumanEval/GSM8K/MATH are single-seed or few-run results if that is the case.

Circularity Check

3 steps flagged · score 3.0 of 10

Optimality of the coherent-rotation target is imported from overlapping-author arXiv [8]; the adapter then realizes that target by algebraic construction (Lemma 1), while accuracy numbers remain independent external benchmarks.

  1. self citation load bearing [Abstract; §3.2 Theorem 1; References [8]]
    "Motivated by recent minimum-perturbation theory, which shows that stable finetuning follows a coherent SVD rotation in which a single orthogonal Q acts on both the left singular basis U0 and the right singular basis V0... Theorem 1 (Coherent rotation form, after [8, Prop. 9.4]). ... the finetuned weight has the form W⋆ = U0 Q (Σ0 + ΔΣ) Q⊤ V⊤0 ... Equivalently, the left and right in-basis rotations satisfy Q⋆U = Q⋆V = Q."

    The paper’s scientific framing—that the coherent form is the Frobenius minimum-perturbation optimum and that QU = QV is required—rests on arXiv:2605.02111 by overlapping co-authors (Ziran Liu, Wei Wang, Pengcheng Wang, Wei Jiang). That premise is not re-derived or externally verified here; it is imported as the target that CORA is then built to hit. Without [8], the claim that this particular form is forced by stability/min-perturbation has no independent support in the manuscript.

  2. uniqueness imported from authors [§3.2 after Theorem 1]
    "Among all finetuned weights that achieve a given task objective, the one with smallest ∥ΔW∥F keeps the pretrained singular coordinate system fixed and acts on Σ0 by a similarity transform Q(·)Q⊤ in that basis. Any mismatch between the left and right in-basis rotations, i.e., QU ≠ QV, breaks this coherent structure and increases the perturbation needed to realize the same adaptation."

    This uniqueness/forbidding claim (mismatch strictly worsens the objective) is presented as a consequence of the minimum-perturbation theory of [8]. It is used to justify why CORA must enforce QU,i = QV,i rather than free left/right rotations. The uniqueness is not proved from first principles in this paper; it is treated as an external fact while originating from the same author group.

1 more flagged steps
  1. self definitional [§4.2 Proposition 1; Appendix C Lemma 1]
    "Lemma 1 (Coherent rotation form holds identically). ... set Qi := Ui⊤ Ri Ui. Then: ... (ii) The CORA parameterization fWi := Ri Ui Σi Q⊤i V⊤i equals fWi = Ui Qi Σi Q⊤i V⊤i, i.e., the coherent rotation form (1) with ΔΣ = 0. ... The argument uses no optimization assumption and verifies the method in Section 4.3."

    Once Qi is defined as the Ui-similarity of an orthogonal Ri, equality with the coherent form is an algebraic identity by construction (Lemma 1 explicitly says it ‘holds identically’ and ‘uses no optimization assumption’). Claiming that CORA ‘realizes’ the min-perturbation form is therefore true by the choice of parameterization, not by an independent derivation that the learned adapter is optimal. This is mild: the paper is transparent that it is a reparameterization, but the identity does not itself confirm the optimality story imported from [8].

full rationale

The paper’s derivation chain has two distinct layers. Layer 1 (optimality): Theorem 1 restates the global coherent-rotation minimum-perturbation form from [8, Prop. 9.4], whose author list overlaps substantially with the present paper (Liu, Wang, Wang, Jiang). The uniqueness claim that any QU ≠ QV strictly increases Frobenius perturbation is therefore not an external mathematical fact but a self-cited premise. Theorem 2 then lifts that premise to row slices under Assumption 1; the lift itself is proved in Appendix A with classical Schur–Horn / von Neumann arguments and is not circular. Layer 2 (realization): Proposition 1 / Lemma 1 show that defining Qi := Ui⊤ Ri Ui makes the CORA parameterization equal the coherent form identically—an intentional algebraic identity, not a fitted prediction. Parameter counts and Tables 1–3 accuracy are independent of the self-citation. Thus the load-bearing circularity is modest and confined to the optimality motivation; the engineering construction and empirical claims have independent content. Score 3 reflects one load-bearing self-citation chain that does not collapse the central technical or empirical results.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The central PEFT claim rests on (i) the global coherent-rotation optimality result imported from [8], (ii) the slice-regularity Assumption 1 that lets the same argument apply row-wise, (iii) the modeling choice to adapt only the rank-r truncation W_lr, and (iv) a handful of architectural free parameters (rank, block size, optional δ). No new physical entities are postulated; the invented objects are the adapter construction itself.

free parameters (4)
  • adapter rank r
    Chosen by hand (r ∈ {8,16,32,64,128}); controls both capacity and the ½m(r−1) parameter count; not derived from data.
  • Cayley block size b
    Default b=r (full block); free architectural choice that changes the free-parameter count per slice.
  • shared spectrum scale δ ∈ R^r
    Optional learnable vector; ablated in Table 4; L2 regularized at 10^{-3}.
  • learning-rate / schedule / batch size
    Standard training hyperparameters listed in Appendix G; fitted to the usual PEFT protocol rather than derived.
assumptions (4)
  • domain assumption Global coherent-rotation minimum-perturbation form (Theorem 1 / [8, Prop. 9.4]): stable finetuning of a well-pretrained W0 is of the form U0 Q (Σ0+ΔΣ) Q^⊤ V0^⊤ with QU=QV=Q.
    Imported wholesale from the authors’ concurrent theory paper; supplies the geometric target that CORA is designed to realize.
  • domain assumption Assumption 1 (slice-level regularity): each row slice has non-degenerate spectrum, bounded condition number, and small stable perturbation, inherited via singular-value interlacing.
    Stated in Section 4.1; required for the local Schur–Horn argument to apply to every slice.
  • standard math Frobenius norm is the correct local objective and is additive over row partitions.
    Used in the proof of Theorem 2; standard but not the only possible PEFT objective.
  • ad hoc to paper Adapting only the rank-r truncation W_lr while freezing the residual is a sufficient spectral regularizer for the target tasks.
    Design choice of Section 4.1/4.3; the MATH results show it is not always sufficient.
invented entities (2)
  • CORA adapter (per-slice Cayley Ri + optional shared δ on W_lr)
    purpose: Concrete trainable realization of the per-slice coherent rotation form with ½m(r−1) parameters.
    The method itself; independent evidence is the empirical tables, not an external physical prediction.
  • Per-slice coherent rotation Qi = U_i^⊤ Ri Ui
    purpose: Enforces QU,i = QV,i by algebraic identity without manifold optimization.
    Reparameterization introduced in Section 4.2 / Proposition 1; holds by construction once Ri is orthogonal.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CORA: Per-Slice Coherent Orthogonal Rotation for SVD-based Low-Rank Adaptation." pith.science (2026). https://pith.science/paper/UI4ZFGR7

@misc{pith2026260702576,
  author       = {Pith},
  title        = {Pith review of: CORA: Per-Slice Coherent Orthogonal Rotation for SVD-based Low-Rank Adaptation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UI4ZFGR7}},
  note         = {Machine review of arXiv:2607.02576}
}
abstract

Parameter-Efficient Fine-Tuning (PEFT) commonly adapts pretrained weights through low-rank updates, and recent methods further exploit the singular value decomposition (SVD) of the base weight for initialization or subspace selection. However, these methods do not explicitly preserve the coupled geometry between the pretrained left and right singular bases. Motivated by recent minimum-perturbation theory, which shows that stable finetuning follows a coherent SVD rotation in which a single orthogonal $Q$ acts on both the left singular basis $U_0$ and the right singular basis $V_0$, we prove a per-slice analogue: each row slice of $W_0$ can be adapted by a shared orthogonal rotation $Q_i$ on its left basis $U_i$ and right basis $V_i$ together with a diagonal spectrum shift. We implement this form as CORA (Coherent Orthogonal Rotation Adaptation), which applies per-slice orthogonal rotations and a per-layer diagonal scale to the rank-$r$ SVD truncation of $W_0$. CORA uses $\tfrac{1}{2}m(r{-}1)$ trainable parameters per linear layer, about $4{\times}$ fewer than LoRA at the same rank. CORA outperforms LoRA, DoRA, PiSSA, and MiLoRA on commonsense reasoning and code generation while using about $8{\times}$ fewer parameters.

Figures

Figures reproduced from arXiv: 2607.02576 by the authors.

Figure 1
Figure 1. Overview of CORA. A weight matrix W is replaced by its rank-r SVD reconstruction Wlr, then partitioned into s = m/r row slices. Per-slice SVD factors are precomputed; only the per-slice orthogonal Qi and a per-layer diagonal scale δ ∈ R r are learned, with QU,i = QV,i = Qi (Thm. 2). Per-layer trainable parameters are 1 2m(r−1), roughly 4× fewer than LoRA at the same r. Recent theoretical work [8] shows that under st… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

29 extracted references · 5 linked inside Pith

  1. [8]

    Geometric and spectral alignment for deep neural network II.arXiv preprint arXiv:2605.02111, 2026

    Ziran Liu, Wei Wang, Jinhao Wang, Pengcheng Wang, Xinyi Sui, Cihan Ruan, Nam Ling, and Wei Jiang. Geometric and spectral alignment for deep neural network II.arXiv preprint arXiv:2605.02111, 2026. URLhttps://arxiv.org/abs/2605.02111

    arXiv 2026

  2. [1]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. InICLR, 2022

    2022

  3. [2]

    PiSSA: Principal singular values and singular vectors adaptation of large language models

    Fanxu Meng, Zhaohui Wang, and Muhan Zhang. PiSSA: Principal singular values and singular vectors adaptation of large language models. InNeurIPS, 2024

    2024

  4. [3]

    MiLoRA: Harnessing minor singular components for parameter-efficient LLM finetuning

    Hanqing Wang, Yixia Li, Shuo Wang, Guanhua Chen, and Yun Chen. MiLoRA: Harnessing minor singular components for parameter-efficient LLM finetuning. InNAACL, 2025

    2025

  5. [4]

    SVFT: Parameter-efficient fine-tuning with singular vectors

    Vijay Lingam, Atula Tejaswi, Aditya Vavre, Aneesh Shetty, Gautham Krishna Gudur, Joydeep Ghosh, Alex Dimakis, Eunsol Choi, Aleksandar Bojchevski, and Sujay Sanghavi. SVFT: Parameter-efficient fine-tuning with singular vectors. InNeurIPS, 2024

    2024

  6. [5]

    KaSA: Knowledge-aware singular-value adaptation of large language models

    Fan Wang, Juyong Jiang, Chansung Park, Sunghun Kim, and Jing Tang. KaSA: Knowledge-aware singular-value adaptation of large language models. InICLR, 2025

    2025

  7. [6]

    Controlling text-to-image diffusion by orthogonal finetuning

    Zeju Qiu, Weiyang Liu, Haiwen Feng, Yuxuan Xue, Yao Feng, Zhen Liu, Dan Zhang, Adrian Weller, and Bernhard Schölkopf. Controlling text-to-image diffusion by orthogonal finetuning. InNeurIPS, 2023

    2023

  8. [7]

    Black, Adrian Weller, and Bernhard Schölkopf

    Weiyang Liu, Zeju Qiu, Yao Feng, Yuliang Xiu, Yuxuan Xue, Longhui Yu, Haiwen Feng, Zhen Liu, Juyeon Heo, Songyou Peng, Yandong Wen, Michael J. Black, Adrian Weller, and Bernhard Schölkopf. Parameter-efficient orthogonal finetuning via butterfly factorization. InICLR, 2024

    2024

Show all 29 references
  1. [9]

    DoRA: Weight-decomposed low-rank adaptation

    Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. DoRA: Weight-decomposed low-rank adaptation. InICML, 2024

    2024

  2. [10]

    AdaLoRA: Adaptive budget allocation for parameter-efficient fine-tuning

    Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. AdaLoRA: Adaptive budget allocation for parameter-efficient fine-tuning. InICLR, 2023

    2023

  3. [11]

    Kopiczko, Tijmen Blankevoort, and Yuki M

    Dawid J. Kopiczko, Tijmen Blankevoort, and Yuki M. Asano. VeRA: Vector-based random matrix adaptation. InICLR, 2024

    2024

  4. [12]

    CorDA: Context-oriented decomposition adaptation of large language models for task-aware parameter-efficient fine-tuning

    Yibo Yang, Xiaojie Li, Zhongzhu Zhou, Shuaiwen Leon Song, Jianlong Wu, Liqiang Nie, and Bernard Ghanem. CorDA: Context-oriented decomposition adaptation of large language models for task-aware parameter-efficient fine-tuning. InNeurIPS, 2024

    2024

  5. [13]

    LoftQ: LoRA-fine-tuning-aware quantization for large language models

    Yixiao Li, Yifan Yu, Chen Liang, Pengcheng He, Nikos Karampatziakis, Weizhu Chen, and Tuo Zhao. LoftQ: LoRA-fine-tuning-aware quantization for large language models. InICLR, 2024

    2024

  6. [14]

    LoRA-GA: Low-rank adaptation with gradient approximation

    Shaowen Wang, Linxi Yu, and Jian Li. LoRA-GA: Low-rank adaptation with gradient approximation. In NeurIPS, 2024

    2024

  7. [15]

    LoRA-Pro: Are low-rank adapters properly optimized? InICLR, 2025

    Zhengbo Wang, Jian Liang, Ran He, Zilei Wang, and Tieniu Tan. LoRA-Pro: Are low-rank adapters properly optimized? InICLR, 2025

    2025

  8. [16]

    Orthogonal finetuning made scalable

    Zeju Qiu, Weiyang Liu, Adrian Weller, and Bernhard Schölkopf. Orthogonal finetuning made scalable. In EMNLP, 2025

    2025

  9. [17]

    Xiao, Maximilian Dax, Bernhard Schölkopf, and Weiyang Liu

    Zeju Qiu, Simon Buchholz, Tim Z. Xiao, Maximilian Dax, Bernhard Schölkopf, and Weiyang Liu. Reparameterized LLM training via orthogonal equivalence transformation. InNeurIPS, 2025

    2025

  10. [18]

    Efficient orthogonal fine-tuning with principal subspace adaptation

    Fei Wu, Jia Hu, Geyong Min, and Shiqiang Wang. Efficient orthogonal fine-tuning with principal subspace adaptation. InICLR, 2026

    2026

  11. [19]

    DiaBlo: Diagonal blocks are sufficient for finetuning.arXiv preprint arXiv:2506.03230, 2025

    Selcuk Gurses, Aozhong Zhang, Yanxia Deng, Xun Dong, Xin Li, Naigang Wang, Penghang Yin, and Zi Yang. DiaBlo: Diagonal blocks are sufficient for finetuning.arXiv preprint arXiv:2506.03230, 2025

    2025

  12. [20]

    G. W. Stewart and Ji-Guang Sun.Matrix Perturbation Theory. Academic Press, 1990

    1990

  13. [21]

    Horn and Charles R

    Roger A. Horn and Charles R. Johnson.Matrix Analysis. Cambridge University Press, 2nd edition, 2012

    2012

  14. [22]

    Tim Salimans and Diederik P. Kingma. Weight normalization: A simple reparameterization to accelerate training of deep neural networks. InNeurIPS, 2016. 10

    2016

  15. [23]

    LLM-Adapters: An adapter family for parameter-efficient fine-tuning of large language models

    Zhiqiang Hu, Lei Wang, Yihuai Lan, Wanyu Xu, Ee-Peng Lim, Lidong Bing, Xing Xu, Soujanya Poria, and Roy Ka-Wei Lee. LLM-Adapters: An adapter family for parameter-efficient fine-tuning of large language models. InEMNLP, 2023

    2023

  16. [24]

    Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu

    Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T. Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. MetaMath: Bootstrap your own mathematical questions for large language models.arXiv preprint arXiv:2309.12284, 2023

    2023 arXiv

  17. [25]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    2021 arXiv

  18. [26]

    Measuring mathematical problem solving with the MATH dataset.arXiv preprint arXiv:2103.03874, 2021

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset.arXiv preprint arXiv:2103.03874, 2021

    2021 arXiv

  19. [27]

    Code Alpaca: An instruction-following LLaMA model for code generation

    Sahil Chaudhary. Code Alpaca: An instruction-following LLaMA model for code generation. https: //github.com/sahil280114/codealpaca, 2023

    2023

  20. [28]

    Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021

    2021 arXiv

  21. [29]

    coherent rotation form

    Juzheng Zhang, Jiacheng You, Ashwinee Panda, and Tom Goldstein. LoRI: Reducing cross-task interfer- ence in multi-task low-rank adaptation. InCOLM, 2025. A Proof of Theorem 2 Proof.The Frobenius norm of a row-partitioned perturbation decomposes as ∥∆W∥ 2 F = Xs i=1 ∥∆Wi∥2 F . ...

    2025

Pith tools

Reviewed July 12, 2026 · model on record in the stance chip above.