Pith. sign in

REVIEW 3 major objections 5 minor 68 references

FraQ: Efficient Coordinate-Space Recompression for Federated Low-Rank Adaptation

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Federated LoRA aggregation can be done exactly and cheaply in coordinate space, avoiding full-matrix SVD while preserving optimal rank-p truncation.

desk verdict Clean algebra, plausible but under-supported empirical claim; worth refereeing with a demand for seed variance and code. read the letter →

arxiv 2608.03605 v1 pith:QGIYDAXT submitted 2026-08-04 cs.AI

Shenghui Li , Thiemo Voigt This is my paper
classification cs.AI
keywords federatedlearningLoRAlow-rankadaptationmatrixrecompressionsingularspectrumGramparameter-efficientfine-tuningcommunicationcompression
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Federated fine-tuning with Low-Rank Adaptation (LoRA) has a mismatch: averaging client adapter factors does not equal the average of the updates those adapters induce. The paper proposes FRAQ, which forms the exact weighted aggregate by stacking client factors and then recompresses it into a low-rank global adapter without ever constructing the full dense weight matrix. Its core result is an isometric coordinate trick: after orthogonalizing one stack, the aggregate's singular values and best low-rank approximations can be recovered from a small coordinate matrix and a Gram eigendecomposition. On text classification and commonsense reasoning benchmarks, FRAQ nearly matches uncompressed aggregation accuracy while cutting downlink payload to 16–68% of the stacked baseline and keeping server-side recompression cheap. The paper argues that this removes the main obstacle to exact, compact federated LoRA aggregation.

What carries the argument

The load-bearing object is the isometric coordinate equivalence of Proposition 1. For any factorization M=XY with X∈R^{d×r}, an orthonormal Q spanning col(X) converts the full matrix to coordinates H=Q^T M ∈ R^{r×D}; the map H ↦ QH preserves Frobenius distance and singular values, so a rank-p truncation of H, mapped back by Q, is a rank-p optimal approximation of M. FRAQ instantiates this on the exact stacked aggregate: it orients along the smaller weight dimension, computes a reduced Householder QR of the left stack to get an implicit Q and the triangular R, forms H=RY and the small Gram matrix G=HH^T, takes its top-p eigenspace by spectral-energy threshold, and reconstructs broadcast LoRA

What would settle it

Construct a federated fine-tuning scenario in which one client's update is correct but orthogonal to the dominant update directions (e.g., a small client that contributes a task-essential rare label with a distinctive LoRA direction), run FRAQ at τ=0.80 and 0.95, and compare task accuracy against uncompressed stacked aggregation. If accuracy degrades substantially while the retained spectral energy stays above the threshold, the energy-truncation premise is false. A cheaper probe: subtract the rank-p FRAQ approximation from the exact aggregate and test whether the residual directions carry abo

Watch

Extended reading notes

Core claim

The paper's central claim is that the exact weighted aggregate of client LoRA updates, formed by stacking factors so BA = Σ_k α_k s_k B_k A_k, can be recompressed into a rank-p global adapter in compact coordinate space without materializing the full d×D matrix. The enabling result is Proposition 1: if Q is an orthonormal basis for the column space of the left stack and H=Q^T M is the coordinate matrix, then M=QH, H and M have the same nonzero singular values, and the Frobenius distance from M to QH̃ equals the distance from H to H̃. Hence a best rank-p approximation of H maps back through Q to a best rank-p approximation of M. FRAQ realizes this with one reduced Householder QR, a Gram eigen

Load-bearing premise

The accuracy result stands on the assumption that retaining a fixed fraction (80% or 95%) of the aggregate's spectral energy preserves everything a downstream task needs, so the singular directions that are dropped are never task-critical; if useful signal can live in low-energy directions, FRAQ's compression will hurt accuracy even though the algebra is exact.

Editorial extensions

If this is right

  • At a chosen rank p, the broadcast adapter remains an optimal Frobenius-norm rank-p approximation of the exact aggregate, so recompression introduces no extra error beyond the rank truncation itself.
  • The energy threshold gives an explicit, server-side knob trading downlink payload against task accuracy; the paper reports 16–68% of stacked-baseline downlink with 0.33–0.61 point accuracy loss at τ=0.95.
  • Heterogeneous local LoRA ranks are supported with no extra machinery, since stacking concatenates factors of different widths.
  • Server-side cost becomes O((d+D)r^2 + r^3) rather than O(dD r + D d^2), removing the dense matrix from memory and making aggregation orders of magnitude faster than dense-SVD baselines in measured latency.
  • The same coordinate-space construction works with any orthogonalization backend satisfying the proposition (QR, polar, or converged iterative methods), so the method is not tied to the particular QR routine.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The core proposition is a general matrix fact: any low-rank product XY whose left factor has r columns can be optimally recompressed from an r-dimensional coordinate space without materialization. This extends FRAQ beyond federated LoRA to model merging, distributed factorizations, or any server that must compress a sum of low-rank updates.
  • The empirical link between spectral energy and task accuracy is the fragile part. A federated task where a minority client's distinctive signal lives in low-energy singular directions would fail at high energy retention despite exact algebra; that scenario is testable by zeroing low-energy subspaces and measuring accuracy on per-client subgroups.
  • Because the paper reports a single seed and no code, the 0.3–0.6 point accuracy gaps may not be stable under reinitialization; an ablation across seeds and client partitions would tell whether the claimed trade-off holds at the distribution level.
  • The paper notes its QR-and-Gram construction could replace SVD-based recompression in non-merge-and-reinitialize methods; if validated, that would generalize the speedup to persistent-adapter federated LoRA.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes FRAQ, a server-side recompression method for federated LoRA. Clients upload low-rank adapters; the server forms a weighted stack whose product equals the exact aggregated update, then recompresses it without materializing the full dense matrix. The key algebraic result (Proposition 1, Section 3.2) shows that if M = XY with an orthonormal Q spanning col(X), then H = QᵀM satisfies M = QH and preserves nonzero singular values and Frobenius-distance geometry, so an optimal rank-p approximation of H maps back to an optimal rank-p approximation of M. FRAQ instantiates this with a reduced Householder QR of the smaller-side factor and a compact Gram eigendecomposition, selecting the rank by an energy-retention threshold τ (Eq. 14). Experiments on RoBERTa-base text classification and LLaMA-3.2-3B commonsense reasoning report accuracy within about 0.6 points of the uncompressed FLoRA baseline while using 16–68% of its downlink, with low server-side latency. The paper also includes a complexity analysis, numerical-fidelity experiments, an ablation, and an appendix discussion of alternative orthogonalization backends.

Significance. The core contribution is an elegant and correct computational reduction: exact Frobenius-optimal rank-p recompression of the aggregated LoRA update can be performed through a one-sided QR and an r×r Gram eigendecomposition, avoiding the full-size SVD. Proposition 1 is proved cleanly, and the pipeline algebra in Eqs. (15)–(17) is consistent. The complexity analysis (Appendix A.3) is careful and gives a concrete asymptotic advantage over dense-materialization and two-sided-SVD approaches. If the empirical accuracy claim holds, FRAQ is a practical and useful method for communication-efficient federated fine-tuning, with a tunable accuracy–communication trade-off. The main weakness is that the central empirical claim is supported by a single seed, no code release, and an unvalidated assumption that spectral-energy truncation preserves downstream task accuracy. These are fixable within the scope of a revision, and they do not undermine the soundness of the algebraic core.

major comments (3)
  1. [Section 4.1 / Table 5] All experiments use a single random seed ('Random seed 42' in Table 5), and Tables 1 and 2 report point accuracies without standard deviations or confidence intervals. The headline differences from FLoRA are often 0.3–0.6 percentage points, which is within typical seed noise for these benchmarks. The central claim of 'accuracy close to uncompressed aggregation' therefore lacks statistical support. Please report results over at least 3–5 seeds with mean and variance, or otherwise demonstrate that seed sensitivity is negligible for the comparisons that support the main claim.
  2. [Section 3.3, Eq. (14) and Section 4.2] The energy-retention rule selects the smallest rank p such that the retained squared-singular-value energy is at least τ of the total. This discards a subspace whose Frobenius norm is sqrt(1−τ)·||ΔW||_F, which at τ=0.80 is about 44.7% of the aggregate norm. Proposition 1 guarantees only that the retained product is the best rank-p Frobenius approximation of the exact aggregate; it does not imply that the discarded directions are unimportant for downstream task accuracy. The paper's main empirical claim is therefore conditional on an unvalidated assumption that task-relevant signal aligns with high-energy singular directions. Please add a sensitivity analysis or targeted experiments (e.g., tasks or client distributions where low-energy directions are plausibly relevant) to justify this assumption, or explicitly qualify the accuracy claim as limited to settings where the assumption holds.
  3. [Section 4 (overall empirical reproducibility)] No source code or model/data artifacts are provided, and the evaluation protocol (particularly the reasoning benchmark and the heterogeneous-rank setup in Table 2) is described only at a high level. Combined with the single-seed reporting, this prevents an independent check of the numerical fidelity and latency results. The paper would be substantially strengthened by releasing the code and specifying the exact evaluation pipeline, including how the per-client LoRA alphas in the heterogeneous-rank case are aligned with the stacking formula in Eq. (6).
minor comments (5)
  1. [Abstract / title] The name is written 'FraQ' in the title and abstract but 'FRAQ' throughout the rest of the paper. Please unify the spelling.
  2. [Section 3.3, Eq. (14)] The condition for zero aggregate (λ_i = 0 for all i) is not handled; if ΔW = 0, the fraction in Eq. (14) is undefined. A small fallback (e.g., p = 1 or broadcast of the zero adapter) would make the algorithm complete.
  3. [Table 3 and Figure 4] Table 3 reports a relative error of 2.56×10⁻⁴ against a coordinate-SVD rank-8 reference, while Figure 4 reports a full-rank reconstruction error around 10⁻⁷ against FP64 references. These metrics are different, but the discrepancy is not explained in the text. Please clarify what each metric measures and why they differ.
  4. [Algorithm 1] The line 'A ← [√α₁ s₁ A₁; ...; √α_K s_K A_K]' uses a semicolon for vertical stacking, which is inconsistent with the transpose convention in Eq. (6). Rewriting it explicitly as in Eq. (6) would avoid confusion.
  5. [References] Several references are dated 2026 (e.g., Yan et al., Ramesh and Dass 2026a,b; Meng et al.). The authors should verify that these are publicly available and correctly cited, and that no citation is a self-citation intended to supply a central claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: FRAQ's optimality guarantee is proven from first principles, and the energy threshold is a user-set hyperparameter evaluated empirically, not a fitted input or a self-citation.

full rationale

FRAQ's derivation chain is self-contained. Proposition 1 (Section 3.2) is a mathematical statement: for M=XY and any orthonormal Q whose span contains col(X), the coordinate matrix H=Q^T M satisfies M=QH and preserves the nonzero singular spectrum and all Frobenius distances (Eqs. 9-10). The proof in Appendix A.1 uses only the relation X=QQ^T X, isometry of Q, and the Eckart-Young-Mirsky theorem; it does not assume the conclusion. The algorithm instantiates Proposition 1 on the exact weighted stack B,A from Eq. (6), whose product equals the exact aggregate by definition, so the optimality of BgAg as a rank-p Frobenius approximation of the exact aggregate follows algebraically. The energy-threshold rank selection in Eq. (14) is a user-prescribed hyperparameter (tau=0.80, 0.95), not a parameter fitted to the measured accuracy, and the reported accuracy numbers are empirical evaluations on external benchmarks, not consequences of the threshold definition. There are no load-bearing self-citations: the paper cites prior work on factor stacking and recompression, but its own proof and experiments stand independently. The Limitations section discloses the cost scaling with total stacked rank and the merge-and-reinitialize scheme; neither indicates circularity. The empirical assumption that spectral-energy truncation preserves downstream task signal is an externally testable regularity, not an identity or a fitted input, so it does not make the central claim circular.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central derivation depends on standard linear algebra and one domain assumption about energy-based truncation. The only hand-set input is the energy threshold tau; the method introduces no new physical or conceptual entities.

free parameters (1)
  • Energy-retention threshold tau = 0.80 and 0.95 (hand-chosen, not fitted to test data)
    Eq. (14) selects the global rank p from this threshold. All headline accuracy and downlink numbers depend on this knob, and the paper gives no principled way to set it for a new task.
assumptions (4)
  • standard math Eckart-Young-Mirsky theorem: the best rank-p Frobenius approximation of a matrix is obtained by truncating its SVD.
    Invoked in Appendix A.1 to justify Eq. (11) and the optimality of QH_p.
  • standard math The stacked factors in Eq. (6) satisfy BA = sum_k alpha_k s_k B_k A_k exactly, with client scaling folded into A.
    This is an algebraic construction, exact by definition; it is the premise that the aggregate being recompressed is the true update-level weighted average.
  • ad hoc to paper Rank truncation by spectral-energy threshold preserves downstream task accuracy across federated rounds.
    Eq. (14) and Section 4 assume low-energy singular directions are safely discarded; this is empirically tested but not derived or bounded, and it is the main load-bearing empirical premise.
  • domain assumption Reduced Householder QR and FP32 arithmetic reconstruct FRAQ factors accurately for the condition numbers encountered.
    Figure 4 gives empirical support for condition numbers up to 1e10, but this is hardware- and precision-dependent and not a formal guarantee.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FraQ: Efficient Coordinate-Space Recompression for Federated Low-Rank Adaptation." pith.science (2026). https://pith.science/paper/QGIYDAXT

@misc{pith2026260803605,
  author       = {Pith},
  title        = {Pith review of: FraQ: Efficient Coordinate-Space Recompression for Federated Low-Rank Adaptation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QGIYDAXT}},
  note         = {Machine review of arXiv:2608.03605}
}
read the original abstract

Federated fine-tuning with Low-Rank Adaptation (LoRA) enables efficient collaborative adaptation of Large Language Models (LLMs) without centralizing private data. However, LoRA's two-factor parameterization creates an aggregation mismatch across clients: naively averaging the factors does not recover the average of their induced updates. This mismatch can be avoided by forming the exact aggregate in the full weight space and then recompressing it, but decomposing the resulting dense matrix is computationally expensive and memory-intensive. We propose FraQ, an efficient coordinate-space recompression method for federated LoRA. Starting from stacked factors that exactly represent the aggregate, FraQ factorizes it into an orthonormal basis and a compact coordinate matrix. It then recovers the singular spectrum from a small Gram matrix, selects the smallest rank satisfying a prescribed energy threshold, and maps the selected coordinate subspace back through the basis to construct the global adapter. Experiments on text classification and commonsense reasoning benchmarks show that FraQ achieves accuracy close to uncompressed baselines while substantially reducing downlink communication with low server-side recompression overhead.

Figures

Figures reproduced from arXiv: 2608.03605 by the authors.

Figure 1
Figure 1. Exact aggregation via unrecompressed stack [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of FRAQ, illustrated for the m > n orientation. Each client initializes and trains a LoRA adapter on the current global model and uploads its factors to the server. The server (1) forms weighted factor stacks whose product equals the exact update-level aggregate; (2) constructs an orthonormal basis and a coordinate representation without materializing the full-size aggregate; (3) recovers the spectrum from … view at source ↗
Figure 3
Figure 3. Server-side recompression runtime and peak memory as (a) the total stacked rank [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Numerical fidelity and stability in FP32 [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

68 extracted references · 36 canonical work pages

  1. [1]

    2017 , editor =

    McMahan, Brendan and Moore, Eider and Ramage, Daniel and Hampson, Seth and Arcas, Blaise Aguera y , booktitle =. 2017 , editor =

    2017

  2. [2]

    Universal Language Model Fine-tuning for Text Classification

    Howard, Jeremy and Ruder, Sebastian. Universal Language Model Fine-tuning for Text Classification. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018. doi:10.18653/v1/P18-1031

    doi:10.18653/v1/p18-1031 2018

  3. [3]

    Fine-tuning a llm using reinforcement learning from human feedback for a therapy chatbot application , author=

    work page

  4. [4]

    Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages =

    Dong, Xin Luna and Moon, Seungwhan and Xu, Yifan Ethan and Malik, Kshitiz and Yu, Zhou , title =. Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages =. 2023 , isbn =. doi:10.1145/3580305.3599572 , abstract =

    arXiv 2023

  5. [5]

    and Delellis, Nicole S

    Kelly, Dominique and Chen, Yimin and Cornwell, Sarah E. and Delellis, Nicole S. and Mayhew, Alex and Onaolapo, Sodiq and Rubin, Victoria L. , title =. Proceedings of the Association for Information Science and Technology , month = oct, pages =. 2023 , issue_date =. doi:10.1002/pra2.927 , abstract =

    work page doi:10.1002/pra2.927 2023

  6. [6]

    Nature medicine , volume=

    Large language models in medicine , author=. Nature medicine , volume=. 2023 , publisher=

    2023

  7. [7]

    arXiv preprint arXiv:2311.07361 , year=

    The impact of large language models on scientific discovery: a preliminary study using gpt-4 , author=. arXiv preprint arXiv:2311.07361 , year=

    arXiv

  8. [8]

    Edward J Hu and yelong shen and Phillip Wallis and Zeyuan Allen-Zhu and Yuanzhi Li and Shean Wang and Lu Wang and Weizhu Chen , booktitle=. Lo. 2022 , url=

    2022

Show all 68 references
  1. [9]

    Towards Building The Federatedgpt: Federated Instruction Tuning , year=

    Zhang, Jianyi and Vahidian, Saeed and Kuo, Martin and Li, Chunyuan and Zhang, Ruiyi and Yu, Tong and Wang, Guoyin and Chen, Yiran , booktitle=. Towards Building The Federatedgpt: Federated Instruction Tuning , year=

  2. [10]

    FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations , volume =

    Wang, Ziyao and Shen, Zheyu and He, Yexiao and Sun, Guoheng and Wang, Hongyi and Lyu, Lingjuan and Li, Ang , booktitle =. FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations , volume =

  3. [11]

    Advances in neural information processing systems , volume=

    Learning both weights and connections for efficient neural network , author=. Advances in neural information processing systems , volume=

  4. [12]

    Journal of Machine Learning Research , year =

    Nitish Srivastava and Geoffrey Hinton and Alex Krizhevsky and Ilya Sutskever and Ruslan Salakhutdinov , title =. Journal of Machine Learning Research , year =

  5. [13]

    Heterogeneous Lo

    Yae Jee Cho and Luyang Liu and Zheng Xu and Aldi Fahrezi and Matt Barnes and Gauri Joshi , booktitle=. Heterogeneous Lo. 2023 , url=

    2023

  6. [14]

    arXiv preprint arXiv:2401.02385 , year=

    Tinyllama: An open-source small language model , author=. arXiv preprint arXiv:2401.02385 , year=

    arXiv

  7. [16]

    Advances in Neural Information Processing Systems , volume=

    Alpacafarm: A simulation framework for methods that learn from human feedback , author=. Advances in Neural Information Processing Systems , volume=

  8. [17]

    The Thirteenth International Conference on Learning Representations , year=

    WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct , author=. The Thirteenth International Conference on Learning Representations , year=

  9. [18]

    International Conference on Learning Representations , year=

    Measuring Massive Multitask Language Understanding , author=. International Conference on Learning Representations , year=

  10. [19]

    CoRR , volume =

    Chaoyang He and Songze Li and Jinhyun So and Mi Zhang and Hongyi Wang and Xiaoyang Wang and Praneeth Vepakomma and Abhishek Singh and Hang Qiu and Li Shen and Peilin Zhao and Yan Kang and Yang Liu and Ramesh Raskar and Qiang Yang and Murali Annavaram and Salman Avestimehr , ti...

    2020 arXiv

  11. [20]

    International conference on machine learning , pages=

    Fedscale: Benchmarking model and system performance of federated learning at scale , author=. International conference on machine learning , pages=. 2022 , organization=

    2022

  12. [21]

    2023 , url=

    Sara Babakniya and Ahmed Elkordy and Yahya Ezzeldin and Qingfeng Liu and Kee-Bong Song and MOSTAFA EL-Khamy and Salman Avestimehr , booktitle=. 2023 , url=

    2023

  13. [22]

    R o L o RA : Fine-tuning Rotated Outlier-free LLM s for Effective Weight-Activation Quantization

    Huang, Xijie and Liu, Zechun and Liu, Shih-Yang and Cheng, Kwang-Ting. R o L o RA : Fine-tuning Rotated Outlier-free LLM s for Effective Weight-Activation Quantization. Findings of the Association for Computational Linguistics: EMNLP 2024. 2024. doi:10.18653/v1/2024.findings-emnlp.444

    2024 doi

  14. [23]

    2024 , eprint=

    Communication-Efficient and Tensorized Federated Fine-Tuning of Large Language Models , author=. 2024 , eprint=

    2024

  15. [24]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=

    Decentralized federated averaging , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=. 2022 , publisher=

    2022

  16. [25]

    Lin, Zheng and Hu, Xuanjie and Zhang, Yuxin and Chen, Zhe and Fang, Zihan and Chen, Xianhao and Li, Ang and Vepakomma, Praneeth and Gao, Yue , journal=

  17. [26]

    The Twelfth International Conference on Learning Representations , year=

    Improving LoRA in Privacy-preserving Federated Learning , author=. The Twelfth International Conference on Learning Representations , year=

  18. [27]

    The Thirty-eighth Annual Conference on Neural Information Processing Systems , year=

    Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resources , author=. The Thirty-eighth Annual Conference on Neural Information Processing Systems , year=

  19. [28]

    Client-Customized Adaptation for Parameter-Efficient Federated Learning

    Kim, Yeachan and Kim, Junho and Mok, Wing-Lam and Park, Jun-Hyung and Lee, SangKeun. Client-Customized Adaptation for Parameter-Efficient Federated Learning. Findings of the Association for Computational Linguistics: ACL 2023. 2023. doi:10.18653/v1/2023.findings-acl.75

    2023 doi

  20. [29]

    and Xu, Daguang and Chen, Tingjun and Chen, Yiran , title =

    Li, Yiming and Sun, Jingwei and Liu, Yudong and Zhang, Yuandong and Li, Ang and Chen, Beidi and Roth, Holger R. and Xu, Daguang and Chen, Tingjun and Chen, Yiran , title =. Proceedings of the 30th Annual International Conference on Mobile Computing and Networking , pages =. 20...

    2024

  21. [30]

    Proceedings of the 41st International Conference on Machine Learning , articleno =

    Qin, Zhen and Chen, Daoyuan and Qian, Bingchen and Ding, Bolin and Li, Yaliang and Deng, Shuiguang , title =. Proceedings of the 41st International Conference on Machine Learning , articleno =. 2024 , publisher =

    2024

  22. [31]

    Proceedings of the 41st International Conference on Machine Learning , articleno =

    Hou, Charlie and Shrivastava, Akshat and Zhan, Hongyuan and Conway, Rylan and Le, Trang and Sagar, Adithya and Fanti, Giulia and Lazar, Daniel , title =. Proceedings of the 41st International Conference on Machine Learning , articleno =. 2024 , publisher =

    2024

  23. [32]

    Layer by Layer: Uncovering Where Multi-Task Learning Happens in Instruction-Tuned Large Language Models

    Zhao, Zheng and Ziser, Yftah and Cohen, Shay B. Layer by Layer: Uncovering Where Multi-Task Learning Happens in Instruction-Tuned Large Language Models. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024

    2024

  24. [33]

    2023 , eprint=

    AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning , author=. 2023 , eprint=

    2023

  25. [34]

    Fast randomized Singular Value Thresholding for Nuclear Norm Minimization , year=

    Oh, Tae-Hyun and Matsushita, Yasuyuki and Tai, Yu-Wing and Kweon, In So , booktitle=. Fast randomized Singular Value Thresholding for Nuclear Norm Minimization , year=

  26. [35]

    IEEE Transactions on Information Theory , year=

    OptShrink: An Algorithm for Improved Low-Rank Signal Matrix Denoising by Optimal, Data-Driven Singular Value Shrinkage , author=. IEEE Transactions on Information Theory , year=

  27. [36]

    A Singular Value Thresholding Algorithm for Matrix Completion , journal =

    Cai, Jian-Feng and Cand\`. A Singular Value Thresholding Algorithm for Matrix Completion , journal =. 2010 , doi =

    2010

  28. [37]

    Linear Algebra and its applications , volume=

    A generalization of the Eckart-Young-Mirsky matrix approximation theorem , author=. Linear Algebra and its applications , volume=. 1987 , publisher=

    1987

  29. [38]

    and Deems, Stephen and Furlani, Thomas R

    Boerner, Timothy J. and Deems, Stephen and Furlani, Thomas R. and Knuth, Shelley L. and Towns, John , title =. Practice and Experience in Advanced Research Computing 2023: Computing for the Common Good , pages =. 2023 , isbn =. doi:10.1145/3569951.3597559 , abstract =

    2023

  30. [39]

    arXiv preprint arXiv:1907.11692 , year=

    RoBERTa: A Robustly Optimized BERT Pretraining Approach , author=. arXiv preprint arXiv:1907.11692 , year=

    1907 arXiv

  31. [40]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Towards Robust and Efficient Federated Low-Rank Adaptation with Heterogeneous Clients , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=. 2025 , address=. doi:10.18653/v1/2025.acl-long.19 , url=

    2025 doi

  32. [41]

    Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI , pages=

    Efficient Intent Detection with Dual Sentence Encoders , author=. Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI , pages=. 2020 , publisher=

    2020

  33. [42]

    Proceedings of the Twelfth International Conference on Machine Learning , pages=

    NewsWeeder: Learning to Filter Netnews , author=. Proceedings of the Twelfth International Conference on Machine Learning , pages=

  34. [43]

    Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing , pages=

    An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction , author=. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing , pages=. 2019 , publisher=

    2019

  35. [44]

    Proceedings of the Tenth International Workshop on Spoken Dialogue Systems Technology , year=

    Benchmarking Natural Language Understanding Services for Building Conversational Agents , author=. Proceedings of the Tenth International Workshop on Spoken Dialogue Systems Technology , year=

  36. [45]

    F ed E x- L o RA : Exact Aggregation for Federated and Efficient Fine-Tuning of Large Language Models

    Singhal, Raghav and Ponkshe, Kaustubh and Vepakomma, Praneeth. F ed E x- L o RA : Exact Aggregation for Federated and Efficient Fine-Tuning of Large Language Models. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)...

    2025 doi

  37. [46]

    Hu, Zhiqiang and Wang, Lei and Lan, Yihuai and Xu, Wanyu and Lim, Ee-Peng and Bing, Lidong and Xu, Xing and Poria, Soujanya and Lee, Roy Ka-Wei , booktitle=

  38. [47]

    Clark, Christopher and Lee, Kenton and Chang, Ming-Wei and Kwiatkowski, Tom and Collins, Michael and Toutanova, Kristina , booktitle=

  39. [48]

    Bisk, Yonatan and Zellers, Rowan and Le Bras, Ronan and Gao, Jianfeng and Choi, Yejin , booktitle=

  40. [49]

    Sap, Maarten and Rashkin, Hannah and Chen, Derek and Le Bras, Ronan and Choi, Yejin , booktitle=

  41. [50]

    Zellers, Rowan and Holtzman, Ari and Bisk, Yonatan and Farhadi, Ali and Choi, Yejin , booktitle=

  42. [51]

    Sakaguchi, Keisuke and Le Bras, Ronan and Bhagavatula, Chandra and Choi, Yejin , journal=

  43. [52]

    Think You Have Solved Question Answering?

    Clark, Peter and Cowhey, Isaac and Etzioni, Oren and Khot, Tushar and Sabharwal, Ashish and Schoenick, Carissa and Tafjord, Oyvind , journal=. Think You Have Solved Question Answering?

  44. [53]

    Can a Suit of Armor Conduct Electricity?

    Mihaylov, Todor and Clark, Peter and Khot, Tushar and Sabharwal, Ashish , booktitle=. Can a Suit of Armor Conduct Electricity?

  45. [54]

    2019 , address =

    Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina , booktitle =. 2019 , address =. doi:10.18653/v1/N19-1423 , pages =

    2019 doi

  46. [55]

    Language Models are Few-Shot Learners , url =

    Brown, Tom and others , editor =. Language Models are Few-Shot Learners , url =. Advances in Neural Information Processing Systems , pages =

  47. [56]

    arXiv preprint arXiv:2302.13971 , url =

    Llama: Open and efficient foundation language models , author =. arXiv preprint arXiv:2302.13971 , url =

    arXiv

  48. [57]

    2022 , url =

    ChatGPT , author =. 2022 , url =

    2022

  49. [58]

    2023 , eprint=

    Qwen Technical Report , author=. 2023 , eprint=

    2023

  50. [59]

    2025 , eprint=

    Qwen2.5 Technical Report , author=. 2025 , eprint=

    2025

  51. [60]

    arXiv preprint arXiv:2407.10671 , year=

    Qwen2 technical report , author=. arXiv preprint arXiv:2407.10671 , year=

    arXiv

  52. [61]

    2025 , eprint=

    Qwen3 Technical Report , author=. 2025 , eprint=

    2025

  53. [62]

    2026 , eprint=

    OpenAI GPT-5 System Card , author=. 2026 , eprint=

    2026

  54. [63]

    2024 , eprint=

    The Llama 3 Herd of Models , author=. 2024 , eprint=

    2024

  55. [64]

    2026 , eprint=

    A Survey of Large Language Models , author=. 2026 , eprint=

    2026

  56. [65]

    2026 , url=

    Hariharan Ramesh and Jyotikrishna Dass , booktitle=. 2026 , url=

    2026

  57. [66]

    Wong , booktitle=

    Chuiyang Meng and Ming Tang and Vincent W.S. Wong , booktitle=. 2026 , url=

    2026

  58. [67]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    Accurate and Efficient Low-Rank Model Merging in Core Space , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  59. [68]

    2026 , eprint=

    FedMomentum: Preserving LoRA Training Momentum in Federated Fine-Tuning , author=. 2026 , eprint=

    2026

  60. [69]

    2026 , eprint=

    Spectral Transformation for Layer-wise Global Rank Discovery in Federated LoRA for Vision Transformers , author=. 2026 , eprint=

    2026

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.