Pith. sign in

REVIEW 4 major objections 5 minor 34 references

PHLoRA extracts LoRA-compatible adapters from an already fine-tuned full-rank model by taking a truncated SVD of the weight delta—no training data, gradients, or task-specific tuning—and its merged adapters match full-rank performance withi

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-04 17:17 UTC pith:5BXN3TPO

load-bearing objection PHLoRA is truncated SVD on the weight delta, packaged with a broad but single-run evaluation; the method mostly works, but the energy-to-performance claim has a concrete counterexample inside the paper. the 4 major comments →

arxiv 2509.10971 v1 pith:5BXN3TPO submitted 2025-09-13 cs.LG cs.AI

PHLoRA: data-free Post-hoc Low-Rank Adapter extraction from full-rank checkpoint

Bhoomit Vasani , Jack FitzGerald , Anjie Fang , Sushmit Vaish This is my paper
classification cs.LG cs.AI
keywords PHLoRApost-hoc LoRA extractionlow-rank adaptertruncated SVDweight deltaEckart-Young theoremdata-free model compressionmulti-adapter inference
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that any full-rank fine-tuned checkpoint can be retroactively converted into LoRA adapters for free: compute the per-layer difference between the fine-tuned and base weights, take its truncated singular value decomposition, and split the top singular directions into two adapter matrices. Because this is exactly the Eckart–Young best rank-r approximation of the weight delta, the adapters can be re-merged into the backbone with minimal reconstruction error. No training data, gradients, or per-task optimization are needed—only the two checkpoints. Across text, image, and video benchmarks on production-scale model families, the extracted rank-32 and rank-64 adapters preserve most of the delta's energy and recover full-rank task performance to within about one point, occasionally exceeding it, while enabling cheaper multi-task serving. If right, PHLoRA turns every existing full-rank fine-tune into a deployable adapter, amortizing latency across many users and cutting inference cost by up to 4x.

Core claim

The central discovery is that a fine-tuned model's full-rank update can be compressed into standard LoRA adapters after training, with no data and no gradients. For each linear layer, the method forms the weight delta ΔW = W_ft − W_base, computes the truncated SVD U_r Σ_r V_r^T, and sets B = U_r √Σ_r and A = √Σ_r V_r^T. The product BA is, by the Eckart–Young–Mirsky theorem, the unique rank-r matrix that minimizes the Frobenius distance to ΔW, so merging W_base + BA restores the fine-tuned model up to the optimal reconstruction error at that rank. Experiments on three model scales and five benchmarks in text, image, and video understanding show preserved energy growing with rank and downstrea

What carries the argument

The load-bearing object is the weight delta ΔW = W_ft − W_base defined for each target linear layer, and the machinery is the truncated singular value decomposition of that delta. The method keeps the top r singular values and their left/right singular vectors, then splits the singular values symmetrically between two LoRA matrices (B = U_r √Σ_r, A = √Σ_r V_r^T). The Eckart–Young–Mirsky theorem guarantees this product is the closest rank-r matrix to the full-rank update in Frobenius norm, which is what allows the adapters to be merged back with minimal loss and what justifies treating the extraction as 'free' (no data, no gradients). The same decomposition also provides a per-layer energy me

Load-bearing premise

The load-bearing premise is that the directions in the weight delta with the largest singular values are the directions that matter for downstream task performance—i.e., that Frobenius-optimal low-rank reconstruction of each layer's update preserves the task-relevant information, even though the paper concedes energy preservation and model quality do not always correlate.

What would settle it

Take a fine-tuned model whose task performance depends on a weight-delta direction with a small singular value (e.g., a narrow set of training examples that activate a single feature). Extract a PHLoRA adapter at rank r that keeps most of the delta's energy (Er above, say, 99%) but drops that direction, re-merge, and evaluate on the task. If task accuracy collapses despite the high preserved energy, the claim that energy-preserving truncation is task-safe is falsified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Any deployed full-rank fine-tuned model whose base weights are available can be converted to LoRA adapters without retraining, enabling dynamic multi-adapter serving on existing infrastructure.
  • Merged inference with PHLoRA adapters matches full-rank behavior to within ~1% on the tested text, image, and video tasks, so legacy checkpoints can be served at smaller memory footprint.
  • Because adapters are small, switching between many fine-tuned tasks at runtime becomes cheap; the paper estimates up to 4x throughput/cost gains in shared-adapter serving relative to naive per-adapter serving.
  • Preserved energy tracks recovery: higher ranks (e.g., 512) recover full-rank accuracy across all scales, while intermediate ranks (32–64) often match or beat full-rank on multimodal benchmarks.
  • Adapter extraction is decoupled from training, so third-party full-rank checkpoints can be made adapter-ready as long as both base and fine-tuned weights are in hand.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same spectral-truncation recipe should apply to any checkpoint delta—not just supervised fine-tuning but preference or reinforcement fine-tuning—since it never looks at the objective, only at the weight difference; this is a direct but untested extension the paper lists as future work.
  • Per-layer adaptive rank selection based on each layer's energy curve could shrink total adapter size further than the fixed global rank used here; the paper provides code for per-layer thresholds but does not evaluate it.
  • Frobenius accuracy is not the same as task accuracy: the paper's own limitation notes that energy and task quality do not always correlate, so there may exist low-energy but task-critical directions that a purely spectral method would discard; identifying them would delineate when PHLoRA fails.
  • The 4x cost reduction is borrowed from a reference serving system's benchmarks rather than measured directly on PHLoRA; the actual savings will depend on batch size, adapter count, and workload mix.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. PHLoRA proposes extracting LoRA-compatible adapters from full-rank fine-tuned checkpoints in a data-free, gradient-free manner. For each linear layer, the weight delta ΔW = W_ft − W_base is decomposed by truncated SVD, and the top-r singular vectors are repackaged as A and B matrices in the standard LoRA form. The paper claims these post-hoc adapters preserve high energy of the delta, can be pruned safely, and re-merge with negligible task degradation, while enabling scalable multi-adapter serving. Experiments are reported on three Amazon Nova model sizes across text, image, and video benchmarks (TAT-QA, MKFE, MedMCQA, VQA-RAD, CaptionGen), with ablations over ranks 32, 64, and 512 and a preserved-energy metric Er.

Significance. If the central claims hold, the method has clear practical value: it converts existing full-rank checkpoints into LoRA adapters without training data, enabling dynamic serving frameworks such as S-LoRA to host them at lower marginal cost. The mathematical core is a textbook application of the Eckart-Young theorem and is correct; there are no fitted constants or hidden optimization steps. The paper also explicitly acknowledges several limitations, including imperfect correlation between energy preservation and task quality. However, the empirical support for the key energy-to-performance transfer is incomplete and, in one reported ablation, internally contradicted. Reproducibility is currently blocked by missing code and dataset-processing scripts, which is especially important given the proprietary base models used.

major comments (4)
  1. [§4.3, Table 6; §3.3] The paper repeatedly infers that high preserved energy Er transfers to task performance: §3.3 states 'High preserved energy typically correlates with the adapter's ability to recover full-rank performance,' and §4.3 claims 'a clear correlation between preserved energy and downstream task performance.' Table 6 directly contradicts this for Nova Pro MKFE: r512 has Er=68% but Key Overlap collapses to 50.0, while r32 (Er=36%) and r64 (Er=43%) both retain ~100.0. This is a within-paper counterexample to the abstract's 'negligible degradation' and 'can be pruned safely.' The Eckart-Young theorem only bounds ||ΔW − BA||_F; no argument in the paper connects that bound to task loss, and because PHLoRA is data-free it cannot correct for high-energy but task-irrelevant directions. The anomaly might be evaluation noise, but with a single run and no error bars it cannot be dismissed. Please either ex
  2. [§4.1, Tables 3–6] All reported results are from a single run, with seeds fixed to 42 but no error bars or repeated-seed statistics. Differences of 1–2 points are treated as meaningful (e.g., PHLoRA 'surpasses' full-rank on MedMCQA and VQA-RAD). In addition, the base-model prompts are deliberately chosen to produce near-zero base scores (§4.1), which inflates the apparent fine-tuning delta and may exaggerate how much of the gain the adapter recovers. Please report means and standard deviations over at least three seeds, and include a standard, formatting-instructed prompt as a control so that the base model is not artificially handicapped.
  3. [§4.2, Conclusion] The claimed 'up to 4×' inference cost reduction is not measured in this paper. It is extrapolated from S-LoRA's reported results ('as shown in Table 3 and Figure 4 of S-LoRA'), and no S-LoRA or multi-adapter serving experiment is conducted here. The Conclusion states the method 'reduces inference GPU-hour costs by up to 4-fold' as an established result. Either run the serving experiment or explicitly frame the 4× figure as a literature-based estimate for PHLoRA-like adapters, with the caveat that actual savings depend on batch size and workload characteristics.
  4. [§C, Reproducibility] The reproducibility checklist says 'Code: All code—including SVD extraction, energy calculation, and evaluation scripts will be released' with a placeholder URL, and 'github URL to be added' appears in the contributions. Since the paper relies on proprietary Nova weights and custom sub-sampled datasets, the empirical claims cannot be verified without the actual code and dataset processing scripts. Please provide the repository and exact dataset splits before the paper is considered publishable.
minor comments (5)
  1. [§3.2, Eq. (4)] The text says the '1/2 exponent represents the element-wise square root.' For a diagonal Σ this is correct, but Σ^{1/2} conventionally denotes the matrix square root. Please state explicitly that Σ is diagonal and that Σ^{1/2} = diag(√σ_i).
  2. [§1, Contributions] The contribution bullet says the method requires 'a single forward pass and no gradients or data.' In fact the method uses no forward pass at all; the extraction is purely a decomposition of the weight delta. Replace 'single forward pass' with 'single SVD pass' or 'no forward pass.'
  3. [§4.2] 'As shown in Table 3 and Figure 4 of S-LoRA' is ambiguous because Table 3 already appears in this paper. Cite as 'Sheng et al. (2024), Table 3 and Figure 4.'
  4. [Table 2] CaptionGen is cited only to Chen and Dolan (2011), which is the MSVD dataset. The caption source, the Multi-Source Video Captioning dataset (DAMO-NLP-SG), should be cited explicitly in the table caption or in the dataset description.
  5. [Throughout] Spelling is inconsistent: 'Posthoc' in the contributions and appendix titles vs. 'post-hoc' elsewhere. Also, the footnote 'Pronounced “flora”' appears in the abstract and again as a footnote; one occurrence suffices.

Circularity Check

0 steps flagged

No circularity; PHLoRA is a direct, self-contained application of truncated SVD with externally grounded optimality.

full rationale

PHLoRA's derivation chain is: define ΔW = W_ft − W_base; compute the truncated SVD; set B and A from the top-r singular components; and invoke Eckart-Young–Mirsky for Frobenius optimality. Each step is either definitional or a classical external theorem, and no parameter is fitted to the evaluation metric. The rank r is an ablative hyperparameter swept by the authors, not estimated from target-task performance. The preserved energy E_r is a descriptive statistic computed from the same singular values, not a fitted input to the method. The abstract's performance claims are supported by direct evaluation tables, not derived from E_r; indeed the Limitations explicitly concede that 'model quality on the target task does not always correlate perfectly with energy preservation,' and Table 6's Nova Pro MKFE r=512 row is a within-paper counterexample to any strong energy-to-performance implication. That is a correctness/robustness concern, not circularity. The only self-citations (Amazon Nova model card, S-LoRA throughput) are provenance or external system references and are not load-bearing premises. The Eckart-Young result is external, machine-independent mathematics and is even re-proved in Appendix D. Therefore no circular step; score 0.

Axiom & Free-Parameter Ledger

1 free parameters · 4 axioms · 0 invented entities

PHLoRA introduces no new entities, forces, or conserved quantities. Its free-parameter load is small: a single global rank r, chosen by ablation. The heavier burden is the domain assumption that Frobenius-optimal low-rank weight deltas preserve task-relevant function, which the paper itself qualifies, and the external assumption that S-LoRA's serving benefits transfer unchanged. No new math is introduced beyond the standard Eckart-Young theorem.

free parameters (1)
  • global adapter rank r = 32 (default), 64, 512 in ablation
    The rank of the truncated SVD is fixed globally for all layers and chosen by hand/ablation, not derived from data or an energy threshold. The paper notes adaptive rank selection is left to future work, so this is a genuine free parameter of the method.
axioms (4)
  • standard math Eckart-Young-Mirsky theorem: truncated SVD gives the best rank-r Frobenius-norm approximation.
    Invoked in Section 3.2 and Appendix D to justify that BA is the optimal rank-r reconstruction of ΔW. This is a standard mathematical result.
  • domain assumption Low Frobenius error in weight delta transfers to low downstream task loss.
    Section 3.3 states 'high preserved energy typically correlates with the adapter's ability to recover full-rank performance,' but the Limitations admit 'model quality on the target task does not always correlate perfectly with energy preservation.' This unproven transfer is the load-bearing assumption behind the whole method.
  • domain assumption Both the base and the fully fine-tuned checkpoints are available.
    Stated as a requirement in the Limitations: 'PHLoRA assumes access to both base and fully fine-tuned weights.' This is an external precondition that limits the claimed generality to third-party checkpoints where the base is also accessible.
  • domain assumption S-LoRA's measured speedup applies to PHLoRA adapters.
    The cost-saving claim in Section 4.2 is not measured with PHLoRA; it is assumed to carry over from S-LoRA's benchmark results. The paper calls this 'a strong indication' rather than evidence specific to PHLoRA.

pith-pipeline@v1.3.0-alltime-deepseek · 9661 in / 9995 out tokens · 118537 ms · 2026-08-04T17:17:06.002321+00:00 · methodology

0 comments
read the original abstract

We introduce PHLoRA (Pronounced "flora"). (Post-hoc LoRA), a simple yet powerful method to extract low-rank adaptation adapters from full-rank fine-tuned models without requiring access to training data or gradients. By computing the low-rank decomposition of weight differences between a base model and its fine-tuned counterpart, our method reconstructs adapter modules that can be merged or dynamically routed at inference time via S-LoRA, or served in scalable, industry settings using platforms like NVIDIA NIM. This approach amortizes latency overhead across requests and yields substantial cost savings. Unlike prior work that trains each adapter explicitly, our approach decouples fine-tuning from adapter generation, allowing adapter extraction from existing full-rank models or third-party checkpoints. Experiments on text, image, and video benchmarks using the Amazon Nova model family demonstrate that extracted adapters preserve high energy from the full weight delta, can be pruned safely, and yield negligible degradation in downstream task performance when re-merged. Overall, PHLoRA provides a practical path for making all existing full-rank checkpoints adapter-ready, democratizing scalable inference for all models.

Figures

Figures reproduced from arXiv: 2509.10971 by Anjie Fang, Bhoomit Vasani, Jack FitzGerald, Sushmit Vaish.

Figure 1
Figure 1. Figure 1: Although we fix r globally in this work, our code supports per-layer adaptive rank selection based on a desired energy threshold. 4 Experiments 4.1 Experimental Setup We benchmark PHLoRA on three text only datasets, TAT-QA (Zhu et al., 2021), Medical Knowledge from Extracts (MKFE) (Owkin, 2024), MedMCQA (Pal et al., 2022); one image and text dataset, VQA-RAD (Lau et al., 2018); and one video and text datas… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 18 linked inside Pith

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Amazon AGI. 2024. https://www.amazon.science/publications/the-amazon-nova-family-of-models- technical-report-and-model-card The amazon nova family of models: Technical report and model card

    2024

  4. [4]

    Yang Cao. 2024. https://arxiv.org/abs/2409.00055 Sorsa: Singular values and orthonormal regularized singular vectors adaptation of large language models . Preprint, arXiv:2409.00055

    Pith/arXiv arXiv 2024

  5. [5]

    David Chen and Bill Dolan. 2011. https://www.microsoft.com/en-us/research/publication/collecting-highly-parallel-data-for-paraphrase-evaluation/ Collecting highly parallel data for paraphrase evaluation . In Collecting Highly Parallel Data for Paraphrase Evaluation. Association for Computational Linguistics

    2011

  6. [6]

    Jiaao Chen, Aston Zhang, Xingjian Shi, Mu Li, Alex Smola, and Diyi Yang. 2023. https://arxiv.org/abs/2301.01821 Parameter-efficient fine-tuning design spaces . Preprint, arXiv:2301.01821

    Pith/arXiv arXiv 2023

  7. [7]

    Multi-source video captioning dataset

    DAMO-NLP-SG. Multi-source video captioning dataset. https://huggingface.co/datasets/DAMO-NLP-SG/Multi-Source-Video-Captioning. Accessed: 2025-05-19

    2025

  8. [8]

    Tillman, and Luke Zettlemoyer

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Kai Tai, Arthur Szlam, Ari S. Tillman, and Luke Zettlemoyer. 2023. https://arxiv.org/abs/2305.14314 Qlora: Efficient finetuning of quantized llms . In Advances in Neural Information Processing Systems

    Pith/arXiv arXiv 2023

  9. [9]

    Marion Young

    Carl Eckart and G. Marion Young. 1936. https://api.semanticscholar.org/CorpusID:10163399 The approximation of one matrix by another of lower rank . Psychometrika, 1:211--218

    1936

  10. [10]

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2023. https://arxiv.org/abs/2210.17323 Gptq: Accurate post-training quantization for generative pre-trained transformers . Preprint, arXiv:2210.17323

    Pith/arXiv arXiv 2023

  11. [11]

    Thomas Gauthier-Caron. 2024. Lord: Low-rank decomposition from full-rank fine-tuning. https://github.com/thomasgauthier/LoRD. Accessed: 2025-06-15

    2024

  12. [12]

    Golub and Charles F

    Gene H. Golub and Charles F. Van Loan. 2013. Matrix Computations, 4th edition. Johns Hopkins University Press, Baltimore, MD, USA

    2013

  13. [13]

    Soufiane Hayou, Nikhil Ghosh, and Bin Yu. 2024. https://arxiv.org/abs/2402.12354 Lora+: Efficient low rank adaptation of large models . Preprint, arXiv:2402.12354

    Pith/arXiv arXiv 2024

  14. [14]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. 2022. https://arxiv.org/abs/2106.09685 Lora: Low-rank adaptation of large language models . In International Conference on Learning Representations

    Pith/arXiv arXiv 2022

  15. [15]

    Hugging Face . 2023. Peft: Parameter efficient fine-tuning library. https://github.com/huggingface/peft. Accessed: 2025-05-18

    2023

  16. [16]

    Kolda and Brett W

    Tamara G. Kolda and Brett W. Bader. 2009. https://epubs.siam.org/doi/abs/10.1137/07070111X Tensor decompositions and applications . SIAM Review, 51(3):455--500

    doi:10.1137/07070111x 2009

  17. [17]

    Lau, Soumya Gayen, Asma Ben Abacha, and Dina Demner-Fushman

    Jason J. Lau, Soumya Gayen, Asma Ben Abacha, and Dina Demner-Fushman. 2018. https://doi.org/10.1038/sdata.2018.251 A dataset of clinically generated visual questions and answers about radiology images . Scientific Data, 5(1):180251

    doi:10.1038/sdata.2018.251 2018

  18. [18]

    Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. https://arxiv.org/abs/2104.08691 The power of scale for parameter-efficient prompt tuning . Preprint, arXiv:2104.08691

    Pith/arXiv arXiv 2021

  19. [19]

    Muyang Li, Yujun Lin, Zhekai Zhang, Tianle Cai, Xiuyu Li, Junxian Guo, Enze Xie, Chenlin Meng, Jun-Yan Zhu, and Song Han. 2025. https://arxiv.org/abs/2411.05007 Svdquant: Absorbing outliers by low-rank components for 4-bit diffusion models . Preprint, arXiv:2411.05007

    arXiv 2025

  20. [20]

    Fanxu Meng, Zhaohui Wang, and Muhan Zhang. 2025. https://arxiv.org/abs/2404.02948 Pissa: Principal singular values and singular vectors adaptation of large language models . arXiv preprint arXiv:2404.02948

    Pith/arXiv arXiv 2025

  21. [21]

    Mohammad Mozaffari, Amir Yazdanbakhsh, and Maryam Mehri Dehnavi. 2025. https://arxiv.org/abs/2410.09615 Slim: One-shot quantization and sparsity with low-rank approximation for llm weight compression . Preprint, arXiv:2410.09615

    Pith/arXiv arXiv 2025

  22. [22]

    Owkin. 2024. Medical knowledge from extracts (mkfe). https://huggingface.co/datasets/owkin/medical_knowledge_from_extracts. Accessed: 2025-05-18

    2024

  23. [23]

    Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. 2022. Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. In Proceedings of the Conference on Health, Inference, and Learning, pages 248--260. PMLR

    2022

  24. [24]

    Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. 2021. Adapterfusion: Non-destructive task composition for transfer learning. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 487--503

    2021

  25. [25]

    Manning, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2024. https://arxiv.org/abs/2305.18290 Direct preference optimization: Your language model is secretly a reward model . Preprint, arXiv:2305.18290

    Pith/arXiv arXiv 2024

  26. [26]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. https://api.semanticscholar.org/CorpusID:28695052 Proximal policy optimization algorithms . ArXiv, abs/1707.06347

    Pith/arXiv arXiv 2017

  27. [27]

    Gonzalez, and Ion Stoica

    Ying Sheng, Shiyi Cao, Dacheng Li, Coleman Hooper, Nicholas Lee, Shuo Yang, Christopher Chou, Banghua Zhu, Lianmin Zheng, Kurt Keutzer, Joseph E. Gonzalez, and Ion Stoica. 2024. https://arxiv.org/abs/2311.03285 S-lora: Serving thousands of concurrent lora adapters . Preprint, arXiv:2311.03285

    Pith/arXiv arXiv 2024

  28. [28]

    Yi-Lin Sung, Jaemin Cho, and Mohit Bansal. 2022. https://arxiv.org/abs/2112.06825 Vl-adapter: Parameter-efficient transfer learning for vision-and-language tasks . Preprint, arXiv:2112.06825

    Pith/arXiv arXiv 2022

  29. [29]

    Qinsi Wang, Jinghan Ke, Masayoshi Tomizuka, Yiran Chen, Kurt Keutzer, and Chenfeng Xu. 2025 a . https://arxiv.org/abs/2502.02723 Dobi-svd: Differentiable svd for llm compression and some new perspectives . Preprint, arXiv:2502.02723

    Pith/arXiv arXiv 2025

  30. [30]

    Xin Wang, Yu Zheng, Zhongwei Wan, and Mi Zhang. 2025 b . https://arxiv.org/abs/2403.07378 Svd-llm: Truncation-aware singular value decomposition for large language model compression . Preprint, arXiv:2403.07378

    Pith/arXiv arXiv 2025

  31. [31]

    Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel. 2021. Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1--16

    2021

  32. [32]

    Xiaohui Zhang, Xiang Lisa Li, and 1 others. 2023. https://arxiv.org/abs/2303.10512 Adaptive low-rank adapter (adalora): Compressing lora further via rank allocation . arXiv preprint arXiv:2303.10512

    Pith/arXiv arXiv 2023

  33. [33]

    Zihan Zhong, Zhiqiang Tang, Tong He, Haoyang Fang, and Chun Yuan. 2024. https://arxiv.org/abs/2401.17868 Convolution meets lora: Parameter efficient finetuning for segment anything model . Preprint, arXiv:2401.17868

    Pith/arXiv arXiv 2024

  34. [34]

    Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, and Tat-Seng Chua. 2021. https://arxiv.org/abs/2105.07624 Tat-qa: A question answering benchmark on a hybrid of tabular and textual content in finance . Preprint, arXiv:2105.07624

    Pith/arXiv arXiv 2021

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.