Pith. sign in

REVIEW 3 major objections 5 minor 19 references

This paper claims that an ensemble of foundation-model adapters can be built by freezing each weight matrix's singular vectors and training only per-member singular values, matching deep-ensemble calibration with under 1% parameter overhead

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 06:42 UTC pith:MSSHZJJP

load-bearing objection A useful, well-tested parameter-efficient ensemble idea whose claims outrun its evidence in two places—a missing diversity check and imported baselines—but worth serious review. the 3 major comments →

arxiv 2601.22068 v2 pith:MSSHZJJP submitted 2026-01-29 cs.LG

Quantifying the Uncertainty of Foundation Models with Singular Value Ensembles

Mehmet Ozgur Turkoglu , Dominik J. M\"uhlematter , Alexander Becker , Konrad Schindler , Helge Aasen This is my paper
classification cs.LG MSC 68T07
keywords singular value decompositionimplicit ensemblinguncertainty quantificationcalibrationparameter-efficient fine-tuningfoundation modelsdeep ensemblesexpected calibration error
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to make epistemic-uncertainty estimation practical for large pretrained models. Its core move is to treat the singular vectors of each pretrained weight matrix as a shared 'knowledge basis' and create ensemble members that differ only in the singular values, i.e., in how strongly each direction contributes. The paper claims that training several copies of the singular values — with small random perturbations and ordinary mini-batch sampling — produces functionally diverse members, and that averaging their predictions yields calibration comparable to a deep ensemble while adding under 1% to the parameter count. Support comes from fine-tuning experiments on vision and NLP benchmarks with backbones from 22M to 7B parameters, including out-of-distribution detection and corrupted-input stress tests. A sympathetic reader would care because it suggests that calibrated uncertainty need not cost an extra copy of the model.

Core claim

The central claim is that repeated rescaling of one orthonormal singular-vector basis is enough to generate ensemble diversity. Concretely, SVE decomposes each pretrained weight matrix W as UΣV⊤, freezes U and V, replaces Σ with M trainable copies Σ(m) initialized as Σ⊙(1+ε(m)) with ε(m) small Gaussian noise, and reconstructs member weights as W(m)=UΣ(m)V⊤. Members share every layer's singular vectors and differ only in the per-member diagonal rescaling; separate classification heads are added per member. The paper reports that with M=4–16 members, SVE matches or beats deep ensembles on calibration metrics (ECE, NLL, Brier) on most tested settings, stays competitive in accuracy, and does so

What carries the argument

The central object is a singular value decomposition used as an ensembling parameterization. For each target weight matrix, the pretrained singular vectors U and V are frozen and shared by all members; they encode the paper's hypothesized 'knowledge directions.' The only trainable per-member parameters are the diagonal singular-value vectors Σ(m), one per member, so the overhead per layer is the length of that diagonal instead of a full low-rank adapter matrix pair. Diversity is meant to arise from (i) multiplicative Gaussian perturbation at initialization and (ii) random mini-batch order during joint training, which makes members land on distinct rescalings of the same basis. This is also w

Load-bearing premise

The load-bearing premise is that joint training with small random perturbations and mini-batch sampling really makes members diverge to meaningfully different singular-value vectors; the paper offers no quantitative diversity metric, only qualitative trajectories, and if members collapse to similar vectors the ensemble degenerates to a single model with no epistemic uncertainty.

What would settle it

Train SVE with M=16 and measure average pairwise disagreement among members on test inputs (e.g., fraction of inputs with differing argmax, or variance of softmax probabilities), then compare with a 16-member deep ensemble. If SVE members disagree far less yet show similar ECE, the claimed diversity mechanism is not what drives the calibration. A second check: randomize the frozen singular vectors (replace U,V with random orthogonal matrices) and retrain — if calibration still holds, the knowledge-basis assumption is not load-bearing; if it fails, freezing meaningful directions is essential.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Calibrated epistemic uncertainty becomes affordable for single-GPU fine-tuning of large models, not just for well-resourced labs.
  • On ARC-Easy with a 7B-parameter model, SVE at M=16 reaches ECE 3.8%, nearly matching the Bayesian BLoB method (3.6%) while remaining far simpler and cheaper, suggesting implicit ensembles can rival explicit Bayesian approximations on calibration.
  • The advantage of SVE grows with the strength of the pretrained representation: with a strong self-supervised backbone it can beat a deep ensemble, which implies the method will keep improving as foundation models improve.
  • SVE keeps the frozen singular vectors intact, so it can be composed with other parameter-efficient adapters or distilled into a single network to cut inference FLOPs — directions the paper names as future work.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial: A quantitative diversity check is the natural next step. The paper only shows singular-value trajectories; if pairwise prediction disagreement among SVE members is much lower than among deep-ensemble members, the reported calibration gains may come from averaging artifacts rather than genuine epistemic diversity.
  • Editorial: The knowledge-direction premise suggests a geometric reading of uncertainty: a model is uncertain when members strongly disagree about how to reweight shared directions. That could connect uncertainty estimates to interpretability tools that examine a layer's singular vectors.
  • Editorial: Because the trainable parameters are just one vector per layer, SVE members can be stored as tiny deltas on top of the frozen base, enabling on-device uncertainty with essentially no extra weight memory — an extension the paper does not quantify.
  • Editorial: A testable extension is to learn which singular directions matter for a task and perturb only those, instead of all directions uniformly, potentially improving diversity and calibration further.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Singular Value Ensemble (SVE), an implicit ensembling method for fine-tuned foundation models. For each pretrained weight matrix W, SVE computes the SVD, freezes the singular vectors U and V, and trains M copies of the singular values Σ^(m), initialized with small multiplicative noise. Members share the same singular-vector "knowledge basis" but differ in how strongly each direction is used, and their softmax outputs are averaged at test time. The paper argues this creates a diverse ensemble with less than 1% parameter overhead. Experiments cover vision (Flowers102, CIFAR-100, DTD, Oxford Pets, CIFAR-100-C, OOD detection) and NLP (SST-2, ARC-Easy with LLaMA-2-7B), comparing against deep ensembles, LoRA-Ensemble, MC Dropout, Batch Ensemble, and Bayesian LoRA variants.

Significance. The core idea is simple and potentially valuable: if singular vectors of pretrained weights are meaningful knowledge directions, then rescaling singular values is a very cheap way to generate ensemble members. The parameter-count derivation (Eq. 7) is explicit, and the experimental sweep from BERT-base to LLaMA-2-7B is a strength. However, the central mechanism—diversity of per-member singular values—is only supported by qualitative plots, and the strongest calibration comparisons on ARC-Easy rely on imported baselines. If the diversity concern is addressed with direct quantitative metrics and the baseline comparison is made controlled, the paper would be a solid contribution to efficient uncertainty quantification.

major comments (3)
  1. [§4, Eqs. (5)-(6), Appendix B] The central mechanism is member diversity, but the paper provides no quantitative evidence that members do not collapse. Eq. (5) initializes each Σ^(m) with a 1% multiplicative Gaussian perturbation; Eq. (6) trains all members jointly with no diversity penalty or coupling. With shared mini-batches, the expected gradient is the same for all members, so nothing in the objective prevents convergence to a common Σ; the ensemble would then reduce to one feature extractor with M heads. Appendix B shows only relative changes of top singular values from pretrained values, not pairwise distances among Σ^(m) or prediction disagreement on ID/OOD inputs. Please report quantitative diversity metrics (e.g., average pairwise cosine distance of trained Σ^(m), softmax disagreement between members) and relate them to the ECE/OOD improvements; otherwise the reported gains cannot be attributed to the singul
  2. [Table 3 (ARC-Easy)] The headline ARC-Easy calibration results are not obtained under a controlled comparison. Baselines such as Deep Ensemble (M=3), Ckpt Ensemble (M=3), Bayes-LoRA, Blob (N=5/10), and LoRA-Ensemble (M=5) are imported from Yang et al. (2024) and Wang et al. (2023), with different training budgets and ensemble sizes, while SV-Ensemble uses M=16 under the authors' own pipeline. This mixes the method with the training budget and makes the claim that SV-Ensemble 'approaches' Blob's calibration incomparable. Please re-run the baselines with the same protocol, or clearly mark the imported rows and limit the conclusions to the rows that are directly compared.
  3. [Abstract and §1 Contributions] The contributions claim 'state-of-the-art calibration' and the abstract claims SVE 'performs comparable to an explicit ensemble.' In Table 1 (CIFAR-100), LoRA-Ensemble achieves ECE 1.1±0.2 vs SV-Ensemble 2.1±0.2 and better NLL; in Table 4 (OOD), LoRA-Ensemble has higher AUROC/AUPRC and lower FPR@95. The claims should be qualified to name LoRA-Ensemble as a stronger method on these benchmarks. As written, the abstract and contribution list overstate the empirical standing of SVE.
minor comments (5)
  1. [Appendix C, Table 7] Backbone entries are inconsistent: Table 1 headers say Flowers102 uses 'DINO ViT-S/14' while Table 7 lists 'DINO ViT-S/16'; there is also a typo 'DINOv ViT-S/16'. Please harmonize the naming.
  2. [§4, Eq. (5)] The text says σ_init < 1 'guarantees positivity', but 1+ε can be negative for ε < -1, even if unlikely for σ_init=0.01. Rephrase as 'with high probability' and state how negative values are handled during training.
  3. [§4 and Appendix C] The main text says σ_init=0.01 in most experiments, but Table 7 lists 0.005 for Flowers102, CIFAR-100, and DTD. Please reconcile these values.
  4. [References] The reference to 'Bommasani, R. (2021)' should be 'Bommasani et al.' for the Foundation Models report, which is a multi-author document.
  5. [General] The manuscript is marked 'Preliminary work' and code is promised only 'at a later stage.' Adding a reproducibility statement or releasing code would materially strengthen the paper's claims.

Circularity Check

0 steps flagged

No circular reduction in the SVE derivation; minor self-citations in baseline framing are not load-bearing.

full rationale

Walking the claimed derivation chain: the method (Eqs. 3-6) freezes singular vectors and trains per-member singular values; the parameter-efficiency expression (Eq. 7) is a direct count of trainable parameters, not a fitted quantity later relabeled as a prediction. The central 'knowledge directions' premise is imported from external prior work (Millidge & Black 2022; Elhage et al. 2022; Staats et al. 2024), not from the authors' own prior results. The self-citations that do appear (LoRA-Ensemble, Mühlematter et al. 2024; FiLM-Ensemble, Turkoglu et al. 2022) are used as related work and as comparison baselines; even if they create a mild framing bias, the claim that SVE performs comparably to explicit ensembles is tested against external baselines (Deep Ensemble, MC Dropout, Batch Ensemble, Bayes-LoRA, Blob) on independent benchmarks, so the self-citations are not load-bearing. The skeptic's concern that member diversity may collapse is an empirical validation gap: Appendix B gives qualitative singular-value trajectories rather than a quantitative diversity or collapse metric, but nothing in Eqs. 4-6 defines the ensemble quality in terms of the very singular values being trained, so this is not circular. The stated limitations (multiple forward passes, FLOPs comparable to deep ensembles, quantization difficulty) are acknowledged costs, not circular steps.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

No new physical or conceptual entities are introduced. The central claim rests on the knowledge-basis hypothesis and the diversity hypothesis, both imported from prior work or validated only qualitatively. The only free parameters are σ_init and M, both hand-set hyperparameters, and no fitted constants appear in the derivation.

free parameters (2)
  • σ_init (initialization perturbation scale) = 0.005 (Flowers102, CIFAR-100, DTD); 0.01 (Oxford Pets, ARC-Easy, SST-2)
    Hand-chosen scale for the multiplicative Gaussian perturbation that seeds ensemble-member diversity. Not derived; a method-specific hyperparameter.
  • M (ensemble size) = 4 (vision), 8 (SST-2), 16 (ARC-Easy)
    Chosen per task, with Appendix A showing accuracy and calibration improve with M on ARC-Easy. This is effectively tuned to task difficulty and budget.
axioms (4)
  • domain assumption Singular vectors of pretrained weight matrices correspond to meaningful (orthogonal) 'knowledge directions'.
    Borrowed from Millidge & Black 2022, Elhage et al. 2022, Staats et al. 2024 (Section 3). If false, freezing U and V and reweighting Σ has no semantic basis.
  • domain assumption Freezing U and V while training Σ preserves enough expressivity for task adaptation.
    Supported by SVF/SVFit literature, but not proven for all tasks; the CIFAR-100 result (SVE below LoRA-Ensemble by 2.6%) suggests limits.
  • domain assumption Per-member initialization perturbations and mini-batch sampling suffice to keep members diverse during joint training.
    Empirically supported by Appendix B figures, but no diversity metric or collapse analysis is provided. If members collapse, the ensemble degenerates.
  • standard math SVD of any real matrix exists and W = U Σ V^T reconstructs the weight matrix.
    Standard linear algebra; invoked in Eq. (3) and throughout the method.

pith-pipeline@v1.3.0-alltime-deepseek · 17943 in / 11990 out tokens · 113091 ms · 2026-08-03T06:42:31.686382+00:00 · methodology

0 comments
read the original abstract

Foundation models have become a dominant paradigm in machine learning, achieving remarkable performance across diverse tasks through large-scale pretraining. However, they often yield overconfident, uncalibrated predictions. The standard approach to quantifying epistemic uncertainty are ensembles of multiple independently trained models. But their computational cost scales linearly with ensemble size, making them impractical for large foundation models. We propose Singular Value Ensemble (SVE), a parameter-efficient implicit ensembling method. SVE builds on a simple, but powerful core assumption: namely, that the singular vectors of the weight matrices correspond to meaningful directions in the representation space. If the singular vectors are indeed meaningful (orthogonal) "knowledge directions", then a model ensemble can be obtained by modulating only how strongly each direction contributes to the output. Rather than learning new parameters for each ensemble member, we freeze the singular vectors and only train per-member singular values that rescale the contribution of each direction in that shared knowledge basis. Ensemble diversity emerges naturally during joint training as stochastic initialization and random batch sampling cause different members to converge to different combinations of the same underlying knowledge. SVE performs comparable to an explicit ensemble, while increasing the parameter count of the base model by <1%, making principled uncertainty estimation accessible in resource-constrained settings. We validate SVE on NLP and vision tasks with various different backbones and show that it improves calibration while maintaining predictive accuracy.

Figures

Figures reproduced from arXiv: 2601.22068 by Alexander Becker, Dominik J. M\"uhlematter, Helge Aasen, Konrad Schindler, Mehmet Ozgur Turkoglu.

Figure 1
Figure 1. Figure 1: Schematic of Singular Value Ensembles (SV-Ensembles). Left (blue background): A given linear projection W of a Foundation Model visualized in 2D by means of its effect on the unit circle as well as two unit vectors. Middle (red background): W can be decomposed via SVD into orthogonal right-singular vectors V T , diagonal singular values Σ, and orthogonal left-singular vectors U. SV-Ensemble freezes U and V… view at source ↗
Figure 2
Figure 2. Figure 2: CIFAR-100 test performance vs. pretraining scale for a ViT-S model. With progressively stronger backbones (Random Init, DINOv1, DINOv2), SV-Ensembles show increasingly larger gains relative to other ensembling schemes. I.e., the benefits of SV-based ensembling grow with representation quality. Vision Results [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Dataset Robustness Comparison with varying shift intensity on CIFAR-100-C. (i) Accuracy, (ii) Expected Calibration Error (ECE), and (iii) Negative Log-Likelihood (NLL) [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Top Singular Values’ Relative Change (%) from Pretrained Weights for QKV attention projection layers of DINOv2 ViT-S/14 on Oxford Pets dataset. • LoRA-Ensemble: Per-member LoRA adapters with rank r = 8 and α = 8, applied to attention projections and fully connected layers. Learning rate set to 1 × 10−3 . • BatchEnsemble: Rank-one perturbations with M = 4 members. Learning rate set to 1 × 10−4 . All experim… view at source ↗
Figure 5
Figure 5. Figure 5: Top Singular Values’ Relative Change (%) from Pretrained Weights for Output Projection layers of DINOv2 ViT-S/14 on Oxford Pets dataset. SST-2 The Stanford Sentiment Treebank (Socher et al., 2013) is a binary sentiment classification benchmark derived from movie reviews, commonly used to evaluate sentence-level sentiment understanding. We report results on the validation set (872 entries) following standar… view at source ↗
Figure 6
Figure 6. Figure 6: Top Singular Values’ Relative Change (%) from Pretrained Weights for FC1 layers of DINOv2 ViT-S/14 on Oxford Pets dataset. are annotated via crowd-sourcing and are labeled with a primary (key) attribute as well as additional joint attributes. The dataset is provided with predefined train, validation, and test splits, each containing 40 images per class. Oxford Pets (Parkhi et al., 2012) is an image classif… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

19 extracted references · 14 linked inside Pith

  1. [3]

    D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learn- ers.Advances in Neural Information Processing Systems (NeurIPS), 33:1877–1901,

    1901

  2. [4]

    A., Dong, Z., Wang, H., Okuno, T., Nakata, Y ., Keutzer, K., and Zhang, S

    Chen, A., Yang, H., Gan, Y ., Gudovskiy, D. A., Dong, Z., Wang, H., Okuno, T., Nakata, Y ., Keutzer, K., and Zhang, S. Split-Ensemble: Efficient OOD-aware en- semble via task and model splitting.arXiv preprint arXiv:2312.09148,

    Pith/arXiv arXiv

  3. [6]

    Toy models of superposition.arXiv preprint arXiv:2209.10652,

    Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., et al. Toy models of superposition.arXiv preprint arXiv:2209.10652,

    Pith/arXiv arXiv

  4. [10]

    Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D

    Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., de las Casas, D., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., Lavaud, L. R., Lachaux, M.- A., Stock, P., Scao, T. L., Lavril, T., Wang, T., Lacroix, T., and Sayed, W. E. Mistral 7b.preprint arXiv:2310.06825,

    Pith/arXiv arXiv

  5. [12]

    J., Halbheer, M., Becker, A., Narnhofer, D., Aasen, H., Schindler, K., and Turkoglu, M

    M¨uhlematter, D. J., Halbheer, M., Becker, A., Narnhofer, D., Aasen, H., Schindler, K., and Turkoglu, M. O. LoRA-Ensemble: Efficient uncertainty modelling for self- attention networks.arXiv preprint arXiv:2405.14438,

    arXiv

  6. [15]

    LLaMA: Open and efficient founda- tion language models.arXiv preprint arXiv:2302.13971,

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozi`ere, B., Goyal, N., Hambro, E., Azhar, F., et al. LLaMA: Open and efficient founda- tion language models.arXiv preprint arXiv:2302.13971,

    Pith/arXiv arXiv

  7. [16]

    Exploring pre- dictive uncertainty and calibration in nlp: A study on the impact of method & data scarcity.arXiv preprint arXiv:2210.15452,

    Ulmer, D., Frellsen, J., and Hardmeier, C. Exploring pre- dictive uncertainty and calibration in nlp: A study on the impact of method & data scarcity.arXiv preprint arXiv:2210.15452,

    Pith/arXiv arXiv

  8. [17]

    LoRA ensembles for large language model fine-tuning.arXiv preprint arXiv:2310.00035,

    Wang, X., Aitchison, L., and Rudolph, M. LoRA ensembles for large language model fine-tuning.arXiv preprint arXiv:2310.00035,

    Pith/arXiv arXiv

  9. [2009]

    as a semantically shifted dataset relative to models trained on CIFAR-100. For robustness under distribution shift, we use CIFAR-100- C (Hendrycks & Dietterich, 2019), which applies 19 corruption types (e.g., Gaussian noise, blur, weather effects) at five severity levels to the CIFAR-100 test set. E. Definitions of Evaluation Metrics We evaluate our model...

    2019

  10. [2013]

    Small singular values matter: A random matrix analysis of transformer models.arXiv preprint arXiv:2410.17770,

    Staats, M., Thamm, M., and Rosenow, B. Small singular values matter: A random matrix analysis of transformer models.arXiv preprint arXiv:2410.17770,

    arXiv

  11. [2014]

    Think you have solved question answering? try ARC, the AI2 reasoning chal- lenge.arXiv preprint arXiv:1803.05457,

    Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., and Tafjord, O. Think you have solved question answering? try ARC, the AI2 reasoning chal- lenge.arXiv preprint arXiv:1803.05457,

    Pith/arXiv arXiv

  12. [2015]

    On the opportunities and risks of foundation models.arXiv preprint arXiv:2108.07258,

    Bommasani, R. On the opportunities and risks of foundation models.arXiv preprint arXiv:2108.07258,

    Pith/arXiv arXiv

  13. [2017]

    The power of scale for parameter-efficient prompt tuning.arXiv preprint arXiv:2104.08691,

    Lester, B., Al-Rfou, R., and Constant, N. The power of scale for parameter-efficient prompt tuning.arXiv preprint arXiv:2104.08691,

    Pith/arXiv arXiv

  14. [2019]

    and Gimpel, K

    Hendrycks, D. and Gimpel, K. A baseline for detecting misclassified and out-of-distribution examples in neural networks.arXiv preprint arXiv:1610.02136,

    Pith/arXiv arXiv

  15. [2020]

    OSoRA: Output- dimension and singular-value initialized low-rank adapta- tion.arXiv preprint arXiv:2505.14350,

    Han, J., Zhang, S., and Zhang, K. OSoRA: Output- dimension and singular-value initialized low-rank adapta- tion.arXiv preprint arXiv:2505.14350,

    Pith/arXiv arXiv

  16. [2021]

    Qwen Technical Report.arXiv preprint arXiv:2309.16609,

    Bai, J., Bai, S., Chu, Y ., Cui, Z., Dang, K., Deng, X., Fan, Y ., Ge, W., Han, Y ., Huang, F., et al. Qwen Technical Report.arXiv preprint arXiv:2309.16609,

    Pith/arXiv arXiv

  17. [2022]

    and Dietterich, T

    Hendrycks, D. and Dietterich, T. Benchmarking neural network robustness to common corruptions and perturba- tions.arXiv preprint arXiv:1903.12261,

    Pith/arXiv arXiv 1903

  18. [2023]

    Ablation Study: Number of Ensemble Members Both accuracy and calibration improve significantly from M= 1 to M= 16 , on a complex task

    12 Making Foundation Models Probabilistic via Singular Value Ensembles A. Ablation Study: Number of Ensemble Members Both accuracy and calibration improve significantly from M= 1 to M= 16 , on a complex task. See Tab.6. However, we observe that easier tasks like Flowers102 classification do not need many members; M= 4 is sufficient for optimal accuracy an...

    2023

  19. [2024]

    SVFit: Parameter-efficient fine-tuning of large pre-trained models using singular values.arXiv preprint arXiv:2409.05926,

    Sun, C., Wei, J., Wu, Y ., Shi, Y ., He, S., Ma, Z., Xie, N., and Yang, Y . SVFit: Parameter-efficient fine-tuning of large pre-trained models using singular values.arXiv preprint arXiv:2409.05926,

    Pith/arXiv arXiv