Pith. sign in

REVIEW 3 major objections 9 minor 15 references

LoRA creates forgetting-causing intruder singular vectors exactly when update strength crosses a per-layer threshold read from the pretrained weight spectrum alone.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 15:08 UTC pith:LDXN3YDN

load-bearing objection A usable spectrum-only per-layer LoRA intruder threshold with real multi-architecture evidence; the ε/edge convention is the soft spot, not a collapse. the 3 major comments →

arxiv 2607.23711 v1 pith:LDXN3YDN submitted 2026-07-26 cs.LG stat.ML

The Intruder Threshold: A Spectral Law for LoRA Fine-Tuning

Peng Xie This is my paper
classification cs.LG stat.ML
keywords LoRAintruder dimensionscatastrophic forgettingspiked random matricessingular spectrumparameter-efficient fine-tuningsecular equationweight-matrix perturbation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

LoRA can invent new top singular directions in a weight matrix that are nearly orthogonal to everything the pretrained model already knew; those intruders drive catastrophic forgetting. This paper gives a closed-form, per-layer critical update strength, computed from one SVD of the pretrained matrix with no fitted constants, that predicts when an intruder appears. Across thousands of layer scans on Transformers, a state-space model, a mixture-of-experts model, and an encoder-decoder, the predicted threshold lands within a factor of two of the measured onset on most layers, flags which layers will carry intruders at deployment, and tracks where WikiText-2 perplexity starts to collapse. Full fine-tuning never crosses the same threshold because it spreads its energy too thinly. The same numbers turn into a simple per-layer budget that cuts forgetting sharply on fragile models without hurting task performance.

Core claim

There is a universal per-layer critical LoRA strength s* = θ̄ / (γ σ₁(BA)), where θ̄ is obtained by evaluating the rectangular spiked-deformation transform on the measured singular spectrum of the pretrained weight W alone. Above that strength a new leading singular vector of W + sγBA appears that is nearly orthogonal to all pretrained singular vectors; below it, it does not. The same law explains why full fine-tuning produces no intruders and where retention loss begins.

What carries the argument

The rectangular spiked-deformation transform D_μ on the empirical spectrum of W, which yields the critical spike strength θ̄ = D(z₀)^{-1/2} and, with it, the a-priori scale s*; backed by an exact secular equation on a small core matrix that returns the full updated spectrum and overlaps without large-matrix SVDs at each scan point.

Load-bearing premise

The main prediction treats the LoRA update as sitting in generic random position relative to the pretrained singular vectors, so direction-dependent overlaps can be replaced by simple averages; real trained adapters systematically deviate from that ideal.

What would settle it

Scan real LoRA adapters layer by layer over a strength grid and check whether the first appearance of an intruder (max cosine below 0.5 with all pretrained singular vectors) falls within a factor of two of the spectrum-only s* on most layers; systematic large misses, or collapse of deployment AUC near 0.5, would refute the law.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A single pretrained SVD per layer forecasts which layers will grow intruders before any fine-tuning run.
  • Capping per-layer LoRA scale so the end-of-training spike stays below the predicted threshold cuts forgetting on fragile models at no task cost and needs no validation sweeps.
  • Norm-matched edits show that only threshold-crossing layers, not raw update size, carry the bulk of retention loss.
  • Full fine-tuning stays safe because its update energy is diluted far below every layer’s threshold; concentration versus diffusion is the operative distinction.
  • Third-party adapters obey the same numbers, so the law can audit external checkpoints in minutes.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Model cards could publish per-layer θ̄ as a fragility fingerprint, letting practitioners choose ranks and scales before training.
  • Merge and task-arithmetic recipes could weight layers by distance to threshold to reduce interference without extra training.
  • The residual early-arrival bias suggests gradient training itself is anti-aligned; initialization or regularizers that restore free-position statistics might raise the practical threshold.
  • The same spectral instrument could price safe update strength for other low-rank PEFT methods, not only LoRA.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 9 minor

Summary. The paper derives a per-layer critical LoRA update strength s* = θ̄/(γσ₁(BA)), where θ̄ = D_μ(z₀)^{-1/2} is obtained from the rectangular spiked-deformation transform of Benaych-Georges & Nadakuditi evaluated on the measured empirical spectrum of the pretrained weight matrix W alone. An exact secular-equation rung (Prop. 1, Eqs. 1–2) provides both the measurement instrument and a verification path; the universal rung (Eq. 4) assumes free position of the update relative to the pretrained basis. In a pre-specified study (18 adapters, 9,840 layer scans across seven base models spanning dense Transformers, an SSM, an MoE, and an encoder-decoder), the law localizes the empirical intruder onset within a factor of two on 82.2% of layers [0.785, 0.862], ranks layers at ρ = 0.661, separates intruder-bearing from intruder-free layers at deployment at mean AUC 0.89, predicts the WikiText-2 perplexity knee (ρ = 0.86), and transfers unchanged to six third-party Hub adapters. Norm-matched interventions show supercritical layers carry the forgetting, and a threshold-derived per-layer budget rule cuts forgetting by 62% on the most fragile model. The systematic 1.2–1.5× early arrival of real intruders is measured and dissected into alignment structure plus multi-spike order statistics via the exact secular determinant.

Significance. If the results hold, this is the first quantitative, a-priori, per-layer predictor of intruder-dimension emergence validated on real 7–8B models, and it converts a qualitative phenomenon (Shuttleworth et al. 2024) into a measurable spectral scale with a mechanistic resolution of the LoRA-vs-full-FT asymmetry (concentration vs. diffusion against a common threshold). The validation methodology deserves explicit credit: gates and falsification bars fixed before scans; an exact secular equation verified in float64 to ~1e-3 relative error at the fp32 noise floor; bootstrap intervals over adapters; training-seed stability checks; a reported pre-specified negative result (vacuous Wedin control on 0% of layers); honest null baselines including a scan-informed oracle; and out-of-bag confirmation on six externally trained adapters (hit 0.997 under the bracketed edge, Mann-Whitney p = 0.82 against in-house adapters). Full per-layer data and code accompany the submission. The spike-budget rule, while not state-of-the-art mitigation, is a genuinely falsifiable downstream use with controls in both directions (β = 1.3 worsens forgetting; target-mode ablation worsens a safe model).

major comments (3)
  1. [§2.3, Eqs. (3)-(5)] The identifiability of the absolute scale θ̄ rests on the evaluation point z₀ = σ₁(1+ε), which the paper itself calls 'a choice rather than a derived quantity.' Eq. (3) has a pole at every empirical singular value, and Eq. (4) makes s* linear in θ̄(ε), so convention uncertainty transfers directly to the predicted threshold. The paper's defense is the plateau diagnostic R of Eq. (5), but the reported medians (2.2–2.4 causal models; 3.4–4.3 MoE/encoder-decoder) are comparable to, and for the latter families larger than, the factor-of-two tolerance of the primary hit@×2 endpoint. Gate G2 (median R < 10) rules out catastrophic pole domination but does not bound the endpoint at the accuracy the paper uses. Since all scan data are stored, this is directly answerable: please report hit@×2, ρ, and deployment AUC as functions of ε over the full declared range E (or at minimum at the endpoints and
  2. [Abstract; §3 'Calibration against nulls'] The abstract and §3 lead with hit@×2 = 0.822 as the headline localization result, but §3's own calibration shows a scan-informed oracle constant (per-adapter median empirical threshold) reaches 0.83, and the paper correctly concludes that 'localization within a factor of two is therefore the weakest output of the law.' The distinctive evidence is the a-priori scale, the ranking (ρ), the deployment AUC (0.89), the knee prediction, and the interventions — none of which the oracle can produce. As written, a reader of the abstract will take 82% as the primary evidence for the law when the manuscript itself demonstrates that this metric is non-discriminative against a trivial scan-dependent reference. Please reframe the headline (abstract and §3 opening) to foreground the discriminative endpoints, and state alongside the 82% figure that a scan-informed oracle matches it. This is a framing fix
  3. [§3 'Edge-variant analysis'; §4 'External validity'] The strongest localization numbers (hit@×2 = 0.98, ρ = 0.83, median ratio 1.06) come from the geometric-mean combination of the two pre-specified edges, which was 'chosen on our own adapters.' The out-of-bag confirmation on six external adapters (0.997) is meaningful mitigation, but n = 6 adapters (ranks 16–64, two QLoRA) is a small validation set for a claim stated at three significant figures, and the selection process for the external set is not described (were these all Hub adapters meeting some criterion, or a convenience sample?). Please (i) describe how the six external adapters were selected before scanning, (ii) report per-adapter CIs for the out-of-bag estimate, and (iii) state plainly in the abstract that the 98% figure uses a combination selected on the authors' own adapters, with the out-of-bag number as the unbiased estimate of its transfer.
minor comments (9)
  1. [§2.3 and §3] The exact value of ε used for the headline full-edge numbers is never stated; only the range E and the sensitivity ratio are given. Please state the declared ε (and whether it is per-layer or global) alongside the edge convention.
  2. [§3 heading] The section heading says 'Pre-registered validation' while the text clarifies no external registry was used ('pre-specified in this internal sense'). Consider retitling to 'Pre-specified validation' to avoid overstating the protocol.
  3. [§3; §4] The term 'pooled' is used in two senses: pooled hit@×2 averages per-adapter results (§3 definitions) while the external analysis reports 'pooled over 1,288 scans: 0.81/0.84.' Please disambiguate scan-pooled vs adapter-pooled values throughout.
  4. [§3 'The forgetting knee is where the law says it is'] The knee analysis (Fig. 2B, ρ = 0.86) correlates the perplexity knee with the median *empirical* threshold, so it validates the intruder–forgetting link rather than the a-priori scale directly. One sentence noting this, plus the knee against median *predicted* threshold, would close the loop.
  5. [Abstract; §4 'Spike-budgeted LoRA'] The abstract states the budget rule requires 'one SVD and no validation sweeps,' but §4 discloses 'one calibration constant from a prior run' (β) and the uniform-α control matched to the theory-set mean performs within noise on accuracy metrics. The abstract should reflect that the robust gain is the theory-set global scale, with per-layer allocation unresolved (16% perplexity advantage only).
  6. [Figures 1 and 3] Axis tick labels render as garbled powers of ten ('100 101') and several panels (e.g., Mistral r256, OLMo-2 r256, Mamba r256, OLMoE r256) appear to have truncated y-ranges; please check figure export. A shared legend for the factor-of-two band would also help.
  7. [Table 1] The low-ρ families (OLMo-2 0.33/0.39; Qwen 0.45–0.66 under the full edge) are diagnosed only in prose via the edge-variant analysis; adding the bulk-edge ρ alongside the full-edge ρ in Table 1 would make the bracketing claim visible at a glance.
  8. [Table 1] Median |log₂| errors are quantized to the scan grid (0.28) as noted, but Table 1's column would benefit from an explicit note that values 0.28/0.55 correspond to one/two grid steps, to avoid misreading them as continuous fit errors.
  9. [References] Several citations are very recent preprints (e.g., Koubbi et al. 2026, Yao et al. 2026, Park et al. 2026, Abro and Tahir 2026); please verify arXiv identifiers and versions, and clarify in the text that the contrast with Koubbi et al. concerns different constructs (update alignment vs. threshold position), as §5 already does well.

Circularity Check

2 steps flagged

Core s* law is non-circular (BGN on pretrained spectrum vs independent adapter scans); only mild selection/calibration on secondary claims.

specific steps
  1. fitted input called prediction [§3 Edge-variant analysis; Abstract (98% claim)]
    "The two variants bracket the effective edge, and their geometric mean, a parameter-free combination of two pre-specified estimators, is nearly unbiased: mean hit@×2 = 0.98 (13 of 18 adapters at 1.00), mean ρ = 0.83, median ratio 1.06. The combination was chosen on our own adapters; on the six external adapters, which played no role in that choice, it achieves out-of-bag hit@×2 = 0.997"

    The geometric-mean aggregator that produces the strongest localization number (98%) was selected by performance on the same adapter set used to report that number. This is a mild post-hoc choice among two pre-specified edges, not a fit of θ̄ itself; external OOB confirmation largely breaks the loop, and the primary pre-registered endpoint remains the full-edge 82.2%.

  2. fitted input called prediction [§4 Spike-budgeted LoRA]
    "cap the per-layer lora alpha so that the end-of-training spike sits at β θ̄_ℓ (PEFT alpha pattern; one calibration constant from a prior run; cap semantics, meaning the rule never raises a layer toward its threshold). On the most fragile model (Mistral r256 ...), β=0.7 reduces forgetting from +4.71 to +1.77"

    The control rule’s operating point β=0.7 is explicitly taken from a prior run rather than derived from the spectrum. This affects only the intervention application, not the claimed parameter-free emergence law s*; the paper does not present β as a first-principles prediction.

full rationale

The load-bearing derivation is not circular. The free-position critical strength s* = θ̄/(γ σ1(BA)) with θ̄ = D_μ(z0)^{-1/2} is the classical Benaych-Georges–Nadakuditi rectangular spiked-deformation fixed point evaluated on the measured spectrum μ of the pretrained W alone (Eqs. 3–4). Empirical s*_emp is obtained independently by scanning W+sγBA and applying Shuttleworth’s max-cosine intruder criterion; the two quantities are not definitionally linked. Intruder labels, the secular core-matrix instrument (Prop. 1), and the float64 secular check are likewise independent of D. ε and the full/bulk edge are declared conventions with a pre-specified degeneracy gate (R<10), not parameters fitted to s*_emp. Mild residual issues only: (i) the geometric-mean edge combination that yields the headline 98% was selected on the authors’ own adapters (still confirmed out-of-bag at 0.997 on six external adapters); (ii) the spike-budget application uses one calibration constant β from a prior run. Neither infects the primary parameter-free law or the pre-registered full-edge endpoints (hit@×2=0.822, AUC 0.89). No self-citation chain, uniqueness import, or self-definitional loop carries the central claim. Score 2 reflects those secondary selection/calibration steps only.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 2 invented entities

The load-bearing derivation imports classical rectangular spiked-deformation and secular-determinant identities, then adds free-position averaging and a declared edge evaluation to get a parameter-light predictor on real spectra. Almost all predictive content comes from those imports plus one SVD of W; the paper's own additions are the measurement arrangement, the free-position hypothesis under test, edge/ε conventions, and the empirical protocol. Free choices (ε, edge, τ, β, K) are acknowledged but marketed as 'no fitted parameters,' which is narrower than 'no free parameters.'

free parameters (5)
  • ε (edge evaluation offset) = range [1e-4, 3e-2]; median R≈2.2–2.4 on causal bases
    z0=σ1(1+ε); D has a pole at σ1 so ε>0 is required. Swept in E=[1e-4, 3e-2] with sensitivity ratio R; not fitted to intruder labels but still a declared choice that sets θ̄.
  • edge convention (full vs bulk vs geometric mean) = primary=full; combo geometric mean
    Full edge is primary and pre-specified; bulk trims isolated outliers; geometric mean of the two was chosen on own adapters then checked OOB. Changes hit@×2 from 0.82 to 0.98.
  • intruder criterion τ and top-k = τ=0.5, top-10
    Inherited from Shuttleworth (τ=0.5, top-10); robustness shown for τ∈[0.25,0.70] but the definition is conventional, not derived.
  • spike-budget β = β=0.7 (main result); dose at 1.3
    Cap end-of-training spike at β θ̄_ℓ; one calibration constant from a prior run. β=0.7 used for the −62% forgetting result.
  • alignment-correction subspace rank K = K∈{16,64,256}
    Intermediate rung uses σ1(P⊥_K BA P⊥_K) for K∈{16,64,256}; reported, not primary.
axioms (6)
  • standard math Rectangular spiked-deformation outlier condition D(λ)=1/θ² of Benaych-Georges & Nadakuditi (2012), applied to the empirical spectral measure of each real weight matrix rather than a limiting law.
    §2.2 Eq. (3)–(4); collapses the secular equation under free-position averages.
  • standard math Exact secular/determinant characterization of rank-one (and low-rank via core matrix) updates of the symmetric dilation of W (Bunch–Nielsen–Sorensen style identities).
    §2.1 Eq. (1)–(2) and Proposition 1; verified numerically to ~1e-3 relative error.
  • domain assumption Free-position hypothesis: spike directions have concentrated squared overlaps ~1/n, independent left/right signs so cross-term X→0, and null-space mass as in the aspect-ratio formula.
    §2.2; this is the hypothesis under test and is systematically violated by a stable early-arrival factor.
  • domain assumption Shuttleworth intruder definition: a top singular vector is an intruder if max_i |⟨yj, ui⟩| < τ against all pretrained singular vectors; intruders causally drive forgetting.
    Introduction and falsifier protocol; τ and causality imported from Shuttleworth et al. 2024.
  • domain assumption LoRA deployed update is well-modeled as sγBA with leading spike strength sγσ1(BA) for the purpose of the first-crossing threshold.
    §2 opening; multi-spike order statistics are a later correction in §5.
  • ad hoc to paper Evaluation slightly above the spectral edge is a valid surrogate for the critical θ̄ when the sensitivity ratio R is small (plateau).
    §2.3; gate G2 requires median R≤10; observed R~2–4, but the plateau is an empirical regularity of LLM spectra, not a theorem.
invented entities (2)
  • intruder threshold s* (per-layer critical update strength) independent evidence
    purpose: Scalar predicted from spectrum of W that marks onset of Shuttleworth intruders under free-position LoRA updates.
    Named operationalization of the BGN critical spike on measured LLM spectra; not a new physical object but the paper's central constructed predictor.
  • anti-alignment bias (early-arrival residual) independent evidence
    purpose: Name for the stable factor by which real adapters produce intruders earlier than the free-position prediction.
    Descriptive residual (§5), later attributed to measured energy profiles and multi-spike order statistics via the secular equation; falsifiable by adapter scans.

pith-pipeline@v1.2.0-grok45-kimik3 · 19901 in / 4586 out tokens · 88384 ms · 2026-07-30T15:08:32.021266+00:00 · methodology

0 comments
read the original abstract

LoRA fine-tuning can create intruder dimensions: new leading singular vectors of the updated weight matrix $W+BA$ that are nearly orthogonal to all pretrained singular vectors and that drive catastrophic forgetting. Since their discovery, no theory has predicted, layer by layer on measured spectra, when they appear. We derive a per-layer critical update strength $s^\ast=\bar\theta/(\gamma\sigma_1(BA))$, computed from the measured spectrum of $W$ alone through the rectangular spiked-deformation transform, together with an exact secular-equation characterization of the updated spectrum, with no fitted parameters. In a pre-specified study spanning four dense Transformer families, a state-space model, a mixture-of-experts model, and an encoder-decoder (18 adapters, 9{,}840 layer scans), the law localizes the empirical threshold within a factor of two on $82\%$ of layers, separates intruder-bearing from intruder-free layers at deployment with a mean AUC of $0.89$, holds unchanged on six third-party adapters, and predicts where WikiText-2 perplexity begins to degrade; a combination of the two pre-specified edge evaluations reaches $98\%$ and is confirmed out-of-bag on the external adapters ($0.997$). Full fine-tuning disperses its update far below the threshold of every layer, which resolves the asymmetry between LoRA and full fine-tuning. Norm-matched interventions confirm that threshold-crossing layers, rather than update magnitude, carry the forgetting, and a spike-budget rule derived from the thresholds, requiring one SVD and no validation sweeps, reduces forgetting by $62\%$ on the most fragile model at no task cost.

Figures

Figures reproduced from arXiv: 2607.23711 by Peng Xie.

Figure 1
Figure 1. Figure 1: Predicted versus empirical per-layer intruder threshold (log-log), one panel per adapter with its [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Forgetting begins where the spectrum says it will. Left: WikiText-2 perplexity relative to base [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Robustness to the intruder-definition threshold [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

15 extracted references · 13 linked inside Pith

  1. [1]

    Shuttleworth, J

    R. Shuttleworth, J. Collins, P. Hase, A. Raghunathan. LoRA vs Full Fine-tuning: An Illusion of Equivalence. arXiv:2410.21228; NeurIPS

    arXiv

  2. [8]

    Yang et al

    A. Yang et al. Qwen2.5 technical report. arXiv:2412.15115,

    Pith/arXiv arXiv

  3. [9]

    Grattafiori et al

    A. Grattafiori et al. The Llama 3 herd of models. arXiv:2407.21783,

    Pith/arXiv arXiv

  4. [10]

    A. Q. Jiang et al. Mistral 7B. arXiv:2310.06825,

    Pith/arXiv arXiv

  5. [11]

    2 OLMo 2 Furious

    Team OLMo et al. 2 OLMo 2 Furious. arXiv:2501.00656,

    Pith/arXiv arXiv

  6. [12]

    A. Gu, T. Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv:2312.00752; COLM

    Pith/arXiv arXiv

  7. [13]

    Muennighoff et al

    N. Muennighoff et al. OLMoE: Open mixture-of-experts language models. arXiv:2409.02060; ICLR

    Pith/arXiv arXiv

  8. [14]

    Cobbe et al

    K. Cobbe et al. Training verifiers to solve math word problems. arXiv:2110.14168,

    Pith/arXiv arXiv

  9. [15]

    H. Wang, Y. Li, S. Wang, G. Chen, Y. Chen. MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning. NAACL 2025; arXiv:2406.09044. C. H. Martin, M. W. Mahoney. Implicit self-regularization in deep neural networks: Evidence from random matrix theory and implications for learning. JMLR 22(165),

    Pith/arXiv arXiv 2025

  10. [1978]

    Spectral Phase Transitions and Trainability in Neural Network Learning Dynamics

    Park, Bocchi, D’Amico, Lucini, Aarts. Spectral Phase Transitions and Trainability in Neural Network Learning Dynamics. arXiv:2606.28486. F. Coeurdoux, G. Ferr´ e, J.-P. Bouchaud. Random Matrix Theory of Early-Stopped Gradient Flow: A Transient BBP Scenario. arXiv:2604.18450. Bocchi, Biroli, Cammarota, Ricci-Tersenghi. Discontinuous BBP Transitions. arXiv:...

    Pith/arXiv arXiv

  11. [2017]

    Y. Luo, Z. Yang, F. Meng, Y. Li, J. Zhou, Y. Zhang. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv:2308.08747,

    Pith/arXiv arXiv

  12. [2022]

    Koubbi, M

    H. Koubbi, M. Hernandez, Q. Boussard. Understanding Catastrophic Forgetting in LoRA via Mean-Field Attention Dynamics. arXiv:2402.15415 (v2, retitled; v1: The Impact of LoRA on the Emergence of Clusters in Transformers); ICML

    Pith/arXiv arXiv

  13. [2023]

    Kalajdzievski

    D. Kalajdzievski. Scaling laws for forgetting when fine-tuning large language models. arXiv:2401.05605,

    Pith/arXiv arXiv

  14. [2024]

    Xiong, X

    Y. Xiong, X. Xie. OPLoRA: Orthogonal Projection LoRA Prevents Catastrophic Forgetting. arXiv:2510.13003. D. Kalajdzievski. A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA. arXiv:2312.03732. A. Abro, M. Tahir. Spectral Unforgetting: Post-Hoc Recovery of Damaged Capabilities Without Retraining. arXiv:2605.20296. P. Wang, M. Gu, Q. Huang. VeFA:...

    arXiv

  15. [2026]

    H. Yao, X. Chen, A. Murtadha, G. Wang. GAIN: Multiplicative Modulation for Domain Adaptation. arXiv:2604.04516. D. Biderman et al. LoRA Learns Less and Forgets Less. TMLR

    Pith/arXiv arXiv