Pith. sign in

REVIEW 3 major objections 6 minor 54 references

Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning

T0 review · 3 major / 6 minor · reviewed 2026-07-30 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Frozen LLM embeddings plus tiny SVD adapters let models learn tasks in sequence with almost no forgetting and no learned router.

desk verdict Solid PEFT/CL systems paper: training-free GMM routing on frozen embeddings plus SVD latent adapters works, but the SOTA-with-fewer-params claim is undercut by an unmatched r=32 vs r=8 confound. read the letter →

arxiv 2607.23837 v1 pith:DHOWTB56 submitted 2026-07-26 cs.LG cs.CL

Reza Rahimi Azghan , Gautham Krishna Gudur , Giulia Pedrielli , Pavan Turaga , Hassan Ghasemzadeh This is my paper
classification cs.LGcs.CL
keywords continuallearningLoRAcatastrophicforgettinglatent-spaceadaptersGaussianmixtureroutingtask-agnosticinferenceparameter-efficientfine-tuningorthogonalregularization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large language models forget earlier skills when fine-tuned on new tasks one after another. The usual fix is to give each task its own small LoRA adapter, but at test time the system either needs to know which task it faces or must add every adapter together, so the wrong ones pollute the answer. Learned gates that pick the right adapter create a new forgetting problem of their own. This paper shows two facts that remove both problems. First, the model’s own frozen embedding layer already clusters different tasks cleanly enough that a simple Gaussian mixture, fitted once with no gradient training, can route each input to the right adapter. Second, each adapter can be reduced to a tiny square matrix inside the principal subspace of the pretrained weights; orthogonal penalties inside that subspace then keep adapters from interfering. The resulting system stores far fewer parameters per task, never replays old data, never trains a router, and reaches the best reported scores with near-zero forgetting on standard sequential NLP benchmarks across five model sizes.

What carries the argument

Latent-LoRA: each task trains only an r×r matrix inside the frozen SVD factors of the pretrained weight, while a shared-covariance GMM on mean-pooled frozen embeddings supplies soft task posteriors that blend the stored adapters.

What would settle it

Construct two sequential tasks whose mean-pooled embeddings heavily overlap, run the full Latent-LoRA pipeline, and check whether average performance collapses and forgetting rises well above the near-zero levels reported on SuperNI and Long Sequence.

Watch

Extended reading notes

Core claim

Pooled token embeddings from a frozen LLM embedding layer already separate task distributions well enough that a training-free Gaussian mixture model can select adapters at test time, and pairing that router with compact SVD-subspace adapters plus orthogonal regularization yields state-of-the-art continual learning with near-zero forgetting and no trainable routing parameters.

Load-bearing premise

Different tasks must stay separable enough in the frozen embedding space for the Gaussian mixture to put most of its probability on the correct adapter; if two tasks look almost the same to the embeddings, routing fails and forgetting can reappear.

Editorial extensions

If this is right

  • Continual LLM fine-tuning can drop learned gating modules entirely and still remain task-agnostic at inference.
  • Per-task storage shrinks from O(r(m+n)) to O(r²), a gap that widens automatically as base models grow.
  • Because the router never receives gradients, adding a new task cannot corrupt earlier routing decisions.
  • Soft blending plus the orthogonal bound still limits damage when the router is only partially confident.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same frozen-embedding GMM could serve as a cheap task detector for any adapter library, not only the SVD-compact ones used here.
  • If embedding separability holds at 70B+ scale, the method becomes a practical way to keep personal or domain adapters permanently isolated without replay buffers.
  • Tasks that truly share the same input distribution may force a design choice between merging them into one adapter or accepting soft interference as the residual cost.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Latent-LoRA, a continual learning system for LLMs combining (i) per-task adapters parameterized as a trainable r×r matrix R_t inside the frozen rank-r SVD subspace of each pretrained attention weight (following LoRA-XS), and (ii) a training-free router: a shared-covariance Gaussian mixture model fit on mean-pooled frozen input embeddings, whose posterior p(t|x) blends the stored adapter snapshots at inference. The authors derive an interference decomposition (Eq. 7) showing that cross-task output interference reduces to a quadratic form in the r×r matrices alone, and regularize the kernel R̃_iᵀR̃_j directly (Eq. 8), with a bound (Proposition 1) proved in Appendix A. Experiments on SuperNI and Long Sequence across five model scales (T5-Large/XL, Llama-2-7B, Llama-3-8B, Llama-2-13B) report state-of-the-art AP and near-zero FM against O-LoRA, InfLoRA, GainLoRA, and others, with 16–80× fewer per-task trainable parameters.

Significance. If the results hold at matched capacity, the contributions are real and useful. The training-free GMM router is a genuinely simple and appealing idea: it removes the learned gating module (and the second-order CL machinery GainLoRA needs to protect it), is incremental without replay, and the paper supports the underlying separability assumption with t-SNE and per-task posterior diagnostics across all five model scales (Figs. 2–5). The interference decomposition (Eq. 7) and Proposition 1 follow cleanly from the SVD parameterization — the regularization target is exactly the interference kernel, which is a nicer situation than O-LoRA's partial regularization (Appendix B.1 makes this contrast correctly). The parameter reduction in Table 8 is arithmetically correct and grows with model scale, and the evaluation spans two standard benchmarks, four orderings, five models, and three seeds. These are concrete, checkable strengths.

major comments (3)
  1. [§4.1 / §4.5 / Tables 1–4, 8] Rank confound in the headline comparison. All LoRA baselines use r=8 while Latent-LoRA uses r=32 (§4.1, C.3). The parameter-count claim in Table 8 is correct, but rank — not raw parameter count — is the primary capacity axis of a low-rank update: ΔW = U_r Σ_r R_t V_rᵀ has effective rank up to 32, i.e., 4× the baselines' update rank. The AP margins over GainLoRA (1.7–3.6 points, Tables 1–3) are of exactly the magnitude a 4× rank increase could plausibly produce, and the ablation in Table 4 does not isolate rank: it compares Latent-LoRA(r=32) against O-LoRA(r=8), so neither the SVD parameterization nor the Σ-weighted orthogonality term is shown to win at matched capacity. This is load-bearing for the claim of 'state-of-the-art performance with substantially fewer parameters': the two halves of that claim currently rest on different capacity regimes. The fix is well-defined and within scope
  2. [§4.2, Tables 1–3, Eq. (17)] The near-zero FM is largely architectural and carries almost no discriminative information. With frozen per-task snapshots (§3.2, 'Snapshot isolation') and a router whose posteriors are near one-hot (Fig. 3), a_{T,j} ≈ a_{j,j} by construction, so FM ≈ 0 follows from the system design rather than from anything the experiments test. The repeated identical value FM = 0.01 across models and orders in Tables 1–3 is consistent with this. The consequence is that the genuinely surprising part of the empirical claim is the AP advantage — which is precisely the part subject to the rank confound above. The paper would be strengthened by (i) stating explicitly that FM ≈ 0 is expected given snapshot isolation plus near-one-hot routing, and (ii) reporting the full accuracy matrix a_{i,j} (or at least per-task final accuracies) in an appendix, so readers can see where the AP gains come from and whether
  3. [§3.3, Eq. (13)] The shared-covariance assumption is the load-bearing statistical premise of the router, and its justification is thinner than the rest of the paper. A single pooled within-task covariance C (LDA-style) is a strong homoscedasticity assumption across 15 heterogeneous tasks; the empirical support is the t-SNE plots and posterior boxplots, which show separability but do not directly test whether a shared covariance is adequate versus per-task (or per-component) covariances. Given that mis-routing is one of only two forgetting channels the paper identifies (§3.2), an ablation comparing shared vs. per-task diagonal vs. per-task full covariance on routing posterior quality and downstream AP/FM would substantially de-risk the central mechanism. This is cheap to run (the router fits in seconds per task, per §4.5) and would convert the paper's weakest stated assumption into a tested one.
minor comments (6)
  1. [§4.1 / C.3] Results are averaged over three seeds but no standard deviations or error bars are reported in Tables 1–4. Given that some AP margins are ~1.7 points, seed-level variance matters for interpreting the comparisons; please add ± std to the main tables.
  2. [§3.3, Eq. (13)] The incremental update of C 'without requiring access to previous tasks' data' presumably requires storing per-task scatter matrices S_t or running sufficient statistics; please state the storage cost explicitly (it is O(T·d²) or O(d²) depending on the update form), since the paper emphasizes storage efficiency elsewhere.
  3. [§3.2, Proposition 1] The bound uses the spectral norm ‖R̃_iᵀR̃_j‖₂ while the regularizer (Eq. 8) penalizes the Frobenius norm. Since ‖·‖₂ ≤ ‖·‖_F the argument is valid, but a one-line remark noting this would save readers a step. Also, the bound is loose by the factor ‖ψ(h)‖²; a sentence on how tight it is in practice (e.g., typical ‖ψ(h)‖² values) would calibrate expectations.
  4. [§1, p. 2] Typo: 'reducing per-task storage to a smallr×r matrix' — missing space. Similar spacing artifacts appear in Tables 2–4 ('LatentLoRA54.03/0.01').
  5. [§4.5] The inference-cost discussion should note that soft blending (Eq. 15) requires materializing R(x) = Σ_t p(t|x)R_t per input (or per batch) at every target module; with near-one-hot posteriors this is one adapter, but the cost statement 'same overhead as a single LoRA forward pass' holds only in that regime — worth one clarifying sentence.
  6. [References] The self-citation Azghan et al. (2026) in Appendix B.2 is a preprint by the same group used to support the implicit-regularization argument; the argument stands on Biderman et al. (2024) and Aghajanyan et al. (2021) alone, so consider whether this citation is needed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: interference bound is algebraic, router is fit-then-evaluate, and SOTA claims rest on external benchmarks.

full rationale

The paper’s load-bearing chain does not collapse into its inputs. The latent-adapter interference identity (Eq. 7) and Proposition 1 are ordinary linear-algebra consequences of the frozen SVD parameterization ΔW_t = U_r Σ_r R_t V_r^⊤; regularizing ||R̃_i^⊤ R̃_j||_F is a designed loss targeting that derived kernel, not a redefinition of the reported AP/FM metrics. The GMM router is fit on each task’s training embeddings and assessed on held-out posteriors and downstream CL scores—standard train/test practice, not a tautology. Compact adapters cite external LoRA-XS/SVFT work as prior parameterization, not as a uniqueness theorem that forbids alternatives. Benchmarks (SuperNI, Long Sequence), metrics (AP/FM), and baselines are external. Minor related-work self-citation (Azghan et al., 2026) in Appendix B is not load-bearing. Rank-mismatch concerns affect experimental fairness, not circularity of the derivation.

Assumptions & free parameters 5 free parameters · 5 assumptions · 2 invented entities

The claim rests on standard PEFT/CL setup plus two modeling choices: (1) task structure is visible in frozen mean-pooled embeddings under a shared-covariance GMM, and (2) task updates live in the top-r SVD subspace of pretrained weights so that R̃i⊤R̃j controls interference. Free knobs (r, K, λ, ε, α) are set by convention or validation. No new physical entities; ‘latent-space adapter’ and ‘training-free GMM router’ are methodological constructs.

free parameters (5)
  • adapter rank r = 32 (method), 8 (baselines)
    Method uses r=32 while baselines use r=8; directly sets capacity and parameter count.
  • orthogonal penalty λ = 0.05 SuperNI; 0.02 Long Sequence
    Balances LM loss vs Σ-weighted ortho term; chosen from relative loss magnitudes on a T5-Large validation split.
  • GMM components K = 5
    Controls multi-modality of each task density; fixed without broad sweep reported.
  • covariance regularizer ε = 0.01
    Stabilizes shared covariance inversion in high-d embedding space.
  • LoRA alpha = 16
    Standard scaling hyperparameter for adapter updates.
assumptions (5)
  • domain assumption Fine-tuning updates are well approximated in the top-r left/right singular subspace of pretrained W (Eckart–Young / LoRA-XS assumption).
    Invoked in §3.2 to justify ΔW=Ur Σr R Vr⊤ and parameter reduction to r².
  • domain assumption Mean-pooled frozen input embeddings separate tasks enough for a shared-covariance GMM posterior to route adapters.
    Central to §3.3; illustrated by t-SNE and posterior plots; flagged again in Limitations.
  • domain assumption Task-agnostic, exemplar-free sequential supervised CL with no task ID at test time.
    Problem setting in §2.1 and Experiments; defines the evaluation regime.
  • standard math Orthonormality of thin SVD factors lets interference reduce to a quadratic form in the r×r weighted adapters.
    Used in Eq. 7 and Appendix A derivation of Proposition 1.
  • ad hoc to paper Shared pooled within-task covariance (LDA-style) is an adequate metric for embedding space across tasks.
    Eq. 13; chosen for incremental updates without storing old embeddings; not forced by theory.
invented entities (2)
  • Latent-LoRA system (compact SVD latent adapters + training-free GMM router)
    purpose: Name the full continual-learning method that freezes snapshots and blends R(x)=Σ p(t|x) Rt.
    Composite method name; not a new physical object. Independent evidence is only the paper’s own benchmarks.
  • Σ-weighted orthogonal regularizer on R̃i⊤R̃j
    purpose: Penalize the exact interference kernel identified in the latent parameterization.
    Variant of subspace ortho penalties specialized to LoRA-XS; evidence is ablation Table 4, not external measurement.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning." pith.science (2026). https://pith.science/paper/DHOWTB56

@misc{pith2026260723837,
  author       = {Pith},
  title        = {Pith review of: Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DHOWTB56}},
  note         = {Machine review of arXiv:2607.23837}
}
read the original abstract

Large language models generalize well to individual tasks but lack an inherent mechanism for learning them sequentially, leading to catastrophic forgetting. To mitigate this, LoRA-based continual learning methods allocate a separate low-rank adapter per task, yet existing approaches either require task identity at inference or sum all adapters indiscriminately, letting irrelevant branches distort the output. Recent gating-based solutions route inputs to the correct adapter but introduce trainable parameters that themselves need protection against forgetting. In this work, we observe that pooled token embeddings from a frozen LLM embedding layer already separate task distributions throughout the learning sequence. A Gaussian mixture model fitted on these embeddings, without any gradient-based training, is sufficient for task-agnostic adapter selection at test time. This eliminates the need for a learned gating module. On the adapter side, constraining each task's parameters to the principal subspace of the pretrained weights via SVD yields a compact latent-space parameterization. Within this subspace, orthogonal regularization directly controls inter-task interference. The resulting system, Latent-LoRA, is replay-free, requires no trainable routing component, and uses substantially fewer parameters per task. Experiments across five model scales and two established continual learning benchmarks show state-of-the-art performance with near-zero forgetting.

Figures

Figures reproduced from arXiv: 2607.23837 by the authors.

Figure 1
Figure 1. Latent-LoRA trains a per-task square matrix in the frozen SVD subspace and routes inputs via a training [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. t-SNE projection of pooled token embeddings from the frozen T5-Large embedding layer. Left: Long [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Router posterior probability of the correct task across test inputs for each task, evaluated on the [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: t-SNE projections of pooled token embeddings from frozen embedding layers. Left: Long Sequence. [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: Router posterior probability of the correct task across test inputs. Left: Long Sequence. Right: SuperNI. [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

54 extracted references · 2 canonical work pages

  1. [1]

    Psychology of Learning and Motivation , volume=

    Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem , author=. Psychology of Learning and Motivation , volume=. 1989 , publisher=

    1989

  2. [2]

    arXiv preprint arXiv:2404.16789 , year=

    Continual Learning of Large Language Models: A Comprehensive Survey , author=. arXiv preprint arXiv:2404.16789 , year=

    arXiv

  3. [3]

    arXiv preprint arXiv:2305.05968 , year=

    Investigating Forgetting in Pre-Trained Representations Through Continual Learning , author=. arXiv preprint arXiv:2305.05968 , year=

    arXiv

  4. [4]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Measuring Catastrophic Forgetting in Neural Networks , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  5. [5]

    International Conference on Learning Representations (ICLR) , year=

    LoRA: Low-Rank Adaptation of Large Language Models , author=. International Conference on Learning Representations (ICLR) , year=

  6. [6]

    Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    Orthogonal Subspace Learning for Language Model Continual Learning , author=. Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=. 2023 , publisher=

    2023

  7. [7]

    Advances in Neural Information Processing Systems (NeurIPS) , year=

    Gated Integration of Low-Rank Adaptation for Continual Learning of Large Language Models , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=

  8. [8]

    2025 , eprint=

    LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters , author=. 2025 , eprint=

    2025

Show all 54 references
  1. [9]

    SVFT: Parameter-Efficient Fine-Tuning with Singular Vectors , volume =

    Lingam, Vijay and Tejaswi, Atula and Vavre, Aditya and Shetty, Aneesh and Gudur, Gautham Krishna and Ghosh, Joydeep and Dimakis, Alex and Choi, Eunsol and Bojchevski, Aleksandar and Sanghavi, Sujay , booktitle =. SVFT: Parameter-Efficient Fine-Tuning with Singular Vectors , volume =

  2. [10]

    2023 , eprint=

    AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning , author=. 2023 , eprint=

    2023

  3. [11]

    Proceedings of the AAAI Conference on Artificial Intelligence , year=

    Continual Learning Using a Kernel-Based Method Over Foundation Models , author=. Proceedings of the AAAI Conference on Artificial Intelligence , year=

  4. [12]

    Goswami, Dipam and Soutif-Cormerais, Albin and Liu, Yuyang and Kamath, Sandesh and Tuytelaars, Tinne and Bethge, Matthias and van de Weijer, Joost , booktitle=

  5. [13]

    arXiv preprint arXiv:2103.09762 , year=

    Gradient Projection Memory for Continual Learning , author=. arXiv preprint arXiv:2103.09762 , year=

    arXiv

  6. [14]

    Super- N atural I nstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks

    Wang, Yizhong and Mishra, Swaroop and Alipoormolabashi, Pegah and Kordi, Yeganeh and Mirzaei, Amirreza and Naik, Atharva and Ashok, Arjun and Dhanasekaran, Arut Selvan and Arunkumar, Anjana and Stap, David and Pathak, Eshaan and Karamanolakis, Giannis and Lai, Haizhi and Puroh...

    2022

  7. [15]

    arXiv preprint arXiv:2301.12314 , year=

    Progressive prompts: Continual learning for language models , author=. arXiv preprint arXiv:2301.12314 , year=

    arXiv

  8. [16]

    Annals of Eugenics , volume=

    The use of multiple measurements in taxonomic problems , author=. Annals of Eugenics , volume=

  9. [17]

    2023 , eprint=

    Llama 2: Open Foundation and Fine-Tuned Chat Models , author=. 2023 , eprint=

    2023

  10. [18]

    2024 , eprint=

    The Llama 3 Herd of Models , author=. 2024 , eprint=

    2024

  11. [19]

    Journal of Machine Learning Research , volume=

    Visualizing Data using t-SNE , author=. Journal of Machine Learning Research , volume=

  12. [20]

    2024 , eprint=

    SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models , author=. 2024 , eprint=

    2024

  13. [21]

    and Milan, Kieran and Quan, John and Ramalho, Tiago and Grabska-Barwinska, Agnieszka and Hassabis, Demis and Clopath, Claudia and Kumaran, Dharshan and Hadsell, Raia , year=

    Kirkpatrick, James and Pascanu, Razvan and Rabinowitz, Neil and Veness, Joel and Desjardins, Guillaume and Rusu, Andrei A. and Milan, Kieran and Quan, John and Ramalho, Tiago and Grabska-Barwinska, Agnieszka and Hassabis, Demis and Clopath, Claudia and Kumaran, Dharshan and Ha...

    doi

  14. [22]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=

    Learning without forgetting , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=

  15. [23]

    International Conference on Learning Representations , year=

    LFPT5: A unified framework for lifelong few-shot language learning based on prompt tuning of T5 , author=. International Conference on Learning Representations , year=

  16. [24]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , year=

    InfLoRA: Interference-free low-rank adaptation for continual learning , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , year=

  17. [25]

    2025 , eprint=

    KIF: Knowledge Identification and Fusion for Language Model Continual Learning , author=. 2025 , eprint=

    2025

  18. [26]

    T a SL : Continual Dialog State Tracking via Task Skill Localization and Consolidation

    Feng, Yujie and Chu, Xu and Xu, Yongxin and Shi, Guangyuan and Liu, Bo and Wu, Xiao-Ming. T a SL : Continual Dialog State Tracking via Task Skill Localization and Consolidation. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: ...

    2024 doi

  19. [27]

    International Conference on Learning Representations , year=

    Decoupled weight decay regularization , author=. International Conference on Learning Representations , year=

  20. [28]

    Journal of Machine Learning Research , volume=

    Exploring the limits of transfer learning with a unified text-to-text transformer , author=. Journal of Machine Learning Research , volume=

  21. [29]

    Trends in Cognitive Sciences , volume=

    Catastrophic forgetting in connectionist networks , author=. Trends in Cognitive Sciences , volume=

  22. [30]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=

    A continual learning survey: Defying forgetting in classification tasks , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=

  23. [31]

    Advances in Neural Information Processing Systems , year=

    QLoRA: Efficient finetuning of quantized language models , author=. Advances in Neural Information Processing Systems , year=

  24. [32]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , year=

    A comprehensive survey of continual learning: Theory, method and application , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , year=

  25. [33]

    Neural Networks , volume=

    Continual lifelong learning with neural networks: A review , author=. Neural Networks , volume=

  26. [34]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Task-free continual learning , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  27. [35]

    Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics , pages=

    Intrinsic dimensionality explains the effectiveness of language model fine-tuning , author=. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics , pages=

  28. [36]

    2019 , eprint=

    Parameter-Efficient Transfer Learning for NLP , author=. 2019 , eprint=

    2019

  29. [37]

    Advances in Neural Information Processing Systems , year=

    Continual learning in low-rank orthogonal subspaces , author=. Advances in Neural Information Processing Systems , year=

  30. [38]

    Psychometrika , volume=

    The approximation of one matrix by another of lower rank , author=. Psychometrika , volume=

  31. [39]

    arXiv preprint arXiv:1312.6211 , year=

    An empirical investigation of catastrophic forgetting in gradient-based neural networks , author=. arXiv preprint arXiv:1312.6211 , year=

    arXiv

  32. [40]

    IEEE Transactions on Information Theory , volume=

    Least squares quantization in PCM , author=. IEEE Transactions on Information Theory , volume=

  33. [41]

    Encyclopedia of Biometrics , pages=

    Gaussian mixture models , author=. Encyclopedia of Biometrics , pages=. 2009 , publisher=

    2009

  34. [42]

    Advances in Neural Information Processing Systems , year=

    Distributed representations of words and phrases and their compositionality , author=. Advances in Neural Information Processing Systems , year=

  35. [43]

    International Conference on Learning Representations , year=

    Finetuned language models are zero-shot learners , author=. International Conference on Learning Representations , year=

  36. [44]

    Advances in Neural Information Processing Systems , year=

    Attention is all you need , author=. Advances in Neural Information Processing Systems , year=

  37. [45]

    Continual learning of context-dependent processing in neural networks , volume =

    Zeng, Guanxiong and Chen, Yang and Cui, Bo and Yu, Shan , year =. Continual learning of context-dependent processing in neural networks , volume =. Nature Machine Intelligence , doi =

  38. [46]

    arXiv preprint arXiv:2002.06305 , year=

    Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping , author=. arXiv preprint arXiv:2002.06305 , year=

    2002 arXiv

  39. [47]

    2013 , publisher=

    Matrix Computations , author=. 2013 , publisher=

    2013

  40. [48]

    Proceedings of the AAAI Conference on Artificial Intelligence , author=

    Continual Learning for Named Entity Recognition , volume=. Proceedings of the AAAI Conference on Artificial Intelligence , author=. 2021 , month=. doi:10.1609/aaai.v35i15.17600 , number=

    2021 doi

  41. [49]

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages=

    iCaRL: Incremental classifier and representation learning , author=. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages=

  42. [50]

    2024 , eprint=

    LoRA Learns Less and Forgets Less , author=. 2024 , eprint=

    2024

  43. [51]

    Advances in Neural Information Processing Systems , year=

    Dark experience for general continual learning: a strong, simple baseline , author=. Advances in Neural Information Processing Systems , year=

  44. [52]

    Learn more, but bother less: parameter efficient continual learning , url =

    Qiao, Fuli and Mahdavi, Mehrdad , booktitle =. Learn more, but bother less: parameter efficient continual learning , url =. doi:10.52202/079017-3092 , editor =

    doi

  45. [53]

    2026 , eprint=

    Gated Adaptation for Continual Learning in Human Activity Recognition , author=. 2026 , eprint=

    2026

  46. [54]

    ROUGE : A Package for Automatic Evaluation of Summaries

    Lin, Chin-Yew. ROUGE : A Package for Automatic Evaluation of Summaries. Text Summarization Branches Out. 2004

    2004

Pith tools

Reviewed July 30, 2026 · model on record in the stance chip above.