Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

CONEC-LoRA claims state-of-the-art on four continual-learning benchmarks by splitting a frozen ViT into shared and per-domain low-rank adapters, with a learned domain selector replacing replay.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

CONEC-LoRA reports state-of-the-art accuracy on four domain-incremental benchmarks by combining task-shared and task-specific LoRAs with a stochastic classifier and a learned domain-ID selector.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection Solid, well-protocoled DIL method with a genuinely new LoRA-sharing architecture, but the 'over 5% margins' claim only holds on one of four benchmarks and the shared-LoRA ablation doesn't support the story. the 2 major comments →

arxiv 2510.16077 v2 pith:UB3O3KAW submitted 2025-10-17 cs.LG cs.AI

Continual Knowledge Consolidation LORA for Domain Incremental Learning

Naeem Paeedeh , Mahardhika Pratama , Weiping Ding , Jimmy Cao , Wolfgang Mayer , Ryszard Kowalczyk , Ary Shiddiqi This is my paper
classification cs.LG cs.AI
keywords continual learningdomain incremental learninglow-rank adaptationparameter-efficient fine-tuningcatastrophic forgettingstochastic classifierGaussian mixture modelrehearsal-free learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a rehearsal-free domain-incremental learner can beat prompt- and prototype-based rivals on four established benchmarks by splitting a frozen ViT into two zones: early blocks get a shared low-rank adapter that accumulates knowledge common to all domains, late blocks get a per-domain adapter selected by an auxiliary network that predicts the domain of each test image. Training replaces the usual fixed classifier with a stochastic one whose weights are sampled from a learned distribution, then swaps in class prototypes at inference. If right, this shows that explicit per-domain parameter isolation, driven by a learned domain-ID predictor, is a stronger recipe for domain-incremental learning than prompt pools or prototype classifiers, all without storing past examples.

Core claim

The paper's central claim is that CONEC-LoRA, which couples task-shared LoRA in the first six ViT blocks with task-specific LoRA in the last six blocks, a stochastic classifier during training, and a GMM-based auxiliary domain classifier that selects the right task-specific LoRA at test time, achieves state-of-the-art average accuracy on DomainNet (66.79), CORe50 (88.88), CDDB-Hard (88.43), and Office-Home (86.29), outperforming prior rehearsal-free DIL methods by margins the abstract states as over 5%. The paper further claims the gap between its automatic domain selection and an oracle is below 1% on most benchmarks, meaning the learned selector is nearly as good as knowing the true domain

What carries the argument

Two-zone LoRA on a frozen ViT is the core: a shared adapter on the first six blocks (with a fixed random orthogonal down-projection) captures cross-domain knowledge, while per-domain adapters on the last six blocks isolate domain specifics. A stochastic classifier — weights drawn from learned mean and variance vectors — drives training, replaced by class prototypes at test time. A separate auxiliary network runs on the frozen backbone's intermediate embeddings, using a Gaussian-mixture generator with a ball-generator loss and per-layer early-exit classifiers, to predict the domain ID that chooses the task-specific LoRA. Gradient reweighting of the shared adapter by previous-domain norms impl

Load-bearing premise

The domain selector learns to recognise domains from the frozen backbone's intermediate embeddings before any LoRA is applied, yet at test time it has to pick a LoRA that alters those embeddings; the paper never checks that the no-LoRA domain signal stays reliable under that shift, so the selector could mis-route samples after the adapters change the feature geometry.

What would settle it

Take a trained CONEC-LoRA and, for a sample of each domain, compute the auxiliary network's domain predictions using (a) the frozen-backbone embeddings it was trained on and (b) the embeddings from the same backbone plus each candidate task-specific LoRA. If the top-1 domain-ID accuracy drops materially under (b) compared with (a) — or if samples from domain A become more often labelled as domain B once A's LoRA is inserted — the selection mechanism fails. A second check is to run the 'only task-specific LoRA' ablation on all four benchmarks; if that variant matches or exceeds the full model's

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the reported numbers hold, learning a domain-ID predictor from frozen-backbone embeddings is a viable alternative to prompt-key similarity matching for selecting task-specific parameters.
  • A stochastic classifier used only during training, with prototypes at inference, would be confirmed as a cheap way to improve accuracy in rehearsal-free continual learning.
  • The combination would establish that a single frozen pre-trained backbone can serve many domains without storing old samples, addressing both storage and privacy.
  • The small oracle gap reported on three of the four benchmarks implies the learned domain selector is nearly lossless, so the main remaining bottleneck is the quality of the per-domain adapters themselves.
  • The method's reported robustness to the λ1 and λ2 loss coefficients suggests the recipe is not finicky to tune.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's auxiliary domain classifier is trained on embeddings from the frozen backbone with no LoRA, but at inference it must select a LoRA that changes those embeddings. Until that distribution shift is measured, the selector's reported accuracy may not reflect its behaviour in the actual network; one could check this by recomputing domain-ID accuracy with each candidate LoRA applied.
  • The ablation on CDDB-Hard shows the variant with only task-specific LoRAs reaches 88.79 average accuracy versus 88.43 for the full model, so the claimed benefit of the shared LoRA is not consistently visible in that table; re-running the ablation on all four datasets would clarify whether the shared adapter helps or merely trades off average for last-domain accuracy.
  • The abstract's 'over 5%' margin claim is not uniformly supported by Table I: against the strongest rehearsal-free baselines the reported gaps are roughly 1.7% on Office-Home, 2.5% on DomainNet, 2.6% on CORe50, and 6.3% on CDDB-Hard. A careful reader should compare on matched backbones and seeds before treating the margin as a constant.
  • If the selector can be made robust to LoRA-induced activation shifts, this two-zone recipe transfers naturally to class-incremental learning or to mixture-of-experts selection generally, where a lightweight routing network decides which expert to run.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes CONEC-LoRA, a rehearsal-free domain-incremental learning (DIL) method that splits a ViT backbone into task-shared LoRA modules (first 6 blocks) and task-specific LoRA modules (last 6 blocks). During training it uses a stochastic classifier, knowledge distillation with gradient redistribution, and an auxiliary domain-classification network trained on frozen no-LoRA embeddings augmented by GMM-generated synthetic samples, a transformation module, and a ball-generator loss. Inference selects the task-specific LoRA by the most confident auxiliary local classifier, with early exit. The method is evaluated on DomainNet, CORe50, CDDB-Hard, and Office-Home over five domain orders, with ablations and robustness sweeps, and the authors claim margins of over 5% over prior art.

Significance. If the empirical results hold, CONEC-LoRA is a potentially useful contribution to rehearsal-free DIL: it combines parameter-efficient fine-tuning with a learned parameter-selection mechanism, reports results on four benchmarks over multiple domain orders, includes an oracle comparison, ablations, robustness analysis, and provides a public code link. The small oracle gap (<1%) suggests that the auxiliary selection mechanism is not currently the main accuracy bottleneck. However, the headline quantitative claim is overstated, and one ablation does not support the corresponding textual claim, so the paper needs revision before the contribution can be assessed at face value.

major comments (2)
  1. [Abstract; Section V-D; Conclusion; Table I] The abstract and conclusion claim that CONEC-LoRA outperforms prior art with 'over 5% margins'. Table I does not support this. Against the strongest reported baseline per dataset, the margins are: Office-Home 86.29 vs DCE 84.6 = 1.69 pts; DomainNet 66.79 vs DCE 64.3 = 2.49 pts; CORe50 88.88 vs EASE 86.30 = 2.58 pts (or 0.78 pts vs DualCP's 88.10); CDDB-Hard 88.43 vs DualCP 82.16 = 6.27 pts. Only CDDB-Hard exceeds 5%. The text also says CORe50 has an 'over 3% margin', which is contradicted by the 2.58-pt margin. Please correct these claims throughout the paper and report the actual best-baseline margins.
  2. [Section V-F, Table III] The ablation results do not support the conclusion that the shared LoRA component is beneficial. The row 'Only task-specific LoRAs' reports DIL Average 88.79±2.54, which is numerically higher than CONEC-LoRA's 88.43±2.21; only DIL Last drops (87.62±3.80 vs 88.21±0.88). The text states that removing the shared LoRA causes performance degradation and 'shows the CF problem', but this is at best true only for the last-accuracy metric, and the difference is within one standard deviation. Please re-examine the ablation, report per-order results or significance, and adjust the claim accordingly.
minor comments (5)
  1. [Section IV-C and Algorithm 3] The auxiliary domain classifier is trained and evaluated only on embeddings from the frozen backbone without LoRA, while at inference it selects LoRA modules for a network whose later activations are modified by the chosen LoRA. Table II does not directly measure whether no-LoRA embeddings remain predictive of the optimal LoRA choice. The <1% oracle gap in Table I is reassuring, but the paper should at least discuss this distribution-shift issue and, if feasible, report an experiment or oracle-gap analysis under the LoRA-modified features.
  2. [Section IV-B, Eq. (10)] The knowledge-distillation loss is written as a sum of s^τ_{b-1,i} log s^τ_{b,i} without an explicit minus sign. As written, minimizing this term would push the student distribution away from the teacher. Please define it as a proper cross-entropy (e.g., −Σ s^τ_b log s^τ_{b-1}) or clarify the sign convention.
  3. [Section IV-C, Eq. (14) and Algorithm 3] The confidence threshold is introduced as ς in Eq. (14) but Algorithm 3 calls it τ, which is also used for the KD temperature in Section IV-B. Please use consistent notation to avoid ambiguity.
  4. [Table II] The row labeled 'S-iPrompts' should likely be 'S-iPrompt'. Please correct the typo and check that the cited source matches the reported numbers.
  5. [Throughout] There are several typographical and formatting issues: 'LORA' vs 'LoRA' inconsistently, 'with over5%margins' missing spaces, and 'over3%margin' in Section V-D. A careful proofread is needed.

Circularity Check

0 steps flagged

No significant circularity: CONEC-LoRA is an empirical benchmark contribution with no prediction forced by construction.

full rationale

The paper is a method-construction plus empirical-evaluation contribution rather than a formal derivation whose output is baked into its input. The task-shared/task-specific LoRA split (Eq. 6), stochastic classifier (Eq. 5), distillation loss (Eqs. 9-11), GMM-based auxiliary domain classifier (Eqs. 12-20), and inference protocol (Algorithm 3) are all specified as trainable components and evaluated on external benchmarks (Table I). The auxiliary domain classifier is trained on frozen-backbone embeddings and GMM-synthetic samples, then evaluated on held-out test embeddings; this is a self-contained training loop, not a fitted parameter renamed as a prediction. The oracle comparison is a separate diagnostic, not an optimization target. Self-citations [42] and [51] supply the stochastic-classifier and ball-generator-loss ideas, but they are cited transparently as prior components, are integrated into a new DIL architecture, and their contribution is checked by ablation in Table III; no load-bearing claim rests on an unverified self-citation or an imported uniqueness theorem. The one notable manuscript-level discrepancy is the Abstract/Conclusion claim of "over 5% margins," which is not supported by the paper's own Table I: best margins are roughly 1.69 points on Office-Home, 2.49 on DomainNet, 2.58 on CORe50, and 6.27 on CDDB-Hard, so only CDDB-Hard exceeds 5%. This is an evidentiary/overstatement issue, not circularity, and does not raise the circularity score.

Axiom & Free-Parameter Ledger

9 free parameters · 6 axioms · 1 invented entities

The central claims rest on multiple hand-set hyperparameters and on heuristic assumptions about ViT layer specialization, GMM fidelity, and the transferability of no-LoRA embeddings to LoRA-augmented inference. The paper provides no derivation connecting these choices to the reported accuracies.

free parameters (9)
  • l (split point between shared and task-specific LoRA blocks) = 6 of 12 ViT blocks
    Set by hand in Section V-C; no sweep or ablation, yet defines which layers get shared vs per-domain adapters.
  • lambda_1 (KD loss weight) = 5
    Section V-C; sensitivity only shown on CDDB (3-7), not across datasets.
  • lambda_2 (ball-generator loss weight) = 2
    Section V-C; sensitivity only shown on CDDB (1-3).
  • LoRA rank r = 8
    Section V-C; common PEFT default, not swept.
  • ball margin r = 1
    Eq. (19); chosen, no sensitivity analysis.
  • early-exit confidence threshold ς = 0.9
    Eq. (14); chosen, no sensitivity analysis.
  • KD temperature tau = 2
    Eq. (10); chosen, not swept.
  • GMM components C per domain = 2
    Section IV-D says 'usually small (2 in our experiments)'; determines synthetic-sample fidelity.
  • learning rates = 0.02 LoRA/temp classifier, 2e-3 domain classifiers, 1e-4 transformation modules
    Section V-C; chosen, no sensitivity analysis.
axioms (6)
  • domain assumption Early ViT layers encode domain-shared features and later layers encode domain-specific features, justifying a fixed split at block l=6.
    Section IV-A states this and uses it to place shared vs task-specific LoRA; no data-driven validation of the split point.
  • domain assumption Intermediate embeddings of the frozen backbone without LoRA are informative for domain ID and remain reliable after LoRA adaptation.
    Section IV-C trains domain classifiers on frozen no-LoRA embeddings and uses them to select LoRAs; distribution mismatch is not analyzed.
  • domain assumption A Gaussian mixture with C=2 components per domain adequately models previous-domain embedding distributions, and synthetic samples from it improve domain classification.
    Section IV-C equations (15)-(18); synthetic-sample bias is acknowledged and mitigated only heuristically by the transformation module and ball loss.
  • domain assumption Stochastic classifiers trained with sampled weights transfer to prototype-based inference after replacing mu with class prototypes.
    Section IV-A states this replacement improves inference; no theoretical justification is offered.
  • standard math EM algorithm for GMM converges and gives useful density estimates on high-dimensional ViT embeddings.
    Section IV-C invokes EM [50]; standard, but operates on d-dimensional embeddings with full covariances.
  • domain assumption ImageNet-pretrained ViT-B/16 features are reusable across the four DIL benchmarks without continued backbone training.
    Standard PEFT-CL premise, used in all experiments.
invented entities (1)
  • Transformation module kappa_gamma (two-layer MLP) no independent evidence
    purpose: Maps raw and GMM-synthetic embeddings before domain classification to reduce synthetic-sample bias.
    Introduced in Section IV-C; no ablation isolates its contribution (only the ball-generator loss is ablated), so its stated benefit lacks direct support in the paper.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Continual Knowledge Consolidation LORA for Domain Incremental Learning." pith.science (2026). https://pith.science/paper/UB3O3KAW

@misc{pith2026251016077,
  author       = {Pith},
  title        = {Pith review of: Continual Knowledge Consolidation LORA for Domain Incremental Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UB3O3KAW}},
  note         = {Machine review of arXiv:2510.16077}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Domain Incremental Learning (DIL) is a sub-branch of continual learning that aims to address the never-ending arrival of new domains without catastrophic forgetting. Despite the advent of parameter-efficient fine-tuning (PEFT) approaches, prior works create task-specific LoRAs that overlook shared knowledge across tasks. Inaccurate selection of task-specific LoRAs during inference leads to significant drops in accuracy, while existing works rely on linear or prototype-based classifiers, which have suboptimal generalization powers. Our paper proposes continual knowledge consolidation low-rank adaptation (CONEC-LoRA) addressing the DIL problems. CONEC-LoRA is developed from consolidations between task-shared LORA to extract common knowledge and task-specific LORA to incorporate domain-specific knowledge. Unlike existing approaches, CONEC-LoRA integrates the concept of a stochastic classifier whose parameters are sampled from a distribution, thus enhancing the likelihood of correct classifications. Last but not least, an auxiliary network is deployed to optimally predict the task-specific LoRAs for inference and implements the concept of a different-depth network structure in which every layer is connected with a local classifier to leverage intermediate representations. This module integrates the ball-generator loss and transformation module to address the synthetic sample bias problem. Our rigorous experiments demonstrate the advantage of CONEC-LoRA over prior art on 4 popular benchmark problems with over 5% margins.

Figures

Figures reproduced from arXiv: 2510.16077 by Ary Shiddiqi, Jimmy Cao, Mahardhika Pratama, Naeem Paeedeh, Ryszard Kowalczyk, Weiping Ding, Wolfgang Mayer.

Figure 1
Figure 1. Figure 1: A stochastic classifier. a) The decision boundary for a single cosine classifier with two weight vectors for two classes. [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The architecture of the model and pipeline of CONEC-LoRA. CONEC-LoRA consists of two main stages. First, the [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: UMAP plots on the embedding space of the model [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Few-Shot Domain Incremental Learning via Continual Vision-Language Consolidation

    cs.CV 2026-06 unverdicted novelty 6.0

    CVLC fuses calibrated vision prototypes with LLM-generated language prototypes and applies dual coalescent projection plus latent space reservation to enable few-shot adaptation across sequential domains, reporting up...

Reference graph

Works this paper leans on

76 extracted references · 19 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Chen and B

    Z. Chen and B. Liu,Lifelong machine learning. Springer, 2018, vol. 1

    2018

  2. [2]

    Continual lifelong learning with neural networks: A review,

    G. I. Parisi, R. Kemker, J. L. Part, C. Kanan, and S. Wermter, “Continual lifelong learning with neural networks: A review,” Neural networks : the official journal of the International Neural Network Society, vol. 113, pp. 54–71, 2018. [Online]. Available: https://api.semanticscholar.org/CorpusID:73497737

    2018

  3. [3]

    Class-incremental learning: Survey and performance evaluation on image classification,

    M. Masana, X. Liu, B. Twardowski, M. Menta, A. D. Bagdanov, and J. van de Weijer, “Class-incremental learning: Survey and performance evaluation on image classification,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, pp. 5513–5533,

  4. [4]

    Class-incremental learning: A survey

    D.-W. Zhou, Q. Wang, Z. Qi, H.-J. Ye, D. chuan Zhan, and Z. Liu, “Class-incremental learning: A survey.”IEEE transactions on pattern analysis and machine intelligence, vol. PP, 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID:256627357

    2023

  5. [5]

    On the stability-plasticity dilemma of class- incremental learning,

    D. Kim and B. Han, “On the stability-plasticity dilemma of class- incremental learning,”2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 20 196–20 204, 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID:257921372

    2023

  6. [6]

    Overcoming catastrophic forgetting in neural networks,

    J. Kirkpatrick, R. Pascanu, N. C. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, D. Hassabis, C. Clopath, D. Kumaran, and R. Hadsell, “Overcoming catastrophic forgetting in neural networks,”Proceedings of the National Academy of Sciences, vol. 114, pp. 3521 – 3526, 2016. [Online]. Available: https://...

    2016

  7. [7]

    Continual learning via inter-task synaptic mapping,

    F. Mao, W. Weng, M. Pratama, and E. K. Y . Yapp, “Continual learning via inter-task synaptic mapping,”ArXiv, vol. abs/2106.13954,

    Pith/arXiv arXiv

  8. [8]

    Overcoming catastrophic forgetting by neuron-level plasticity control,

    I. Paik, S. Oh, T. Kwak, and I. Kim, “Overcoming catastrophic forgetting by neuron-level plasticity control,” inAAAI Conference on Artificial Intelligence, 2019. [Online]. Available: https://api.semanticscholar.org/ CorpusID:199001153

    2019

  9. [9]

    Continual learning through synaptic intelligence,

    F. Zenke, B. Poole, and S. Ganguli, “Continual learning through synaptic intelligence,”Proceedings of machine learning research, vol. 70, pp. 3987–3995, 2017. [Online]. Available: https://api.semanticscholar.org/ CorpusID:10409742

    2017

  10. [10]

    Memory aware synapses: Learning what (not) to forget,

    R. Aljundi, F. Babiloni, M. Elhoseiny, M. Rohrbach, and T. Tuytelaars, “Memory aware synapses: Learning what (not) to forget,” pp. 139–154,

  11. [11]

    Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting,

    X. Li, Y . Zhou, T. Wu, R. Socher, and C. Xiong, “Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting,”ArXiv, vol. abs/1904.00310, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:90259576

    Pith/arXiv arXiv 1904

  12. [12]

    Progress & compress: A scalable framework for continual learning,

    J. Schwarz, W. M. Czarnecki, J. Luketina, A. Grabska-Barwinska, Y . W. Teh, R. Pascanu, and R. Hadsell, “Progress & compress: A scalable framework for continual learning,”ArXiv, vol. abs/1805.06370, 2018. [Online]. Available: https://api.semanticscholar.org/CorpusID:21718339

    Pith/arXiv arXiv 2018

  13. [13]

    Cpr: Classifier-projection regularization for continual learning,

    S. Cha, H. Hsu, F. du Pin Calmon, and T. Moon, “Cpr: Classifier-projection regularization for continual learning,”ArXiv, vol. abs/2006.07326, 2020. [Online]. Available: https://api.semanticscholar. org/CorpusID:219636462

    Pith/arXiv arXiv 2006

  14. [14]

    Efficient lifelong learning with a-gem,

    A. Chaudhry, M. Ranzato, M. Rohrbach, and M. Elhoseiny, “Efficient lifelong learning with a-gem,”ArXiv, vol. abs/1812.00420, 2018. [Online]. Available: https://api.semanticscholar.org/CorpusID:54443381

    Pith/arXiv arXiv 2018

  15. [15]

    icarl: Incremental classifier and representation learning,

    S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert, “icarl: Incremental classifier and representation learning,”2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5533–5542, 2016. [Online]. Available: https://api.semanticscholar.org/ CorpusID:206596260

    2017

  16. [16]

    On tiny episodic memories in continual learning,

    A. Chaudhry, M. Rohrbach, M. Elhoseiny, T. Ajanthan, P. K. Dokania, P. H. S. Torr, and M. Ranzato, “On tiny episodic memories in continual learning,”arXiv: Learning, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:173188188

    2019

  17. [17]

    Using hindsight to anchor past knowledge in continual learning,

    A. Chaudhry, A. Gordo, P. K. Dokania, P. H. S. Torr, and D. Lopez-Paz, “Using hindsight to anchor past knowledge in continual learning,” in AAAI Conference on Artificial Intelligence, 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:210957697

    2021

  18. [18]

    Dark experience for general continual learning: a strong, simple baseline,

    P. Buzzega, M. Boschini, A. Porrello, D. Abati, and S. Calderara, “Dark experience for general continual learning: a strong, simple baseline,”ArXiv, vol. abs/2004.07211, 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:215768806

    Pith/arXiv arXiv 2004

  19. [19]

    Scalable adversarial online continual learning,

    T. Dam, M. Pratama, M. M. Ferdaus, S. G. Anavatti, and H. Abbas, “Scalable adversarial online continual learning,” inECML/PKDD,

  20. [20]

    Class- incremental learning via knowledge amalgamation,

    M. V . de Carvalho, M. Pratama, J. Zhang, and Y . San, “Class- incremental learning via knowledge amalgamation,” inECML/PKDD,

  21. [21]

    Assessor-guided learning for continual environments,

    M. A. Ma’sum, M. Pratama, E. D. Lughofer, W. Ding, and W. Jatmiko, “Assessor-guided learning for continual environments,” Inf. Sci., vol. 640, p. 119088, 2023. [Online]. Available: https: //api.semanticscholar.org/CorpusID:257636622

    2023

  22. [22]

    Gradient episodic memory for continual learning,

    D. Lopez-Paz and M. Ranzato, “Gradient episodic memory for continual learning,”Advances in neural information processing systems, vol. 30,

  23. [23]

    Continual learning with deep generative replay,

    H. Shin, J. K. Lee, J. Kim, and J. Kim, “Continual learning with deep generative replay,” inNeural Information Processing Systems, 2017. [Online]. Available: https://api.semanticscholar.org/CorpusID:1888776

    2017

  24. [24]

    Adaptive progressive continual learning,

    J. Xu, J. Ma, X. Gao, and Z. Zhu, “Adaptive progressive continual learning,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, pp. 6715–6728, 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:235767757

    2021

  25. [25]

    Available: https://api.semanticscholar.org/CorpusID: 252090326

    [Online]. Available: https://api.semanticscholar.org/CorpusID: 252090326

  26. [26]

    Unsupervised continual learning in streaming environments,

    A. Ashfahani and M. Pratama, “Unsupervised continual learning in streaming environments,”IEEE Transactions on Neural Networks and Learning Systems, vol. 34, pp. 9992–10 003, 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:237572299

    2021

  27. [27]

    Unsupervised continual learning via self-adaptive deep clustering approach,

    M. Pratama, A. Ashfahani, and E. D. Lughofer, “Unsupervised continual learning via self-adaptive deep clustering approach,” inCSSL,

  28. [28]

    Reinforced continual learning for graphs,

    A. Rakaraddi, S.-K. Lam, M. Pratama, and M. V . de Carvalho, “Reinforced continual learning for graphs,”Proceedings of the 31st ACM International Conference on Information & Knowledge Management,

  29. [29]

    Learning to prompt for continual learning,

    Z. Wang, Z. Zhang, C.-Y . Lee, H. Zhang, R. Sun, X. Ren, G. Su, V . Perot, J. G. Dy, and T. Pfister, “Learning to prompt for continual learning,”2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 139–149, 2022. [Online]. Available: https://api.semanticscholar.org/CorpusID:245218925

    2022

  30. [30]

    Dualprompt: Complementary prompting for rehearsal-free continual learning,

    Z. Wang, Z. Zhang, S. Ebrahimi, R. Sun, H. Zhang, C.-Y . Lee, X. Ren, G. Su, V . Perot, J. G. Dy, and T. Pfister, “Dualprompt: Complementary prompting for rehearsal-free continual learning,”ArXiv, p. 631–648,

  31. [31]

    Lifelong learning with dynamically expandable networks,

    J. Yoon, E. Yang, J. Lee, and S. J. Hwang, “Lifelong learning with dynamically expandable networks,”ArXiv, vol. abs/1708.01547, 2017. [Online]. Available: https://api.semanticscholar.org/CorpusID:3693512

    Pith/arXiv arXiv 2017

  32. [32]

    Continual learning using a kernel-based method over foundation models,

    S. Momeni, S. Mazumder, and B. Liu, “Continual learning using a kernel-based method over foundation models,” inAAAI Conference on Artificial Intelligence, 2024. [Online]. Available: https://api. semanticscholar.org/CorpusID:274965301

    2024

  33. [33]

    Continual learning with pre-trained models: A survey,

    D.-W. Zhou, H.-L. Sun, J. Ning, H.-J. Ye, and D. chuan Zhan, “Continual learning with pre-trained models: A survey,” inInternational Joint Conference on Artificial Intelligence, 2024. [Online]. Available: https://api.semanticscholar.org/CorpusID:267312447

    2024

  34. [34]

    Available: https://api.semanticscholar.org/CorpusID: 235658071

    [Online]. Available: https://api.semanticscholar.org/CorpusID: 235658071

  35. [35]

    Lora subtraction for drift-resistant space in exemplar-free continual learning,

    X. Liu and X. Chang, “Lora subtraction for drift-resistant space in exemplar-free continual learning,”ArXiv, vol. abs/2503.18985,

    Pith/arXiv arXiv

  36. [36]

    Available: https://api.semanticscholar.org/CorpusID: 252089650

    [Online]. Available: https://api.semanticscholar.org/CorpusID: 252089650

  37. [37]

    Three scenarios for continual learning,

    G. M. van de Ven and A. S. Tolias, “Three scenarios for continual learning,”ArXiv, vol. abs/1904.07734, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:119309522

    Pith/arXiv arXiv 1904

  38. [38]

    Compositional prompting for anti-forgetting in domain incremental learning,

    Z. Liu, Y . Peng, and J. Zhou, “Compositional prompting for anti-forgetting in domain incremental learning,”Int. J. Comput. Vis., vol. 132, pp. 5783–5800, 2024. [Online]. Available: https: //api.semanticscholar.org/CorpusID:270778693

    2024

  39. [39]

    Available: https://api.semanticscholar.org/CorpusID: 248085201

    [Online]. Available: https://api.semanticscholar.org/CorpusID: 248085201

  40. [40]

    Vision and language synergy for rehearsal free continual learning,

    M. A. Ma’sum, M. Pratama, S. Ramasamy, L. Liu, Habibullah, and R. Kowalczyk, “Vision and language synergy for rehearsal free continual learning,” inInternational Conference on Learning Representations, JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 12

    2015

  41. [41]

    Dual consolidation for pre-trained model-based domain-incremental learning,

    D.-W. Zhou, Z.-W. Cai, H.-J. Ye, L. Zhang, and D.-C. Zhan, “Dual consolidation for pre-trained model-based domain-incremental learning,”ArXiv, vol. abs/2410.00911, 2024. [Online]. Available: https://api.semanticscholar.org/CorpusID:273022628

    Pith/arXiv arXiv 2024

  42. [42]

    Few-shot class incremental learning via robust transformer approach,

    N. Paeedeh, M. Pratama, S. Wibirama, W. Mayer, Z. Cao, and R. Kowalczyk, “Few-shot class incremental learning via robust transformer approach,”Inf. Sci., vol. 675, p. 120751, 2024. [Online]. Available: https://api.semanticscholar.org/CorpusID:269741067

    2024

  43. [43]

    Ad- dressing imbalanced domain-incremental learning through dual- balance collaborative experts,

    L. Li, D.-W. Zhou, H.-J. Ye, and D.-C. Zhan, “Ad- dressing imbalanced domain-incremental learning through dual- balance collaborative experts,” inForty-second International Conference on Machine Learning, 2025. [Online]. Available: https://openreview.net/forum?id=dwjwvTwV3V

    2025

  44. [44]

    Cl-lora: Continual low-rank adaptation for rehearsal-free class-incremental learning,

    J. He, Z. Duan, and F. M. Zhu, “Cl-lora: Continual low-rank adaptation for rehearsal-free class-incremental learning,”ArXiv, vol. abs/2505.24816, 2025. [Online]. Available: https://api.semanticscholar. org/CorpusID:279070575

    Pith/arXiv arXiv 2025

  45. [45]

    Lora: Low-rank adaptation of large language models,

    J. E. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,”ArXiv, vol. abs/2106.09685, 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:235458009

    Pith/arXiv arXiv 2021

  46. [46]

    Available: https://api.semanticscholar.org/CorpusID: 277313510

    [Online]. Available: https://api.semanticscholar.org/CorpusID: 277313510

  47. [47]

    Adapter merging with centroid prototype mapping for scalable class-incremental learning,

    T. Fukuda, H. Kera, and K. Kawamoto, “Adapter merging with centroid prototype mapping for scalable class-incremental learning,”ArXiv, vol. abs/2412.18219, 2024. [Online]. Available: https://api.semanticscholar. org/CorpusID:274992202

    Pith/arXiv arXiv 2024

  48. [48]

    Calculating the singular values and pseudo- inverse of a matrix,

    G. Golub and W. Kahan, “Calculating the singular values and pseudo- inverse of a matrix,”Journal of the Society for Industrial and Applied Mathematics, Series B: Numerical Analysis, vol. 2, no. 2, pp. 205–224, 1965

    1965

  49. [49]

    Improving continual learning performance and efficiency with auxiliary classifiers,

    F. Szatkowski, Y . Zheng, F. Yang, B. Twardowski, T. Trzci’nski, and J. van de Weijer, “Improving continual learning performance and efficiency with auxiliary classifiers,” 2024. [Online]. Available: https://api.semanticscholar.org/CorpusID:268363428

    2024

  50. [50]

    Dualcp: Rehearsal-free domain-incremental learning via dual-level concept prototype,

    Q. Wang, Y . He, S. Dong, X. Song, J. Han, H. Luo, and Y . Gong, “Dualcp: Rehearsal-free domain-incremental learning via dual-level concept prototype,” inAAAI Conference on Artificial Intelligence,

  51. [51]

    Available: https://api.semanticscholar.org/CorpusID: 277272564

    [Online]. Available: https://api.semanticscholar.org/CorpusID: 277272564

  52. [52]

    Boosting domain incremental learning: Selecting the optimal parameters is all you need,

    Q. Wang, X. Song, Y . He, J. Han, C. Ding, X. Gao, and Y . Gong, “Boosting domain incremental learning: Selecting the optimal parameters is all you need,”ArXiv, vol. abs/2505.23744, 2025. [Online]. Available: https://api.semanticscholar.org/CorpusID:278996449

    Pith/arXiv arXiv 2025

  53. [53]

    Core50: a new dataset and benchmark for continuous object recognition,

    V . Lomonaco and D. Maltoni, “Core50: a new dataset and benchmark for continuous object recognition,” inConference on robot learning. PMLR, 2017, pp. 17–26

    2017

  54. [54]

    A continual deepfake detection benchmark: Dataset, methods, and essentials,

    C. Li, Z. Huang, D. P. Paudel, Y . Wang, M. Shahbazi, X. Hong, and L. Van Gool, “A continual deepfake detection benchmark: Dataset, methods, and essentials,” inProceedings of the IEEE/CVF winter conference on applications of computer vision, 2023, pp. 1339–1349

    2023

  55. [55]

    Deep hashing network for unsupervised domain adaptation,

    H. Venkateswara, J. Eus ´ebio, S. Chakraborty, and S. Panchanathan, “Deep hashing network for unsupervised domain adaptation,”2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5385–5394, 2017. [Online]. Available: https://api.semanticscholar. org/CorpusID:2928248

    2017

  56. [56]

    Componential prompt- knowledge alignment for domain incremental learning,

    K. Xu, X. Zou, G. Hua, and J. Zhou, “Componential prompt- knowledge alignment for domain incremental learning,” inForty-second International Conference on Machine Learning, 2025. [Online]. Available: https://openreview.net/forum?id=QWCdBzLOsk

    2025

  57. [57]

    Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning,

    J. Smith, L. Karlinsky, V . Gutta, P. Cascante-Bonilla, D. Kim, A. Arbelle, R. Panda, R. S. Feris, and Z. Kira, “Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning,”2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11 909–11 919, 2022. [Online]. Available: https://api.semanticsch...

    2023

  58. [58]

    S3c: Self-supervised stochastic classifiers for few-shot class-incremental learning,

    J. Kalla and S. Biswas, “S3c: Self-supervised stochastic classifiers for few-shot class-incremental learning,” inEuropean Conference on Computer Vision, 2022

    2022

  59. [59]

    Asymmetry in low-rank adapters of foundation models,

    J. Zhu, K. H. Greenewald, K. Nadjahi, H. S. de Oc’ariz Borde, R. B. Gabrielsson, L. Choshen, M. Ghassemi, M. Yurochkin, and J. Solomon, “Asymmetry in low-rank adapters of foundation models,”ArXiv, vol. abs/2402.16842, 2024. [Online]. Available: https://api.semanticscholar.org/CorpusID:268033026

    Pith/arXiv arXiv 2024

  60. [60]

    Revisiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need,

    D.-W. Zhou, Z.-W. Cai, H.-J. Ye, D.-C. Zhan, and Z. Liu, “Revisiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need,”International Journal of Computer Vision, vol. 133, no. 3, pp. 1012–1032, 2025

    2025

  61. [61]

    A model or 603 exemplars: Towards memory-efficient class-incremental learning,

    D.-W. Zhou, Q.-W. Wang, H.-J. Ye, and D.-C. Zhan, “A model or 603 exemplars: Towards memory-efficient class-incremental learning,”arXiv preprint arXiv:2205.13218, 2022

    Pith/arXiv arXiv 2022

  62. [62]

    C. M. Bishop and N. M. Nasrabadi,Pattern recognition and machine learning. Springer, 2006, vol. 4, no. 4

    2006

  63. [63]

    Few-shot continual learning via flat-to-wide approaches,

    M. A. Ma’sum, M. Pratama, L. Liu, E. D. Lughofer, Habibullah, and R. Kowalczyk, “Few-shot continual learning via flat-to-wide approaches,”IEEE Transactions on Neural Networks and Learning Systems, vol. 36, pp. 8966–8978, 2023. [Online]. Available: https: //api.semanticscholar.org/CorpusID:259251661

    2023

  64. [64]

    Moment matching for multi-source domain adaptation,

    X. Peng, Q. Bai, X. Xia, Z. Huang, K. Saenko, and B. Wang, “Moment matching for multi-source domain adaptation,” inProceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 1406– 1415

    2019

  65. [68]

    S-prompts learning with pre-trained transformers: An occam’s razor for domain incremental learning,

    Y . Wang, Z. Huang, and X. Hong, “S-prompts learning with pre-trained transformers: An occam’s razor for domain incremental learning,”ArXiv, vol. abs/2207.12819, 2022. [Online]. Available: https://api.semanticscholar.org/CorpusID:251066766

    Pith/arXiv arXiv 2022

  66. [70]

    Ranpac: Random projections and pre-trained models for continual learning,

    M. D. McDonnell, D. Gong, A. Parvaneh, E. Abbasnejad, and A. van den Hengel, “Ranpac: Random projections and pre-trained models for continual learning,”ArXiv, vol. abs/2307.02251, 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID:259342681

    Pith/arXiv arXiv 2023

  67. [71]

    Expandable subspace ensemble for pre-trained model-based class-incremental learning,

    D.-W. Zhou, H.-L. Sun, H.-J. Ye, and D. chuan Zhan, “Expandable subspace ensemble for pre-trained model-based class-incremental learning,”2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 23 554–23 564, 2024. [Online]. Available: https://api.semanticscholar.org/CorpusID:268531460

    2024

  68. [74]

    icarl: Incremental classifier and representation learning,

    S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert, “icarl: Incremental classifier and representation learning,” inProceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2017, pp. 2001–2010

    2017

  69. [75]

    Connectionist models of recognition memory: constraints imposed by learning and forgetting functions

    R. Ratcliff, “Connectionist models of recognition memory: constraints imposed by learning and forgetting functions.”Psychological review, vol. 97, no. 2, p. 285, 1990

    1990

  70. [76]

    Umap: Uniform manifold approximation and projection for dimension reduction,

    L. McInnes, J. Healy, and J. Melville, “Umap: Uniform manifold approximation and projection for dimension reduction,”arXiv preprint arXiv:1802.03426, 2018

    Pith/arXiv arXiv 2018

  71. [2017]

    Available: https://api.semanticscholar.org/CorpusID: 37308416

    [Online]. Available: https://api.semanticscholar.org/CorpusID: 37308416

  72. [2018]

    Available: https://api.semanticscholar.org/CorpusID: 4254748

    [Online]. Available: https://api.semanticscholar.org/CorpusID: 4254748

  73. [2020]

    Available: https://api.semanticscholar.org/CorpusID: 234353728

    [Online]. Available: https://api.semanticscholar.org/CorpusID: 234353728

  74. [2021]

    Available: https://api.semanticscholar.org/CorpusID: 233710658

    [Online]. Available: https://api.semanticscholar.org/CorpusID: 233710658

  75. [2022]

    Available: https://api.semanticscholar.org/CorpusID: 252089827

    [Online]. Available: https://api.semanticscholar.org/CorpusID: 252089827

  76. [2025]

    Available: https://api.semanticscholar.org/CorpusID: 278602871

    [Online]. Available: https://api.semanticscholar.org/CorpusID: 278602871

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.