Pith. sign in

REVIEW 3 major objections 4 minor 52 references

Incrementally Learning Multiple Diverse Data Domains via Multi-Source Dynamic Expansion Model

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A continual-learning model built from frozen vision transformers plus a learned expert router reports higher average accuracy than replay, prompt, and mixture-of-experts baselines on multi-domain class-incremental streams, with far fewer…

desk verdict A useful empirical study of multi-backbone dynamic expansion for multi-domain continual learning, but the SOTA claim rests on an unstated task-inference protocol and relative error numbers that flatter the absolute gains. read the letter →

arxiv 2501.08878 v2 pith:TKZ6DR6H submitted 2025-01-15 cs.LG cs.AI

Runqing Wu , Fei Ye , Qihe Liu , Guoxi Huang , Jinyu Guo , Rongyao Hu This is my paper · ORCID
classification cs.LGcs.AI
keywords continuallearningmulti-domaindynamicexpansionmixtureofexpertsGumbel-Softmaxroutervisiontransformerclass-incrementalcatastrophicforgetting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MSDEM is a continual-learning model for streams that shift in both class and data domain. Its central claim is that a new per-task expert built on several frozen pre-trained ViT backbones, recombining their features with attention and a Gumbel-Softmax graph router, learns each new domain well without replay or prompt tuning. In the paper's experiments the two-backbone version reports average accuracy at or above StarPrompt's in all nine domain-stream configurations while cutting trainable parameters by about 70% and training time by about 89% in the dual-backbone setup. A sympathetic reader would take away that multi-domain continual learning can be treated as a feature-recombination problem over a fixed pool of extractors.

What carries the argument

The machinery that carries the claim is the combination of a frozen multi-backbone feature pool, a per-task Dynamic Expandable Attention Mechanism (DEAM), and a Dynamic Graph Weight Router (DGWR). DGWR models the experts as nodes of a graph: row $t$ of a relation matrix $C$ is turned into a weight vector by a Gumbel-Softmax draw with Gaussian noise and a temperature $\tau$, so the router can interpolate between hard one-hot expert selection and soft weighting of all experts. The router's learned weights combine normalized representations of previous experts with the current expert's feature, and a graph-attention block then forms the final representation for the current classifier. Only the current task's attention matrices, router row, embedding, and classifier are updated; all prior parameters are frozen.

What would settle it

Rerun the six dual-domain streams with the same training protocol but at test time give each sample no task or domain label and force the model to select an expert from the input alone (for example, choose the expert whose classifier has the highest max softmax score). If the Average metric drops materially below the table values, or if the model cannot be run at all without the task oracle, then the reported state-of-the-art result does not extend to task-agnostic multi-domain continual learning.

Watch

Extended reading notes

Core claim

The core discovery, on the paper's own terms, is that freezing multiple ViT backbones and dynamically growing a lightweight expert per task yields better multi-domain continual learning than replay-based, prompt-based, or mixture-of-experts baselines. Each expert consists of a Dynamic Expandable Attention Mechanism (DEAM), which reweights the concatenated output of the backbones for the new task, an adaptive embedding, and a linear classifier. The Dynamic Graph Weight Router (DGWR) keeps a relation matrix over experts, converts its current row into selection weights via Gumbel-Softmax, and uses those weights to combine previous experts' representations with the new expert's; only the current row is trained. The paper reports that MSDEM2 trained for three epochs reaches average accuracy 98.1-99.7 on the dual-domain streams (Table 1), 97.59-98.15 on the three- and four-domain streams (Table 2), and reduces trainable parameters by 70.36% and training time by 89.35% relative to StarPrompt-2nd.

Load-bearing premise

The model assumes each test sample comes with its task or domain identity so the right expert and classifier can be chosen; without that identity, the paper provides no mechanism to decide which expert should make the prediction.

Editorial extensions

If this is right

  • Multi-domain class-incremental streams can be learned without a memory buffer or generative replay, because freezing old expert parameters and adding only a small per-task expert prevents forgetting.
  • The dual-backbone configuration beats the triple-backbone one on most average scores, so more frozen feature extractors do not automatically improve continual learning and the choice of backbones matters more than their count.
  • With three training epochs, MSDEM2 reaches average accuracy at or above StarPrompt in every reported stream, indicating the gains come from dynamic multi-source expansion rather than from using more training passes.
  • The router's learned weights are asymmetric and order-dependent across domain pairs, so the amount of knowledge transferred between two domains changes with which domain is learned first.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper evaluates under an implicit task oracle: each test sample's domain is known, so the matching expert and router row are used. A task-agnostic version, where the model must infer the expert from the input, is not trained or evaluated; that is the most direct extension to test.
  • Because the router and attention operate on feature vectors rather than pixels, the same scheme could be transplanted to other input types (audio, text, sensor streams) by swapping the frozen backbones for frozen encoders.
  • The reported asymmetric domain weights imply a concrete prediction: permuting the order of the same two domains should flip which domain receives the larger router weight, and the paper's heatmaps already show such order dependence; a quantitative test would compare router rows across all 16 permutations.
  • A held-out generalization test—introducing a domain whose images are unlike any pretraining source—would separate true domain transfer from recombining familiar feature spaces.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper proposes MSDEM, a dynamic expansion method for multi-domain class-incremental learning. The model uses two or three frozen pre-trained ViT backbones to extract features, concatenates them, and creates a new expert for each task, where each expert contains a Dynamic Expandable Attention Mechanism (DEAM), an adaptive layer, a linear classifier, and a Dynamic Graph Weight Router (DGWR) based on a learned relation matrix and Gumbel-Softmax. Previous experts are frozen, and only the current expert's parameters are updated. Experiments cover six two-domain, two three-domain, and one four-domain scenarios, with comparisons against replay, prompt, and mixture-of-experts baselines, plus ablations and a computational-cost analysis. The paper claims state-of-the-art performance with fewer trainable parameters and less training time than StarPrompt.

Significance. If the claims hold, MSDEM would be a useful recipe for combining multiple frozen backbones in multi-domain continual learning, and the released code and the breadth of the experimental matrix are strengths. The computational-cost comparison is also informative. However, the significance is currently conditional: the evaluation protocol is not specified and appears to assume task identity at test time, so the comparison against class-incremental baselines such as StarPrompt is not yet established on equal footing. The headline '16.98% improvement' is a relative error reduction, not an absolute accuracy gain, which further weakens the strength of the claimed margin.

major comments (3)
  1. [Section 3.1, Eqs. (3) and (10)] The paper does not specify an inference protocol for unlabeled test samples, and every prediction rule is task-indexed. The problem statement says the model is evaluated on all testing datasets when training is finished, but it never states whether each test set comes with its task/domain index. Equations (3) and (10) use the j-th or t-th expert by construction, and the router in Eq. (6) is trained per task and is never used to choose an expert for a sample of unknown origin. If test-time task labels are assumed, the evaluation is task-incremental (multi-head) rather than the class-incremental setting claimed in Section 3.1, and the comparison with StarPrompt, which operates without task labels, is not on equal footing. If task labels are not assumed, the architecture has no mechanism to select among the per-task classifiers. Please state the exact evaluation protocol, provide task-agnostic results, or narrow the claim to task-incremental learning.
  2. [Section 4.2, Table 1] The headline '16.98% improvement in the Average metric' is a relative error reduction, not an absolute accuracy improvement. Averaging the six dual-domain scenarios in Table 1, MSDEM2-3ep reaches about 98.68% average accuracy versus about 98.42% for StarPrompt, an absolute gap of about 0.27 points; the improvement as a fraction of StarPrompt's error (about 1.58 points) is roughly 17%, which matches the quoted figure. The per-scenario absolute differences in Table 1 are 0.0–0.5 points and mostly lie within one standard deviation. The text should explicitly say 'relative error reduction' and should report per-scenario absolute differences and a significance check alongside the aggregate percentage.
  3. [Appendix C.1, Table 1 caption] The caption of Table 1 in the appendix first says that the MSDEM2 models use ViT 1 and ViT 2 as backbones and then says that the results reported in the paper for MSDEM2 correspond to the configuration using ViT 1 and ViT 3; this is contradictory. Combined with the implementation list in Section 4.1, which does not unambiguously map 'ViT 1' to 'fine-tuned on ImageNet-1K', the reader cannot tell which backbone pair produced Tables 1 and 2 in the main text or which configuration the ablations refer to. Please reconcile the naming and state, for every table, exactly which two or three backbones are used.
minor comments (4)
  1. [Algorithm 1, Step 3] The set-builder expression and the subsequent summation contain mismatched indices: it should be ˜{z}_j = ¯{z}_j cM_t[j] and Z_t = Σ_{j=1}^t ¯{z}_j cM_t[j], rather than the current ˜z_c / ˜z_t notation.
  2. [Eq. (2)] The expression 'z1 ⊗ z2⊗, · · ·, ⊗ zt′' has a stray comma and should define ⊗ explicitly as concatenation along the feature dimension.
  3. [Appendix B.3] The dataset configuration section gives the task-to-domain mapping only for the T-C100-B-C10 scenario; the class splits and their order for the other nine scenarios are not specified, which impedes exact reproduction.
  4. [Throughout] There are numerous spelling and grammar issues, including 'behavrious' in Section 3.3, 'permutating' in Section 4.3, and 'A verage' in Table 2; a careful copyedit is needed.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: MSDEM is an empirical fit-and-evaluate paper; the SOTA claim rests on benchmark comparisons, not on a derivation that re-uses its own output.

full rationale

The paper contains no theoretical derivation in which a predicted quantity is defined in terms of the target quantity. The router weights, attention parameters, relation matrix, and classifiers are trained parameters; the reported accuracies are measured on held-out test data, not read off from the fitting procedure. The 'domain dependency' analysis in Section 4.3 interprets the trained router weights after the fact, which is post-hoc interpretation rather than a circular step, and it is not load-bearing for the SOTA claim. The strongest concern is that the evaluation protocol is underspecified: Eq. (3) and Eq. (10) select expert t by construction, and Section 3.1 never states whether test samples arrive with task/domain identity or how an unlabelled sample would be routed. If task identity is provided, the comparison is task-incremental rather than the class-incremental setting implied by the problem statement. This is a validity and fairness-of-comparison issue, not a circular derivation: the model's accuracy is still an empirical outcome of fitting and testing, and the paper does not invoke a self-citation or uniqueness theorem to force the result. No load-bearing self-citations are present. Accordingly, the circularity score is low.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The central claim is an empirical performance claim, so most learned network parameters are not listed as free parameters; the entries above capture the hand-set or validation-tuned hyperparameters that the experiments depend on. The axioms are the unstated premises about backbone sufficiency, inference-time task identity, and the reliability of the Gumbel-Softmax router. The invented entities are the two new architectural modules, which lack independent evidence outside this paper's benchmarks.

free parameters (4)
  • Number of attention heads h = not reported (search space {4, 8, 16, 32})
    Tuned via Bayesian optimization on the validation set; affects both the expandable attention and the graph attention modules; the final value is not disclosed.
  • Gumbel temperature tau and noise scale sigma = not reported
    Control the hardness of expert selection in Eq. (6); the paper does not state the values used in the experiments, and the router output depends on them.
  • Per-module learning rates = from {1e-4, 1e-2}, actual values not reported
    Separate Adam optimizers are used for DEAM, Router, Graph Block, and Classifier; exact values are omitted, and performance depends on them.
  • Embedding output dimension de = 288 (dual ViT), 768 (triple ViT)
    Chosen to balance parameter growth and feature dimensionality; it is a hand-set design parameter that affects all experts.
assumptions (4)
  • domain assumption Multiple frozen pre-trained ViT backbones provide complementary and sufficient representations for any future domain
    Invoked in Section 3.2 ('The multi-source backbones'); the backbones are frozen and the attention and router only recombine fixed features, so the model has no mechanism to adapt to domains far from the pretraining distribution.
  • domain assumption Task identity (or expert index) is known at inference time
    The problem statement in Section 3.1 and Algorithm 1 create a dedicated expert and classifier per task; no task-agnostic inference procedure is presented, yet the reported evaluation requires choosing the correct expert.
  • domain assumption Freezing all previously trained parameters fully prevents forgetting
    Standard assumption of dynamic expansion frameworks, used throughout Sections 3.2 to 3.4; the paper does not analyze interference through the router or attention paths.
  • ad hoc to paper Gumbel-Softmax on a learned relation matrix yields a reliable and trainable expert selection
    Core of the DGWR (Eq. (6)); the paper provides no theoretical or empirical analysis of selection quality beyond final accuracy, and the relation matrix is fitted on each task's training data.
invented entities (2)
  • Dynamic Expandable Attention Mechanism (DEAM)
    purpose: Fuses features from multiple backbones with per-task trainable K, Q, V matrices
    New module introduced in Section 3.3; its only evidence is the paper's experiments, and the ablations in Appendix C show it sometimes hurts (e.g., MSDEM3 without attention improves on several tasks).
  • Dynamic Graph Weight Router (DGWR)
    purpose: Selects and reweights previously learned experts via a learnable relation matrix and Gumbel-Softmax
    New module introduced in Section 3.4; its only evidence is the paper's experiments, and the ablations show mixed benefits (e.g., MSDEM3 without router sometimes improves).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Incrementally Learning Multiple Diverse Data Domains via Multi-Source Dynamic Expansion Model." pith.science (2026). https://pith.science/paper/TKZ6DR6H

@misc{pith2026250108878,
  author       = {Pith},
  title        = {Pith review of: Incrementally Learning Multiple Diverse Data Domains via Multi-Source Dynamic Expansion Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TKZ6DR6H}},
  note         = {Machine review of arXiv:2501.08878}
}
read the original abstract

Continual Learning seeks to develop a model capable of incrementally assimilating new information while retaining prior knowledge. However, current research predominantly addresses a straightforward learning context, wherein all data samples originate from a singular data domain. This paper shifts focus to a more complex and realistic learning environment, characterized by data samples sourced from multiple distinct domains. We tackle this intricate learning challenge by introducing a novel methodology, termed the Multi-Source Dynamic Expansion Model (MSDEM), which leverages various pre-trained models as backbones and progressively establishes new experts based on them to adapt to emerging tasks. Additionally, we propose an innovative dynamic expandable attention mechanism designed to selectively harness knowledge from multiple backbones, thereby accelerating the new task learning. Moreover, we introduce a dynamic graph weight router that strategically reuses all previously acquired parameters and representations for new task learning, maximizing the positive knowledge transfer effect, which further improves generalization performance. We conduct a comprehensive series of experiments, and the empirical findings indicate that our proposed approach achieves state-of-the-art performance.

Figures

Figures reproduced from arXiv: 2501.08878 by the authors.

Figure 1
Figure 1. Overall framework of the proposed method. (a) During the initial training task, task [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The expert selection process with different val [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 4
Figure 4. (a) Feature maps of trained experts across four [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figures from the paper (6 more)
Figure 1
Figure 1. Figure 1: (a) Model weight allocation across historical domains, the current domain, [PITH_FULL_IMAGE:figures/full_fig_p016_1.png]
Figure 2
Figure 2. Figure 2: (a)-(i) Performance comparison of forgetting curves for MSDEM [PITH_FULL_IMAGE:figures/full_fig_p017_2.png]
Figure 3
Figure 3. Figure 3: (a)-(f) PCA feature distribution of experts across four task domains under [PITH_FULL_IMAGE:figures/full_fig_p018_3.png]
Figure 4
Figure 4. Figure 4: (a)-(d) Weight distribution across 16 dual-domain task scenarios under differ [PITH_FULL_IMAGE:figures/full_fig_p019_4.png]
Figure 5
Figure 5. Figure 5: (a)-(d) Weight allocation of different domains across 16 dual-domain task [PITH_FULL_IMAGE:figures/full_fig_p020_5.png]
Figure 1
Figure 1. Figure 1: (a) further summarizes the weights from all permutation schemes, in [PITH_FULL_IMAGE:figures/full_fig_p022_1.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 44 canonical work pages

  1. [1]

    Achille, T

    A. Achille, T. Eccles, L. Matthey, C. Burgess, N. Watters, A. Lerchner, and I. Higgins. Life-long disentangled representation learning with cross-domain latent homologies. In Advances in Neural Information Processing Systems (NeurIPS) , pages 9873--9883, 2018

    work page 2018

  2. [2]

    Uncertainty-based continual learning with adaptive regularization

    Hongjoon Ahn, Sungmin Cha, Donggyu Lee, and Taesup Moon. Uncertainty-based continual learning with adaptive regularization. In Advances in Neural Information Processing Systems , pages 4394--4404, 2019

    work page 2019

  3. [3]

    Rainbow memory: Continual learning with a memory of diverse samples

    Jihwan Bang, Heesu Kim, YoungJoon Yoo, Jung-Woo Ha, and Jonghyun Choi. Rainbow memory: Continual learning with a memory of diverse samples. In Proc. of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 8218--8227, 2021

    work page 2021

  4. [4]

    Online continual learning on a contaminated data stream with blurry task boundaries

    Jihwan Bang, Hyunseo Koh, Seulki Park, Hwanjun Song, Jung-Woo Ha, and Jonghyun Choi. Online continual learning on a contaminated data stream with blurry task boundaries. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 9275--9284, June 2022

    work page 2022

  5. [5]

    Dark experience for general continual learning: a strong, simple baseline

    Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. Advances in neural information processing systems , 33:15920--15930, 2020

    work page 2020

  6. [6]

    Co2l: Contrastive continual learning

    Hyuntak Cha, Jaeho Lee, and Jinwoo Shin. Co2l: Contrastive continual learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 9516--9525, 2021

    work page 2021

  7. [7]

    Chaudhry, M

    A. Chaudhry, M. Rohrbach, M. Elhoseiny, T. Ajanthan, P. Dokania, P. H. S. Torr, and M.'A. Ranzato. On tiny episodic memories in continual learning. arXiv preprint arXiv:1902.10486 , 2019

    arXiv 1902

  8. [8]

    Cortes, X

    C. Cortes, X. Gonzalvo, V. Kuznetsov, M. Mohri, and S. Yang. Adanet: Adaptive structural learning of artificial neural networks. In Proc. of Int. Conf. on Machine Learning (ICML), vol. PMLR 70 , pages 874--883, 2017

    work page 2017

Show all 52 references
  1. [9]

    Flattening sharpness for dynamic gradient projection memory benefits continual learning

    Danruo Deng, Guangyong Chen, Jianye Hao, Qiong Wang, and Pheng-Ann Heng. Flattening sharpness for dynamic gradient projection memory benefits continual learning. Advances in Neural Information Processing Systems , 34:18710--18721, 2021

    2021

  2. [10]

    Kernel continual learning

    Mohammad Mahdi Derakhshani, Xiantong Zhen, Ling Shao, and Cees Snoek. Kernel continual learning. In International Conference on Machine Learning , pages 2621--2631. PMLR, 2021

    2021

  3. [11]

    Loss of plasticity in deep continual learning

    Shibhansh Dohare, J Fernando Hernandez-Garcia, Qingfeng Lan, Parash Rahman, A Rupam Mahmood, and Richard S Sutton. Loss of plasticity in deep continual learning. Nature , 632(8026):768--774, 2024

    2024

  4. [12]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv...

    2010 arXiv

  5. [13]

    Dytox: Transformers for continual learning with dynamic token expansion

    Arthur Douillard, Alexandre Ram \'e , Guillaume Couairon, and Matthieu Cord. Dytox: Transformers for continual learning with dynamic token expansion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9285--9295, 2022

    2022

  6. [14]

    Goodfellow, J

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Proc. Advances in Neural Inf. Proc. Systems (NIPS) , pages 2672--2680, 2014

    2014

  7. [15]

    Knowledge distillation: A survey

    Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. Knowledge distillation: A survey. International Journal of Computer Vision , 129:1789--1819, 2021

    2021

  8. [16]

    Not just selection, but exploration: Online class-incremental continual learning via dual view consistency

    Yanan Gu, Xu Yang, Kun Wei, and Cheng Deng. Not just selection, but exploration: Online class-incremental continual learning via dual view consistency. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 7442--7451, June 2022

    2022

  9. [17]

    Online continual learning through mutual information maximization

    Yiduo Guo, Bing Liu, and Dongyan Zhao. Online continual learning through mutual information maximization. In International Conference on Machine Learning , pages 8109--8126. PMLR, 2022

    2022

  10. [18]

    Hinton, O

    G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network. In Proc. NIPS Deep Learning Workshop, arXiv preprint arXiv:1503.02531 , 2014

    2014 arXiv

  11. [19]

    Compacting, picking and growing for unforgetting continual learning

    Ching-Yi Hung, Cheng-Hao Tu, Cheng-En Wu, Chien-Hung Chen, Yi-Ming Chan, and Chu-Song Chen. Compacting, picking and growing for unforgetting continual learning. In Advances in Neural Information Processing Systems , pages 13647--13657, 2019

    2019

  12. [20]

    Non-exemplar online class-incremental continual learning via dual-prototype self-augment and refinement

    Fushuo Huo, Wenchao Xu, Jingcai Guo, Haozhao Wang, and Yunfeng Fan. Non-exemplar online class-incremental continual learning via dual-prototype self-augment and refinement. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 38, pages 12698--12707, 2024

    2024

  13. [21]

    Npcl: Neural processes for uncertainty-aware continual learning

    Saurav Jha, Dong Gong, He Zhao, and Lina Yao. Npcl: Neural processes for uncertainty-aware continual learning. Advances in Neural Information Processing Systems , 36, 2024

    2024

  14. [22]

    Generating instance-level prompts for rehearsal-free continual learning

    Dahuin Jung, Dongyoon Han, Jihwan Bang, and Hwanjun Song. Generating instance-level prompts for rehearsal-free continual learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 11847--11857, 2023

    2023

  15. [23]

    Forget-free continual learning with winning subnetworks

    Haeyong Kang, Rusty John Lloyd Mina, Sultan Rizky Hikmawan Madjid, Jaehong Yoon, Mark Hasegawa-Johnson, Sung Ju Hwang, and Chang D Yoo. Forget-free continual learning with winning subnetworks. In International Conference on Machine Learning , pages 10734--10750. PMLR, 2022

    2022

  16. [24]

    Measuring catastrophic forgetting in neural networks

    Ronald Kemker, Marc McClure, Angelina Abitino, Tyler Hayes, and Christopher Kanan. Measuring catastrophic forgetting in neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 32, 2018

    2018

  17. [25]

    Sddgr: Stable diffusion-based deep generative replay for class incremental object detection

    Junsu Kim, Hoseong Cho, Jihyeon Kim, Yihalem Yimolal Tiruneh, and Seungryul Baek. Sddgr: Stable diffusion-based deep generative replay for class incremental object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 28772--28...

    2024

  18. [26]

    D. P. Kingma and M. Welling. Auto-encoding variational B ayes. arXiv preprint arXiv:1312.6114 , 2013

    2013 arXiv

  19. [27]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, Univ. of Toronto, 2009

    2009

  20. [28]

    Tiny image Net visual recognition challenge

    Ya Le and Xuan Yang. Tiny image Net visual recognition challenge. Technical report, Univ. of Stanford, 2015

    2015

  21. [29]

    Li and D

    Z. Li and D. Hoiem. Learning without forgetting. IEEE Trans. on Pattern Analysis and Machine Intelligence , 40(12):2935--2947, 2017

    2017

  22. [30]

    Gradient episodic memory for continual learning

    David Lopez-Paz and Marc'Aurelio Ranzato. Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems , pages 6467--6476, 2017

    2017

  23. [31]

    James Martens and Roger B. Grosse. Optimizing neural networks with kronecker-factored approximate curvature. In Francis R. Bach and David M. Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015 , volume 3...

    2015

  24. [32]

    McDonnell, Dong Gong, Amin Parvaneh, Ehsan Abbasnejad, and Anton van den Hengel

    Mark D. McDonnell, Dong Gong, Amin Parvaneh, Ehsan Abbasnejad, and Anton van den Hengel. Ranpac: Random projections and pre-trained models for continual learning. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in N...

    2023

  25. [33]

    Semantic residual prompts for continual learning

    Martin Menabue, Emanuele Frascaroli, Matteo Boschini, Enver Sangineto, Lorenzo Bonicelli, Angelo Porrello, and Simone Calderara. Semantic residual prompts for continual learning. arXiv preprint arXiv:2403.06870 , 2024

    2024 arXiv

  26. [34]

    Variational continual learning

    Cuong V Nguyen, Yingzhen Li, Thang D Bui, and Richard E Turner. Variational continual learning. In Proc. of Int. Conf. on Learning Representations (ICLR), arXiv preprint arXiv:1710.10628 , 2018

    2018 arXiv

  27. [35]

    G. I. Parisi, R. Kemker, J. L. Part, C. Kanan, and S. Wermter. Continual lifelong learning with neural networks: A review. Neural Networks , 113:54--71, 2019

    2019

  28. [36]

    Polikar, L

    R. Polikar, L. Upda, S. S. Upda, and Vasant Honavar. Learn++: An incremental learning algorithm for supervised neural networks. IEEE Trans. on Systems Man and Cybernetics, Part C , 31(4):497--508, 2001

    2001

  29. [37]

    Ramapuram, M

    J. Ramapuram, M. Gregorova, and A. Kalousis. Lifelong generative modeling. In Proc. Int. Conf. on Learning Representations (ICLR), arXiv preprint arXiv:1705.09847 , 2017

    2017 arXiv

  30. [38]

    iCaRL : Incremental classifier and representation learning

    Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. iCaRL : Incremental classifier and representation learning. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) , pages 2001--2010, 2017

    2001

  31. [39]

    B. Ren, H. Wang, J. Li, and H. Gao. Life-long learning based on dynamic combination model. Applied Soft Computing , 56:398--404, 2017

    2017

  32. [40]

    Online structured L aplace approximations for overcoming catastrophic forgetting

    Hippolyt Ritter, Aleksandar Botev, and David Barber. Online structured L aplace approximations for overcoming catastrophic forgetting. In Advances in Neural Information Processing Systems (NeurIPS) , volume 31, pages 3742--3752, 2018

    2018

  33. [41]

    Progressive neural networks

    Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.04671 , 2016

    2016 arXiv

  34. [42]

    Continual learning via bit-level information preserving

    Yujun Shi, Li Yuan, Yunpeng Chen, and Jiashi Feng. Continual learning via bit-level information preserving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 16674--16683, 2021

    2021

  35. [43]

    H. Shin, J. K. Lee, J. Kim, and J. Kim. Continual learning with deep generative replay. In Advances in Neural Inf. Proc. Systems (NIPS) , pages 2990--2999, 2017

    2017

  36. [44]

    Gcr: Gradient coreset based replay buffer selection for continual learning

    Rishabh Tiwari, Krishnateja Killamsetty, Rishabh Iyer, and Pradeep Shenoy. Gcr: Gradient coreset based replay buffer selection for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 99--108, June 2022

    2022

  37. [45]

    Efficient feature transformations for discriminative and generative continual learning

    Vinay Kumar Verma, Kevin J Liang, Nikhil Mehta, Piyush Rai, and Lawrence Carin. Efficient feature transformations for discriminative and generative continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 13865--13875, 2021

    2021

  38. [46]

    Training networks in null space of feature covariance for continual learning

    Shipeng Wang, Xiaorong Li, Jian Sun, and Zongben Xu. Training networks in null space of feature covariance for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 184--193, 2021

    2021

  39. [47]

    A unified and general framework for continual learning

    Zhenyi Wang, Yan Li, Li Shen, and Heng Huang. A unified and general framework for continual learning. arXiv preprint arXiv:2403.13249 , 2024

    2024 arXiv

  40. [48]

    BatchEnsemble : an alternative approach to efficient ensemble and lifelong learning

    Yeming Wen, Dustin Tran, and Jimmy Ba. BatchEnsemble : an alternative approach to efficient ensemble and lifelong learning. In Proc. Int. Conf. on Learning Representations (ICLR), arXiv preprint arXiv:2002.06715 , 2020

    2002 arXiv

  41. [49]

    Meta-attention for vit-backed continual learning

    Mengqi Xue, Haofei Zhang, Jie Song, and Mingli Song. Meta-attention for vit-backed continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 150--159, 2022

    2022

  42. [50]

    Boosting continual learning of vision-language models via mixture-of-experts adapters

    Jiazuo Yu, Yunzhi Zhuge, Lu Zhang, Ping Hu, Dong Wang, Huchuan Lu, and You He. Boosting continual learning of vision-language models via mixture-of-experts adapters. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 23219--23230, 2024

    2024

  43. [51]

    M. Zhai, L. Chen, F. Tung, J He, M. Nawhal, and G. Mori. Lifelong GAN : Continual learning for conditional image generation. In Proc. of the IEEE/CVF Int. Conf. on Computer Vision (ICCV) , pages 2759--2768, 2019

    2019

  44. [52]

    Online incremental feature learning with denoising autoencoders

    Guanyu Zhou, Kihyuk Sohn, and Honglak Lee. Online incremental feature learning with denoising autoencoders. In Proc. Int. Conf. on Artificial Intelligence and Statistics (AISTATS), vol. PMLR 22 , pages 1453--1461, 2012

    2012

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.