Pith. sign in

REVIEW 4 major objections 5 minor 59 references

This paper shows that a self-organizing map storing per-unit Gaussian statistics in a VAE latent space can replay old classes without storing raw data or task labels, beating most memory-based baselines on CIFAR-10 and CIFAR-100 single-clas

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A SOM-VAE generative replay method stores per-unit Gaussian statistics instead of raw data and reports competitive class-incremental accuracy on standard benchmarks.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Useful extension of c-SOM with real headroom on one-class CIFAR, but missing c-SOM baseline, error bars, and replay-fidelity checks leave the headline margins unproven. the 4 major comments →

arxiv 2508.21240 v1 pith:VA522FR7 submitted 2025-08-28 cs.LG

Class Incremental Continual Learning with Self-Organizing Maps and Variational Autoencoders Using Synthetic Replay

Pujan Thapa , Alexander Ororbia , Travis Desell This is my paper
classification cs.LG
keywords class-incremental learningcontinual learningself-organizing mapsvariational autoencodersgenerative replaysynthetic replaycatastrophic forgettingunsupervised learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to show that a continual learner need not store raw examples or task labels to keep old classes alive. It builds a self-organizing map on top of a variational autoencoder's latent space and, for every neuron (best-matching unit), stores only a running mean, variance, and covariance of the latent codes that land on it. At each new class step, it samples synthetic latent vectors from those Gaussians and decodes them with the VAE to replay old classes beside new data. On the hardest one-class-per-task setting, the method reaches 54.16% on CIFAR-10 and 12.41% on CIFAR-100, which the authors state improves on previous best single-class incremental results by nearly 10 and 7 percentage points respectively. A sympathetic reader would care because the memory is compact, fixed-size, and interpretable, and because generative replay has historically struggled on high-dimensional images.

Core claim

The central claim is that per-BMU Gaussian statistics in a compressed latent space are enough to sustain an unsupervised SOM classifier across a 100-class stream. For MNIST and FashionMNIST the SOM works directly on pixels; for CIFAR-10 and CIFAR-100 a global VAE compresses images to a 128-dimensional latent, the SOM clusters those latents, and each unit's full covariance models the local latent distribution. Replay is generated by sampling N(μ, Σ) at each BMU, decoding through the VAE, and mixing with new-class data; no task identifiers and no raw examples are used. The authors report that this scheme outperforms most memory-free, memory-based, and bio-inspired baselines, including a roughl

What carries the argument

The carrying mechanism is the SOM-as-memory with per-unit running statistics. Each BMU accumulates a mean vector, a per-dimension variance vector, and a full covariance matrix over the latent codes assigned to it; Algorithm 1 regularizes the covariance by eigenvalue clamping before sampling. In the VAE-SOM variant, the VAE's latent compression drops raw CIFAR dimensionality from 3,072 to 128, making full per-BMU covariance matrices feasible, and its decoder turns sampled latents back into images for replay. The class label recorded per BMU is used only for evaluation, not for updating the model.

Load-bearing premise

The load-bearing premise is that the latent codes landing on each SOM unit are well described by a single multivariate Gaussian and that the continually retrained VAE decoder still reconstructs old-class latents faithfully at replay time; if either fails, sampled replays become poor and the reported accuracy gains collapse.

What would settle it

Measure, across the stream, each BMU's latent distribution for deviation from Gaussianity (e.g., multivariate normality tests or the fraction of variance explained by the Gaussian fit) and track the VAE decoder's reconstruction error on held-out images of early classes. If early-class reconstructions degrade substantially or per-BMU latent residuals become multimodal by the 100-class CIFAR-100 stream, the replay mechanism is falsified. A simpler decisive test: during replay, replace the sampled Gaussians with the true per-BMU empirical latent codes; a large accuracy jump would show the Gaussia

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A fixed-size memory whose cost scales with SOM grid size, not number of classes or stream length, can support class-incremental learning without raw data storage.
  • The model doubles as a trained generative model: after the stream, sampling from BMU Gaussians and decoding yields feasible class-conditioned examples.
  • Larger SOM grids improve accuracy up to a point; beyond that, sparsity and underused units hurt, so replay quality depends on matching grid resolution to latent dimensionality.
  • Compact latent spaces (32x2x2 features) outperform larger ones in this framework, suggesting that too much latent capacity spreads BMUs too thin for reliable Gaussian replay.
  • In the one-class-per-task stream the global VAE-SOM variant outperforms the per-BMU VAE variant even though per-BMU VAE is more complex; a shared representation supports better replay than fine-grained local decoders.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If per-BMU Gaussianity degrades for very large class streams, a natural extension would be to test mixture-of-Gaussians or nonparametric density estimates per BMU; the paper's own per-BMU VAE variant suggests local specialization has limits.
  • The method's success implies that replay quality, not memory size, is the binding constraint in generative continual learning; future comparisons should report reconstruction fidelity of old classes over time, not just final accuracy.
  • A testable extension is to replace the VAE decoder with a pretrained foundational decoder to see whether representation drift or decoder drift is the main source of forgetting; the paper names pretrained autoencoders as future work.
  • Because the SOM stores class-label hit counts, the same mechanism could be used for open-set or novelty detection, flagging inputs whose BMU has no dominant class.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a continual class-incremental learning framework that combines self-organizing maps (SOMs) with variational autoencoders (VAEs). For low-dimensional inputs (MNIST, Fashion-MNIST) the SOM operates directly on pixels; for CIFAR-10/100 a VAE compresses images into a latent space and the SOM operates on latent codes. Each SOM best-matching unit (BMU) stores running mean, variance, and covariance, from which synthetic replay samples are drawn. Replayed samples are decoded (where applicable) and mixed with new-class data. Three variants are presented: SOM-only, global VAE-SOM, and per-BMU VAE-SOM. The paper reports results on Split-MNIST, Split-CIFAR-10, Split-CIFAR-100, and one-class-at-a-time MNIST/CIFAR-10/CIFAR-100, claiming competitive or superior accuracy relative to memory-based and memory-free baselines, including one-class CIFAR-10 accuracy of 54.16% and CIFAR-100 accuracy of 12.41% without external exemplars or task labels.

Significance. If the empirical claims hold, the framework is a meaningful contribution: it offers a memory-efficient generative replay mechanism whose storage cost scales with the fixed SOM grid rather than the dataset, it avoids task labels during model updates, and the UML visualization of SOM units over time is a practical diagnostic tool. The algorithmic variants are described clearly with pseudocode, and the appendix gives substantial hyperparameter detail. The strongest claimed result—large gains on one-class CIFAR-10/100 over prior memory-free and even memory-based methods—would be significant for task-free class-incremental learning. However, the current experimental reporting and the absence of replay-fidelity measurements leave the central accuracy claims insufficiently supported.

major comments (4)
  1. [§3.3, Table 1; §3.4, Table 2] The repeated-run reporting is inconsistent and load-bearing. The text states every experiment was repeated 10 times, but Table 1's caption says results are over 5 repetitions, and Table 2 provides no error bars or repetition count at all. Since the headline is the one-class CIFAR-10 54.16% versus 44.54% (DisCOIL), the absence of variance makes it impossible to assess reliability. Additionally, all results are reported from 'best-found hyperparameter settings' (Section 3) with no validation procedure described; if configurations were selected using the test set, the accuracy claims are inflated. The authors should specify the validation protocol, report means and standard deviations for all tables, and provide significance tests for the claimed improvements, especially 53.01% vs. DER++ 52.01% in Table 1, where the DER++ standard deviation is 3.06.
  2. [§2.1 and Algorithm 3] The method's two core assumptions are unmeasured: (i) that latent codes assigned to each SOM BMU are well approximated by a single multivariate Gaussian N(µ, Σ), and (ii) that the continually retrained VAE decoder can still decode samples drawn from stored statistics of old classes into recognizable inputs. Algorithm 3 is a self-referential loop: the VAE is retrained on a mix of new real data and decoded replay, and the SOM is updated on latents of those decoded replay images. If decoder drift occurs or a BMU's latent distribution becomes multimodal, replay quality degrades and the reported accuracies could reflect classification of artifacts rather than retained knowledge. The paper does not report any replay-fidelity metric (e.g., FID, reconstruction error for old classes, or oracle-classifier accuracy on replayed samples). I would require such measurements, per task, as a condition fo
  3. [§3.3, Table 1] The claim of outperforming DER++ on Split-CIFAR-10 is not supported by the reported statistics: VAE-SOM achieves 53.01 ± 0.92 while DER++ achieves 52.01 ± 3.06. The difference is far smaller than the DER++ standard deviation, and no paired test or confidence interval is given. The abstract's 'competitive with state-of-the-art' is more accurate than 'outperforming' used later in Section 3.3. Either add proper statistical comparison or soften the claim.
  4. [Related work and Results] The direct predecessor continual SOM (c-SOM, Vaidya et al. 2021) is discussed in the introduction but never benchmarked. Since c-SOM also uses internal Gaussian replay (though in input space), a quantitative comparison is essential to substantiate the paper's claim that adding a VAE latent space improves scalability. Omitting this baseline weakens the novelty and contribution claims. The authors should add c-SOM to Table 1 or Table 2, or at minimum provide a dedicated ablation showing the VAE component's benefit over c-SOM under the same protocol.
minor comments (5)
  1. [Section 3, Table 1 caption] The text says each experiment was repeated 10 times, but Table 1's caption says '5 repetitions.' Please reconcile.
  2. [Section 2.1, variance update] The running variance update σ² ← (1−α)σ² + α(µ−x)² is written after µ has already been updated, so the deviation is computed relative to the new mean. This is a legitimate momentum form but should be stated explicitly to avoid confusion.
  3. [Appendix B, Tables 6 and 7] The latent dimension '32 × 32 × 2 × 2' in the text and table header appears to be a typo for '32 × 2 × 2'. Similarly, '128 × 32 × 2 × 2' should likely be '128 × 2 × 2'.
  4. [Throughout] There are minor language issues: 'widely-used used', '1 < i < n; 1 < jn;', and 'iCARL' vs. 'iCaRL' in the references. These should be corrected.
  5. [Section 3.2 and Figure 2] The paper notes that some generated samples are 'hallucinatory.' This is presented qualitatively; quantifying the fraction of such samples (or using a perceptual metric) would strengthen the generative-capability claims.

Circularity Check

0 steps flagged

No significant circularity: the central accuracy claims are empirical comparisons against external baselines, not reductions to fitted constants or self-citation.

full rationale

The paper's load-bearing claims are test-set classification accuracies on standard continual learning benchmarks (Section 3, Tables 1-2). These are measured on held-out inputs after training, so they are not by construction equal to the SOM's stored statistics or to any fitted parameter. The generative replay mechanism (Algorithms 2-4) does use the model's own BMU statistics to train itself, but this is the algorithm's design, not a 'prediction' that reduces to its inputs; the evidence for the method is external test accuracy. The self-citations (c-SOM, Ororbia 2021, Ororbia et al. 2022) are related-work references and are not load-bearing; no uniqueness theorem or central premise is imported from them. The paper's per-BMU Gaussian assumption and decoder stability are unvalidated and could be empirically fragile, and the paper itself concedes some generations are 'hallucinatory' (Section 3.2), but an unmeasured assumption is a correctness risk, not circularity. No passage asserts a missing proof or limitation that would constitute self-definitional circularity.

Axiom & Free-Parameter Ledger

9 free parameters · 6 axioms · 0 invented entities

The central results rest on assumptions about SOM memory retention, Gaussianity of latent codes, and decoder stability over 100-task streams, none of which are directly measured. Several implementation constants (alpha, epsilon, L) are unreported, so the ledger is heavier than the prose suggests.

free parameters (9)
  • SOM grid size = 10x10 to 50x50; best 35x35 for MNIST/FMNIST, 40x40 for CIFAR-10, 50x50 for CIFAR-100
    Swept in Appendix B; reported accuracies use the best grid per dataset.
  • VAE latent dimensionality = 32x2x2 (128 dims) best; 64x2x2 and 128x2x2 also tested
    Swept in Appendix B; larger latent spaces degrade accuracy.
  • SOM learning rate and neighborhood sigma = 0.5; sigma 0.95/0.94
    Hand-set hyperparameters, not fitted to data, but accuracy depends on them.
  • SOM training epochs = 150 for CIFAR; 10 to 50 for MNIST/FMNIST
    Selected from Appendix A ranges.
  • VAE training epochs = 200
    Fixed; no sensitivity analysis.
  • Running statistics momentum alpha = not reported
    Controls all running mean/variance/covariance updates in Section 2.1; absent from the paper.
  • Eigenvalue regularization constant epsilon = not reported
    Required by Algorithm 1; without it sampling is underspecified.
  • Feature loss scale and KL loss scale = 1.0 each
    Hand-set; not swept.
  • Number of VGG feature layers L = not reported
    Feature loss in Eq. 1 sums over L selected layers; L is never specified.
axioms (6)
  • domain assumption Sequential SOM updates preserve enough old-class structure for replay to work
    The memory mechanism assumes old-class regions remain in the map while new classes overwrite parts of it (Section 3.1, Figure 1).
  • domain assumption Latent codes within each BMU are approximately Gaussian
    Synthetic replay samples are drawn from one multivariate Gaussian per BMU (Section 2.1, Algorithm 1).
  • domain assumption The continually trained VAE decoder remains a valid decoder for old-class latent samples
    Replay decodes stored-latent samples with the current global decoder (Algorithm 3); VAE forgetting is not measured.
  • domain assumption Majority-vote BMU labeling is a valid inference procedure
    Test accuracy uses the most frequent training label per BMU (Section 2).
  • domain assumption Eigenvalue-clamped Gaussian sampling yields useful replay data
    Algorithm 1 assumes this; no quantitative validity check of generated replay beyond visual inspection.
  • standard math Eigen-decomposition and Gaussian sampling are used without proof
    Standard linear algebra and probability, invoked in Algorithm 1 and Section 2.1.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Class Incremental Continual Learning with Self-Organizing Maps and Variational Autoencoders Using Synthetic Replay." pith.science (2026). https://pith.science/paper/VA522FR7

@misc{pith2026250821240,
  author       = {Pith},
  title        = {Pith review of: Class Incremental Continual Learning with Self-Organizing Maps and Variational Autoencoders Using Synthetic Replay},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VA522FR7}},
  note         = {Machine review of arXiv:2508.21240}
}
Share X Bluesky LinkedIn Reddit HN
abstract

This work introduces a novel generative continual learning framework based on self-organizing maps (SOMs) and variational autoencoders (VAEs) to enable memory-efficient replay, eliminating the need to store raw data samples or task labels. For high-dimensional input spaces, such as of CIFAR-10 and CIFAR-100, we design a scheme where the SOM operates over the latent space learned by a VAE, whereas, for lower-dimensional inputs, such as those found in MNIST and FashionMNIST, the SOM operates in a standalone fashion. Our method stores a running mean, variance, and covariance for each SOM unit, from which synthetic samples are then generated during future learning iterations. For the VAE-based method, generated samples are then fed through the decoder to then be used in subsequent replay. Experimental results on standard class-incremental benchmarks show that our approach performs competitively with state-of-the-art memory-based methods and outperforms memory-free methods, notably improving over best state-of-the-art single class incremental performance on CIFAR-10 and CIFAR-100 by nearly $10$\% and $7$\%, respectively. Our methodology further facilitates easy visualization of the learning process and can also be utilized as a generative model post-training. Results show our method's capability as a scalable, task-label-free, and memory-efficient solution for continual learning.

Figures

Figures reproduced from arXiv: 2508.21240 by Alexander Ororbia, Pujan Thapa, Travis Desell.

Figure 1
Figure 1. Figure 1: Visualization of decoded SOM unit vectors after the first, second and third classes on CIFAR-10 (40x40 SOM). [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Synthetic CIFAR-100 samples for the first [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Synthetic samples generated using Gaussian sam [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Synthetic CIFAR-10 samples generated by sam [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Visualization of the SOM unit vectors after each task for MNIST (35×35 SOM). [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Visualization of SOM unit vectors after each task for Fashion-MNIST (35×35 SOM). [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Visualization of decoded SOM unit vectors after each task on CIFAR-10 (40x40 SOM). [PITH_FULL_IMAGE:figures/full_fig_p016_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Visualization of decoded SOM unit vectors after every 10 tasks on CIFAR-100 (50x50 SOM). [PITH_FULL_IMAGE:figures/full_fig_p017_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

59 extracted references · 36 canonical work pages · 9 internal anchors

  1. [1]

    Aljundi, R.; Caccia, L.; Belilovsky, E.; Caccia, M.; Lin, M.; Charlin, L.; and Tuytelaars, T. 2019 a . Online Continual Learning with Maximally Interfered Retrieval. arXiv:1908.04742

    Pith/arXiv arXiv 2019

  2. [2]

    Aljundi, R.; Lin, M.; Goujaud, B.; and Bengio, Y. 2019 b . Gradient based sample selection for online continual learning. arXiv:1903.08671

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  3. [3]

    Aly, S. 2014. Learning invariant local image descriptor using convolutional Mahalanobis self-organising map. Neurocomputing, 142: 239--247. SI Computational Intelligence Techniques for New Product Development

    work page 2014

  4. [4]

    Ashfahani, A.; and Pratama, M. 2023. Unsupervised Continual Learning in Streaming Environments. IEEE Transactions on Neural Networks and Learning Systems, 34(12): 9992--10003

    work page 2023

  5. [5]

    Bardes, A.; Ponce, J.; and LeCun, Y. 2022. VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning. arXiv:2105.04906

    Pith/arXiv arXiv 2022

  6. [6]

    Bashivan, P.; Schrimpf, M.; Ajemian, R.; Rish, I.; Riemer, M.; and Tu, Y. 2019. Continual Learning with Self-Organizing Maps. arXiv:1904.09330

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  7. [7]

    Buzzega, P.; Boschini, M.; Porrello, A.; Abati, D.; and Calderara, S. 2020. Dark Experience for General Continual Learning: a Strong, Simple Baseline. arXiv:2004.07211

    Pith/arXiv arXiv 2020

  8. [8]

    Caccia, L.; Aljundi, R.; Asadi, N.; Tuytelaars, T.; Pineau, J.; and Belilovsky, E. 2022. New Insights on Reducing Abrupt Representation Change in Online Continual Learning. arXiv:2104.05025

    Pith/arXiv arXiv 2022

  9. [9]

    Chaudhry, A.; Ranzato, M.; Rohrbach, M.; and Elhoseiny, M. 2019. Efficient Lifelong Learning with A-GEM. arXiv:1812.00420

    Pith/arXiv arXiv 2019

  10. [10]

    Cignoni, G.; Cossu, A.; Gomez-Villa, A.; van de Weijer, J.; and Carta, A. 2025. CLA: Latent Alignment for Online Continual Self-Supervised Learning. arXiv:2507.10434

    Pith/arXiv arXiv 2025

  11. [11]

    Deng, L. 2012. The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web]. IEEE Signal Processing Magazine, 29(6): 141--142

    work page 2012

  12. [12]

    Doon, R.; Kumar Rawat, T.; and Gautam, S. 2018. Cifar-10 Classification using Deep Convolutional Neural Network. In 2018 IEEE Punecon, 1--5

    work page 2018

  13. [13]

    Gao, R.; and Liu, W. 2023. Ddgr: Continual learning with deep diffusion-based generative replay. In International Conference on Machine Learning, 10744--10763. PMLR

    work page 2023

  14. [14]

    Guo, Y.; Liu, M.; Yang, T.; and Rosing, T. 2020. Improved Schemes for Episodic Memory-based Lifelong Learning. In Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M.; and Lin, H., eds., Advances in Neural Information Processing Systems, volume 33, 1023--1035. Curran Associates, Inc

    work page 2020

  15. [15]

    He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770--778

    2016

  16. [16]

    I.-K.; and Abdulla, W

    Hirani, G.; Wang, K. I.-K.; and Abdulla, W. 2024. Continual Learning with Self-Organizing Maps: A Novel Group-Based Unsupervised Sequential Training Approach. In 2024 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 1--6

    work page 2024

  17. [17]

    R.; Wang, K

    Hirani, G. R.; Wang, K. I.-K.; and Abdulla, W. H. 2025. A Scalable Unsupervised and Back Propagation Free Learning With SACSOM: A Novel Approach to SOM-Based Architectures. IEEE Transactions on Artificial Intelligence, 6(4): 955--967

    work page 2025

  18. [18]

    Hu, W.; Lin, Z.; Liu, B.; Tao, C.; Tao, Z.; Ma, J.; Zhao, D.; and Yan, R. 2019. Overcoming Catastrophic Forgetting via Model Adaptation. In International Conference on Learning Representations

    work page 2019

  19. [19]

    Hu, W.; Qin, Q.; Wang, M.; Ma, J.; and Liu, B. 2021. Continual Learning by Using Information of Each Class Holistically. Proceedings of the AAAI Conference on Artificial Intelligence, 35(9): 7797--7805

    work page 2021

  20. [20]

    P.; and Welling, M

    Kingma, D. P.; and Welling, M. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114

    Pith/arXiv arXiv 2013

  21. [21]

    P.; and Welling, M

    Kingma, D. P.; and Welling, M. 2019. An Introduction to Variational Autoencoders. Foundations and Trends® in Machine Learning, 12(4): 307–392

    work page 2019

  22. [22]

    A.; Milan, K.; Quan, J.; Ramalho, T.; Grabska-Barwinska, A.; Hassabis, D.; Clopath, C.; Kumaran, D.; and Hadsell, R

    Kirkpatrick, J.; Pascanu, R.; Rabinowitz, N.; Veness, J.; Desjardins, G.; Rusu, A. A.; Milan, K.; Quan, J.; Ramalho, T.; Grabska-Barwinska, A.; Hassabis, D.; Clopath, C.; Kumaran, D.; and Hadsell, R. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13): 3521--3526

    work page 2017

  23. [23]

    Kohonen, T. 1990. The self-organizing map. Proceedings of the IEEE, 78(9): 1464--1480

    work page 1990

  24. [24]

    Lee, S.; Ha, J.; Zhang, D.; and Kim, G. 2020. A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning. In International Conference on Learning Representations

    work page 2020

  25. [25]

    Li, Z.; and Hoiem, D. 2017. Learning without Forgetting. arXiv:1606.09282

    Pith/arXiv arXiv 2017

  26. [26]

    Li, Z.; and Hoiem, D. 2018. Learning without Forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(12): 2935--2947

    work page 2018

  27. [27]

    Lopez-Paz, D.; and Ranzato, M. 2022. Gradient Episodic Memory for Continual Learning. arXiv:1706.08840

    Pith/arXiv arXiv 2022

  28. [28]

    Lopez-Paz, D.; and Ranzato, M. A. 2017. Gradient Episodic Memory for Continual Learning. In Guyon, I.; Luxburg, U. V.; Bengio, S.; Wallach, H.; Fergus, R.; Vishwanathan, S.; and Garnett, R., eds., Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc

    work page 2017

  29. [29]

    Madaan, D.; Yoon, J.; Li, Y.; Liu, Y.; and Hwang, S. J. 2022. Representational Continuity for Unsupervised Continual Learning. arXiv:2110.06976

    Pith/arXiv arXiv 2022

  30. [30]

    Madireddy, S.; Yanguas-Gil, A.; and Balaprakash, P. 2023. Improving performance in continual learning tasks using bio-inspired architectures. In Conference on Lifelong Learning Agents, 992--1008. PMLR

    work page 2023

  31. [31]

    McCloskey, M.; and Cohen, N. 1989. Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem. Psychology of Learning and Motivation - Advances in Research and Theory, 24(C): 109--165

    work page 1989

  32. [32]

    L.; and Kifer, D

    Ororbia, A.; Mali, A.; Giles, C. L.; and Kifer, D. 2022. Lifelong neural predictive coding: Learning cumulatively online without forgetting. Advances in Neural Information Processing Systems, 35: 5867--5881

    work page 2022

  33. [33]

    Ororbia, A. G. 2021. Continual competitive memory: A neural system for online task-free lifelong learning. arXiv preprint arXiv:2106.13300

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  34. [34]

    I.; Kemker, R.; Part, J

    Parisi, G. I.; Kemker, R.; Part, J. L.; Kanan, C.; and Wermter, S. 2019. Continual lifelong learning with neural networks: A review. Neural Networks, 113: 54--71

    work page 2019

  35. [35]

    Pinitas, K.; Chavlis, S.; and Poirazi, P. 2021. Dendritic Self-Organizing Maps for Continual Learning. arXiv:2110.13611

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  36. [36]

    Rajasegaran, J.; Hayat, M.; Khan, S.; Khan, F.; Shao, L.; and Yang, M.-H. 2020. An Adaptive Random Path Selection Approach for Incremental Learning

    work page 2020

  37. [37]

    W.; and Hadsell, R

    Rao, D.; Visin, F.; Rusu, A.; Pascanu, R.; Teh, Y. W.; and Hadsell, R. 2019. Continual Unsupervised Representation Learning. In Wallach, H.; Larochelle, H.; Beygelzimer, A.; d Alch\' e -Buc, F.; Fox, E.; and Garnett, R., eds., Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc

    work page 2019

  38. [38]

    Rebuffi, S.-A.; Kolesnikov, A.; Sperl, G.; and Lampert, C. H. 2017. iCaRL: Incremental Classifier and Representation Learning. arXiv:1611.07725

    Pith/arXiv arXiv 2017

  39. [39]

    Roy, D.; Panda, P.; and Roy, K. 2019. Tree-CNN: A Hierarchical Deep Convolutional Neural Network for Incremental Learning. arXiv:1802.05800

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  40. [40]

    A.; Rabinowitz, N

    Rusu, A. A.; Rabinowitz, N. C.; Desjardins, G.; Soyer, H.; Kirkpatrick, J.; Kavukcuoglu, K.; Pascanu, R.; and Hadsell, R. 2022. Progressive Neural Networks. arXiv:1606.04671

    Pith/arXiv arXiv 2022

  41. [41]

    J.; and Muller, K.-R

    Samek, W.; Montavon, G.; Lapuschkin, S.; Anders, C. J.; and Muller, K.-R. 2021. Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications. Proceedings of the IEEE, 109(3): 247–278

    work page 2021

  42. [42]

    M.; Grabska-Barwinska, A.; Teh, Y

    Schwarz, J.; Luketina, J.; Czarnecki, W. M.; Grabska-Barwinska, A.; Teh, Y. W.; Pascanu, R.; and Hadsell, R. 2018. Progress & Compress: A scalable framework for continual learning. arXiv:1805.06370

    Pith/arXiv arXiv 2018

  43. [43]

    K.; Kim, J.; and Kim, J

    Shin, H.; Lee, J. K.; Kim, J.; and Kim, J. 2017. Continual Learning with Deep Generative Replay. arXiv:1705.08690

    Pith/arXiv arXiv 2017

  44. [44]

    Simonyan, K.; and Zisserman, A. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv:1409.1556

    Pith/arXiv arXiv 2015

  45. [45]

    Sun, W.; Li, Q.; Zhang, J.; Wang, D.; Wang, W.; and Geng, Y. 2022. Exemplar-free Class Incremental Learning via Discriminative and Comparable One-class Classifiers. CoRR, abs/2201.01488

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  46. [46]

    Thrun, S. 1998. Lifelong learning algorithms. In Learning to learn, 181--209. Springer

    work page 1998

  47. [47]

    Vaidya, H.; Desell, T.; and Ororbia, A. 2021. Reducing Catastrophic Forgetting in Self Organizing Maps with Internally-Induced Generative Replay. CoRR, abs/2112.04728

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  48. [48]

    M.; and Tolias, A

    van de Ven, G. M.; and Tolias, A. S. 2019. Generative replay with feedback connections as a general strategy for continual learning. arXiv:1809.10635

    Pith/arXiv arXiv 2019

  49. [49]

    M.; Tuytelaars, T.; and Tolias, A

    van de Ven, G. M.; Tuytelaars, T.; and Tolias, A. S. 2022. Three types of incremental learning. Nature Machine Intelligence, 4(12): 1185--1197

    work page 2022

  50. [50]

    Wang, L.; Zhang, X.; Su, H.; and Zhu, J. 2024. A Comprehensive Survey of Continual Learning: Theory, Method and Application. arXiv:2302.00487

    Pith/arXiv arXiv 2024

  51. [51]

    Xiao, H.; Rasul, K.; and Vollgraf, R. 2017. Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms. arXiv:1708.07747

    Pith/arXiv arXiv 2017

  52. [52]

    Yi, H.; Shiyu, S.; Xiusheng, D.; and Zhigang, C. 2016. A study on Deep Neural Networks framework. In 2016 IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC), 1519--1522

    work page 2016

  53. [53]

    Ying, H.; Song, M.; Tang, Y.; Xiao, S.; and Xiao, Z. 2024. Enhancing Deep Neural Network Training Efficiency and Performance through Linear Prediction. arXiv:2310.10958

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  54. [54]

    Yu, X.; Rosing, T.; and Guo, Y. 2024. Evolve: Enhancing Unsupervised Continual Learning With Multiple Experts. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2366--2377

    work page 2024

  55. [55]

    Zeng, G.; Chen, Y.; Cui, B.; and Yu, S. 2018. Continuous Learning of Context-dependent Processing in Neural Networks. CoRR, abs/1810.01256

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  56. [56]

    Zenke, F.; Poole, B.; and Ganguli, S. 2017. Continual Learning Through Synaptic Intelligence. arXiv:1703.04200

    Pith/arXiv arXiv 2017

  57. [57]

    Zhao, X.; Wang, H.; Huang, W.; and Lin, W. 2024. A Statistical Theory of Regularization-Based Continual Learning. arXiv:2406.06213

    Pith/arXiv arXiv 2024

  58. [58]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  59. [59]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.