Pith. sign in

REVIEW 2 major objections 3 minor 52 references

Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis

T0 review · 2 major / 3 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A streaming linear classifier on frozen deep features beats incremental learners that update the whole network.

desk verdict Simple, cheap, strong streaming-learning baseline—but the abstract's ImageNet claim is contradicted by the paper's own Table 1. read the letter →

arxiv 1909.01520 v3 pith:HWXLIBVA submitted 2019-09-04 cs.LG cs.CVstat.ML

Tyler L. Hayes , Christopher Kanan This is my paper · ORCID
classification cs.LGcs.CVstat.ML
keywords streaminglearninglineardiscriminantanalysiscatastrophicforgettingcontinualincrementalImageNetCORe50online
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that catastrophic forgetting in deep networks can be largely sidestepped by keeping the CNN's hidden layers fixed and learning only a streaming linear discriminant analysis classifier on top. The authors show that this simple output-layer-only approach outperforms both incremental batch learners that update the entire network and prior streaming learners on ImageNet and CORe50. If true, it means that for many continual learning problems, expensive replay buffers, distillation losses, and network updates may be unnecessary, and that a small, fast, memory-light classifier can match or exceed them. The method can be tested at any moment during a single pass through the data, which fits real-time and embedded applications.

What carries the argument

The central object is streaming linear discriminant analysis applied to the output layer of a CNN. SLDA maintains one running mean vector $\mu_k$ per class, a shared covariance matrix $\Sigma$ (optionally updated online), and a shrinkage-regularized precision matrix $\Lambda = [(1-\varepsilon)\Sigma + \varepsilon I]^{-1}$. Predictions come from the linear scores $w_k = \Lambda \mu_k$ and $b_k = -\frac{1}{2}(\mu_k \cdot \Lambda \mu_k)$, which give the closest-Gaussian decision rule and are equivalent in form to a softmax classifier under a Gaussian assumption. This machinery lets the classifier update in a single pass, instance by instance, with constant memory per class plus one shared covariance matrix.

What would settle it

Run Deep SLDA on a fine-grained or domain-shifted benchmark, for example classifying bird species or medical images with a feature extractor pretrained on ImageNet, and compare it against an incremental batch learner that updates the full network. If accuracy drops well below the batch learner's, the claim that a frozen feature extractor plus streaming LDA suffices for continual learning is falsified for those settings.

Watch

Extended reading notes

Core claim

The paper claims that Deep Streaming Linear Discriminant Analysis (Deep SLDA), which keeps the feature extractor $G(\cdot)$ fixed and updates only a linear mapping $F(\cdot)$ online, achieves normalized accuracy $\Omega_{all}$ of 0.752 on ImageNet and 0.958 or better on CORe50 under challenging class orderings, surpassing iCaRL, End-to-End, and ExStream. SLDA resists forgetting because it stores one running mean per class, so updating one class does not alter another class's mean. The shared covariance matrix can be frozen after initialization or updated online; the plastic version gives the best results, especially when the initial covariance estimate is based on few samples. Compared with iCaRL and End-to-End, Deep SLDA is over 100 times faster to train and uses roughly 1,000 times less memory, and it does not require task labels at inference time.

Load-bearing premise

The method depends on the frozen CNN features staying discriminative for every class seen later, and on each class's feature distribution being reasonably well approximated by a single Gaussian with a shared covariance matrix.

Editorial extensions

If this is right

  • If the central claim holds, incremental batch learning methods that store and replay exemplars may be over-engineered: a streaming linear classifier on fixed features can beat them even when those methods are allowed multiple epochs over each batch.
  • Deep SLDA provides a practical recipe for on-device continual learning: freeze a pretrained feature extractor, initialize a shared covariance estimate, and update class means and covariance in constant time per sample.
  • Because SLDA needs no task labels at test time, it can be deployed in settings where task identity is unknown, unlike several regularization-based continual learning methods.
  • The result suggests that the main bottleneck for continual learning is not the classifier but the quality and stability of the feature representation, so improving frozen features (e.g., through better pretraining or self-supervised learning) could directly raise streaming accuracy.
  • The small gap between SLDA with a plastic covariance matrix and an offline upper bound on CORe50 indicates that a well-initialized linear model can come close to full offline training on temporally ordered visual data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to feed Deep SLDA with features from self-supervised or unsupervised pretraining rather than a supervised base-initialization phase; if the paper's logic is right, the method should still work well even when no labeled base classes overlap with the target classes.
  • If frozen features are the real driver, then a similar streaming Gaussian classifier should also work on other modalities such as audio or text embeddings, provided the per-class Gaussian assumption holds; this is an extrapolation the paper does not test.
  • The paper's domain-transfer experiment suggests that the covariance matrix can be initialized from a different dataset or even a matrix of ones and still yield competitive results, implying that the method's robustness to initialization is broader than the main experiments show.
  • A likely failure mode is fine-grained or highly domain-shifted tasks where frozen ImageNet features are not discriminative enough; testing Deep SLDA there would clarify how much of the result depends on feature quality.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. Deep SLDA couples a frozen deep CNN feature extractor with a streaming linear discriminant analysis classifier: it maintains per-class running means and a shared covariance matrix that can be held fixed or updated online, and it classifies via shrinkage-regularized LDA. The authors evaluate two SLDA variants against streaming baselines (Fine-Tuning, ExStream) and incremental batch baselines (iCaRL, End-to-End) on ImageNet ILSVRC-2012 and CORe50, reporting the normalized Ω_all metric (Eq. 8), final accuracies, memory/compute costs, and an additional comparison to task-label-requiring regularization methods. The central claim is that Deep SLDA outperforms both incremental batch and streaming learners on both datasets.

Significance. If the empirical results are taken at face value, the paper provides a valuable, simple baseline: a streaming linear classifier on deep, frozen features can be competitive with, and often beat, much more complex rehearsal- and distillation-based continual learning methods, while using orders of magnitude less memory and compute. The CORe50 experiments are multi-run and the code is publicly available, which support reproducibility. The main caveats are that the headline ImageNet claim is not consistent with the paper's own primary metric, and the streaming evaluation protocol appears to replay the base-initialization data for the streaming models, which weakens the fairness of the comparison to incremental batch learners.

major comments (2)
  1. [Abstract; Sec. 4.4.1; Table 1] The abstract states that Deep SLDA 'outperform[s] both incremental batch learning and streaming learning algorithms on both ImageNet ILSVRC-2012 and CORe50', but this is not supported by the paper's primary metric on ImageNet. In Table 1, End-to-End (an incremental batch method) achieves Ω_all = 0.780, while SLDA with a plastic covariance achieves 0.752. The text of Sec. 4.4.1 only claims SLDA 'ends with a higher accuracy than End-to-End', and Sec. 5 claims Ω_all superiority over iCaRL only. Thus the abstract either implicitly switches from Ω_all to final top-5 accuracy, or overstates the result. In addition, the ImageNet End-to-End numbers are taken from the original authors without re-running (Sec. 4.4) and the ImageNet experiment is a single run with no error bars, so the claimed advantage over End-to-End is not robustly established. Please reconcile the claims with the reported Ω_all values and either re-run the End-to-End baseline or clearly qualify the comparison.
  2. [Sec. 4.1; Sec. 4.3] The streaming protocol appears to replay the base-initialization data for the streaming models. Sec. 4.1 states that 'ExStream and SLDA re-start a streaming learning phase from the beginning of the dataset' after base initialization on 100 ImageNet classes (Sec. 4.2) or 1,200 CORe50 samples (Sec. 4.3). This means the first 100 ImageNet classes (or first 1,200 CORe50 samples) are seen twice by SLDA and ExStream: once during base initialization and once during the streaming pass, whereas iCaRL and End-to-End only see the base classes during base initialization and subsequently through a limited exemplar set. Since Ω_all (Eq. 8) averages over all evaluation points, the earlier evaluations, which are inflated by the double pass, bias the comparison in favor of the streaming methods. The paper does not address this asymmetry. Please either adopt a base-initialization scheme that does not replay the same samples (e.g., the ImageNet-to-CORe50 transfer setting in Table 3), or provide an analysis of the impact of this double pass on the reported comparisons.
minor comments (3)
  1. [Sec. 4.4.1] The sentence 'Updating the SLDA covariance matrix only yielded marginal improvement' is specific to ImageNet; on CORe50 (Table 1), the plastic covariance variant gives a large improvement. Please qualify the statement to avoid confusion.
  2. [Table 1] The table does not report the final top-5 accuracy for ImageNet, even though the abstract and Fig. 1 emphasize final accuracy. Adding a final-accuracy column alongside Ω_all would make the metric switch explicit and easier to verify.
  3. [Table 1] The column labels for the four CORe50 orderings are difficult to parse in the rendered table; please ensure the orderings (IID, CLS IID, INST, CLS INST) are clearly labeled and cross-referenced to Sec. 4.2.

Circularity Check

0 steps flagged · score 0.0 of 10

No load-bearing circularity; Deep SLDA's result is an independently evaluated empirical comparison.

full rationale

The manuscript is an empirical methods paper rather than a derivation. The claimed result—that Deep SLDA achieves high normalized accuracy—comes from running the algorithm on held-out test splits, and no parameter in Sec. 3 is fitted to the evaluation metric. The shrinkage parameter is fixed at epsilon = 10^-4, and the covariance initialization uses the same base set allowed to all baselines, so this is not a fitted input renamed as a prediction. The same-author prior work invoked is ExStream [23], used as a comparison baseline and implemented in this paper, not as a premise that entails SLDA's accuracy; SLDA itself is attributed to Pang et al. [40], the covariance update to Dasgupta and Hsu [14], and feature-transferability to Yosinski et al. [51]. No uniqueness theorem from the authors is invoked, no ansatz is justified solely by self-citation, and no evaluation quantity is used to define the model. For completeness, the abstract's blanket wording conflicts with Table 1 (End-to-End Omega_all = 0.780 > SLDA plastic Sigma = 0.752 on ImageNet), and the End-to-End ImageNet numbers were borrowed from the original authors rather than re-run; these are correctness or robustness concerns, not circularity, and they do not change the circularity score.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The method rests on two domain assumptions: transferability of frozen deep features and Gaussian class-conditional feature distributions for LDA. The only manually set free parameter is the shrinkage epsilon. No invented entities are introduced.

free parameters (1)
  • shrinkage parameter epsilon = 1e-4
    Used in the precision matrix Lambda = [(1-epsilon)Sigma + epsilon I]^{-1} to ensure invertibility. Chosen by hand, constant across experiments; not fitted to performance. Central to the classification rule but likely not highly sensitive.
assumptions (3)
  • domain assumption CNN features learned on a base dataset are sufficiently transferable that freezing G does not hurt incremental classification performance.
    Stated in Sec. 3 (filters in early layers vary little and are highly transferable, citing Yosinski et al.). The entire streaming phase keeps theta_G fixed; if features are not transferable, the method's accuracy collapses. Empirically tested in Fig. 3 but not guaranteed for new domains.
  • domain assumption The feature distribution of each class is approximately Gaussian with a shared covariance matrix, justifying the LDA decision rule.
    The prediction rule uses Mahalanobis distance based on class means and the shared covariance (Eq. 6-7). If the Gaussian assumption is strongly violated, the classifier is not optimal. The paper does not test this directly, but the strong empirical results suggest it holds for the datasets used.
  • standard math The online covariance update from Dasgupta and Hsu (2007) yields a consistent estimate of the true covariance in the streaming setting.
    Eq. 4-5 is used without derivation and cited to [14]. The method's plastic-covariance variant depends on this update being numerically stable, particularly in the class instance orderings.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis." pith.science (2026). https://pith.science/paper/HWXLIBVA

@misc{pith2026190901520,
  author       = {Pith},
  title        = {Pith review of: Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HWXLIBVA}},
  note         = {Machine review of arXiv:1909.01520}
}
read the original abstract

When an agent acquires new information, ideally it would immediately be capable of using that information to understand its environment. This is not possible using conventional deep neural networks, which suffer from catastrophic forgetting when they are incrementally updated, with new knowledge overwriting established representations. A variety of approaches have been developed that attempt to mitigate catastrophic forgetting in the incremental batch learning scenario, where a model learns from a series of large collections of labeled samples. However, in this setting, inference is only possible after a batch has been accumulated, which prohibits many applications. An alternative paradigm is online learning in a single pass through the training dataset on a resource constrained budget, which is known as streaming learning. Streaming learning has been much less studied in the deep learning community. In streaming learning, an agent learns instances one-by-one and can be tested at any time, rather than only after learning a large batch. Here, we revisit streaming linear discriminant analysis, which has been widely used in the data mining research community. By combining streaming linear discriminant analysis with deep learning, we are able to outperform both incremental batch learning and streaming learning algorithms on both ImageNet ILSVRC-2012 and CORe50, a dataset that involves learning to classify from temporally ordered samples.

Figures

Figures reproduced from arXiv: 1909.01520 by the authors.

Figure 1
Figure 1. Learning curve for incremental ImageNet. Our [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Streaming learning requires agents to learn sample-by-sample in real time, making it better suited for embedded [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Final SLDA accuracy on ImageNet as a function [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 49 canonical work pages

  1. [1]

    Memory retention–the synaptic stability versus plasticity dilemma

    Wickliffe C Abraham and Anthony Robins. Memory retention–the synaptic stability versus plasticity dilemma. Trends in Neurosciences, 2005. 1

    work page 2005

  2. [2]

    On demand classification of data streams

    Charu C Aggarwal, Jiawei Han, Jianyong Wang, and Philip S Yu. On demand classification of data streams. In ACM SIGKDD International Conference on Knowledge Discov- ery and Data Mining , pages 503–508. ACM, 2004. 1

    work page 2004

  3. [3]

    Memory aware synapses: Learning what (not) to forget

    Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In ECCV, pages 139–154, 2018. 3, 7

    work page 2018

  4. [4]

    Adaptive learning from evolving data streams

    Albert Bifet and Ricard Gavald `a. Adaptive learning from evolving data streams. In International Symposium on In- telligent Data Analysis , pages 249–260. Springer, 2009. 1, 3

    work page 2009

  5. [5]

    New ensemble methods for evolving data streams

    Albert Bifet, Geoff Holmes, Bernhard Pfahringer, Richard Kirkby, and Ricard Gavald `a. New ensemble methods for evolving data streams. In ACM SIGKDD International Con- ference on Knowledge Discovery and Data Mining , pages 139–148. ACM, 2009. 3

    work page 2009

  6. [6]

    Reacting to dif- ferent types of concept drift: The accuracy updated ensemble algorithm

    Dariusz Brzezinski and Jerzy Stefanowski. Reacting to dif- ferent types of concept drift: The accuracy updated ensemble algorithm. IEEE Trans. on Neural Networks and Learning Systems, 25(1):81–94, 2014. 3

    work page 2014

  7. [7]

    Fuzzy artmap: A neu- ral network architecture for incremental supervised learning of analog multidimensional maps

    Gail A Carpenter, Stephen Grossberg, Natalya Markuzon, John H Reynolds, and David B Rosen. Fuzzy artmap: A neu- ral network architecture for incremental supervised learning of analog multidimensional maps. IEEE Trans. on Neural Networks, 3(5):698–713, 1992. 3

    work page 1992

  8. [8]

    Artmap: Supervised real-time learning and classification of nonstationary data by a self-organizing neural network

    Gail A Carpenter, Stephen Grossberg, and John H Reynolds. Artmap: Supervised real-time learning and classification of nonstationary data by a self-organizing neural network. Neu- ral Networks, 4(5):565–588, 1991. 3

    work page 1991

Show all 52 references
  1. [9]

    End-to-end incremen- tal learning

    Francisco M Castro, Manuel J Mar´ın-Jim´enez, Nicol´as Guil, Cordelia Schmid, and Karteek Alahari. End-to-end incremen- tal learning. In ECCV, pages 233–248, 2018. 2, 3, 4, 5, 6, 7

    2018

  2. [10]

    Riemannian walk for incremental learning: Understanding forgetting and intransigence

    Arslan Chaudhry, Puneet K Dokania, Thalaiyasingam Ajan- than, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In ECCV, pages 532–547, 2018. 2, 3, 7

    2018

  3. [11]

    Efficient lifelong learning with A-GEM

    Arslan Chaudhry, Marc’Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with A-GEM. In ICLR, 2019. 3, 5, 7, 8

    2019

  4. [12]

    Shrinkage algorithms for mmse covariance estimation

    Yilun Chen, Ami Wiesel, Yonina C Eldar, and Alfred O Hero. Shrinkage algorithms for mmse covariance estimation. IEEE Trans. on Signal Processing, 58(10):5016–5029, 2010. 6

    2010

  5. [13]

    Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting

    Robert Coop, Aaron Mishtal, and Itamar Arel. Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE Trans. on Neural Networks and Learning Systems, 24(10):1623–1634, 2013. 3

    2013

  6. [14]

    On-line estimation with the multivariate gaussian distribution

    Sanjoy Dasgupta and Daniel Hsu. On-line estimation with the multivariate gaussian distribution. In International Con- ference on Computational Learning Theory , pages 278–292. Springer, 2007. 4

    2007

  7. [15]

    Learning without mem- orizing

    Prithviraj Dhar, Rajat Vikram Singh, Kuan-Chuan Peng, Ziyan Wu, and Rama Chellappa. Learning without mem- orizing. In CVPR, pages 5138–5146, 2019. 3

    2019

  8. [16]

    Mining high-speed data streams

    Pedro Domingos and Geoff Hulten. Mining high-speed data streams. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages 71–80. ACM,

  9. [17]

    Towards robust evaluations of continual learning

    Sebastian Farquhar and Yarin Gal. Towards robust evaluations of continual learning. arXiv preprint arXiv:1805.09733, 2018. 3

    2018 arXiv

  10. [18]

    Pathnet: Evolution channels gradient descent in super neural networks

    Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A Rusu, Alexander Pritzel, and Daan Wierstra. Pathnet: Evolution channels gradient descent in super neural networks. arXiv preprint arXiv:1701.08734,

    arXiv

  11. [19]

    A survey of classification methods in data streams

    Mohamed Medhat Gaber, Arkady Zaslavsky, and Shonali Krishnaswamy. A survey of classification methods in data streams. In Data Streams, pages 39–59. Springer, 2007. 1, 2, 3

    2007

  12. [20]

    Knowledge discovery from data streams

    Jo˜ao Gama. Knowledge discovery from data streams. Chap- man and Hall/CRC, 2010. 1

    2010

  13. [21]

    Decision trees for mining data streams

    Jo˜ao Gama, Ricardo Fernandes, and Ricardo Rocha. Decision trees for mining data streams. Intelligent Data Analysis , 10(1):23–45, 2006. 3

    2006

  14. [22]

    On evaluating stream learning algorithms

    Jo˜ao Gama, Raquel Sebasti˜ao, and Pedro Pereira Rodrigues. On evaluating stream learning algorithms. Machine learning, 90(3):317–346, 2013. 1

    2013

  15. [23]

    Memory efficient experience replay for streaming learning

    Tyler L Hayes, Nathan D Cahill, and Christopher Kanan. Memory efficient experience replay for streaming learning. In ICRA, 2019. 1, 3, 4, 5, 6

    2019

  16. [24]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016. 3, 4

    2016

  17. [25]

    Distill- ing the knowledge in a neural network

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distill- ing the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015. 3

    2015 arXiv

  18. [26]

    Lifelong learning via progressive distillation and retrospection

    Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Lifelong learning via progressive distillation and retrospection. In ECCV, pages 437–452, 2018. 3

    2018

  19. [27]

    Mining time-changing data streams

    Geoff Hulten, Laurie Spencer, and Pedro Domingos. Mining time-changing data streams. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages 97–106. ACM, 2001. 1, 3

    2001

  20. [28]

    Learn- ing model trees from evolving data streams

    Elena Ikonomovska, Jo˜ao Gama, and Saˇso Dˇzeroski. Learn- ing model trees from evolving data streams. Data Mining and Knowledge Discovery, 23(1):128–168, 2011. 3

    2011

  21. [29]

    Revisiting distillation and incremental classifier learning

    Khurram Javed and Faisal Shafait. Revisiting distillation and incremental classifier learning. ACCV, 2018. 3

    2018

  22. [30]

    FearNet: Brain- inspired model for incremental learning

    Ronald Kemker and Christopher Kanan. FearNet: Brain- inspired model for incremental learning. In ICLR, 2018. 2, 3

    2018

  23. [31]

    Measuring catastrophic for- getting in neural networks

    Ronald Kemker, Marc McClure, Angelina Abitino, Tyler L Hayes, and Christopher Kanan. Measuring catastrophic for- getting in neural networks. In AAAI, pages 3390–3398, 2018. 2, 3, 4, 5

    2018

  24. [32]

    Overcoming catastrophic forgetting in neural networks

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting in...

    2017

  25. [33]

    A simple unified framework for detecting out-of-distribution samples and adversarial attacks

    Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In NeurIPS, pages 7167– 7177, 2018. 3

    2018

  26. [34]

    Over- coming catastrophic forgetting with unlabeled data in the wild

    Kibok Lee, Kimin Lee, Jinwoo Shin, and Honglak Lee. Over- coming catastrophic forgetting with unlabeled data in the wild. In ICCV, pages 312–321, 2019. 3

    2019

  27. [35]

    Core50: a new dataset and benchmark for continuous object recognition

    Vincenzo Lomonaco and Davide Maltoni. Core50: a new dataset and benchmark for continuous object recognition. In Conference on Robot Learning, pages 17–26, 2017. 4, 5

    2017

  28. [36]

    Gradient episodic memory for continual learning

    David Lopez-Paz and Marc’Aurelio Ranzato. Gradient episodic memory for continual learning. In NeurIPS, pages 6467–6476, 2017. 3, 5

    2017

  29. [37]

    Catastrophic inter- ference in connectionist networks: The sequential learning problem

    Michael McCloskey and Neal J Cohen. Catastrophic inter- ference in connectionist networks: The sequential learning problem. Psychology of Learning and Motivation , 24:109– 165, 1989. 1

    1989

  30. [38]

    Selection and sorting with limited storage

    J Ian Munro and Mike S Paterson. Selection and sorting with limited storage. Theoretical Computer Science, 12(3):315– 323, 1980. 3

    1980

  31. [39]

    Learning to remember: A synaptic plasticity driven framework for continual learning

    Oleksiy Ostapenko, Mihai Puscas, Tassilo Klein, Patrick J¨ahnichen, and Moin Nabi. Learning to remember: A synaptic plasticity driven framework for continual learning. In CVPR,

  32. [40]

    Incre- mental linear discriminant analysis for classification of data streams

    Shaoning Pang, Seiichi Ozawa, and Nikola Kasabov. Incre- mental linear discriminant analysis for classification of data streams. IEEE Trans. on Systems, Man, and Cybernetics, part B (Cybernetics), 35(5):905–914, 2005. 2, 3

    2005

  33. [41]

    Lifelong learning of spatiotemporal representa- tions with dual-memory recurrent self-organization

    German I Parisi, Jun Tani, Cornelius Weber, and Stefan Wermter. Lifelong learning of spatiotemporal representa- tions with dual-memory recurrent self-organization. Frontiers in Neurorobotics, 12:78, 2018. 5, 8

    2018

  34. [42]

    Self-taught learning: transfer learning from unlabeled data

    Rajat Raina, Alexis Battle, Honglak Lee, Benjamin Packer, and Andrew Y Ng. Self-taught learning: transfer learning from unlabeled data. In ICML, pages 759–766, 2007. 8

    2007

  35. [43]

    Scalable and efficient multi-label classification for evolving data streams

    Jesse Read, Albert Bifet, Geoff Holmes, and Bernhard Pfahringer. Scalable and efficient multi-label classification for evolving data streams. Machine Learning, 88(1-2):243–272,

  36. [44]

    icarl: Incremental classifier and representation learning

    Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In CVPR, 2017. 2, 3, 4, 5, 6, 7, 8

    2017

  37. [45]

    On- line structured laplace approximations for overcoming catas- trophic forgetting

    Hippolyt Ritter, Aleksandar Botev, and David Barber. On- line structured laplace approximations for overcoming catas- trophic forgetting. In NeurIPS, pages 3738–3748, 2018. 3

    2018

  38. [46]

    Berg, and Li Fei-Fei

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. IJCV, 115(3):211–252, 2015. 5

    2015

  39. [47]

    Overcoming catastrophic forgetting with hard atten- tion to the task

    Joan Serra, Didac Suris, Marius Miron, and Alexandros Karat- zoglou. Overcoming catastrophic forgetting with hard atten- tion to the task. In ICML, pages 4555–4564, 2018. 3

    2018

  40. [48]

    Min- ing concept-drifting data streams using ensemble classifiers

    Haixun Wang, Wei Fan, Philip S Yu, and Jiawei Han. Min- ing concept-drifting data streams using ensemble classifiers. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages 226–235. ACM, 2003. 3

    2003

  41. [49]

    Gaussian artmap: A neural network for fast incremental learning of noisy multidimensional maps

    James R Williamson. Gaussian artmap: A neural network for fast incremental learning of noisy multidimensional maps. Neural Networks, 9(5):881–897, 1996. 3

    1996

  42. [50]

    Memory replay gans: learning to generate images from new categories without forgetting

    Chenshen Wu, Luis Herranz, Xialei Liu, Yaxing Wang, Joost van de Weijer, and Bogdan Raducanu. Memory replay gans: learning to generate images from new categories without forgetting. In NeurIPS, 2018. 3

    2018

  43. [51]

    How transferable are features in deep neural networks? In NeurIPS, pages 3320–3328, 2014

    Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? In NeurIPS, pages 3320–3328, 2014. 4

    2014

  44. [52]

    Continual learning through synaptic intelligence

    Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In ICML, pages 3987– 3995, 2017. 2, 3, 7

    2017

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.