Pith. sign in

REVIEW 6 major objections 7 minor 46 references

Revised Regularization for Efficient Continual Learning through Correlation-Based Parameter Update in Bayesian Neural Networks

T0 review · 6 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that its revised Bayesian regularizer, which learns parameter distributions and splits the weight space into common and distinctive subspaces, outperforms prior continual-learning methods while storing far fewer parameters.

desk verdict Interesting empirical continual-learning recipe whose theoretical justification and storage claims both break under scrutiny; worth a major revision, not a desk reject. read the letter →

arxiv 2411.14202 v1 pith:523G7N5V submitted 2024-11-21 cs.LG cs.CV

Sanchar Palit , Biplab Banerjee , Subhasis Chaudhuri This is my paper · ORCID
classification cs.LGcs.CV
keywords continuallearningBayesianneuralnetworksvariationalinferencecatastrophicforgettingknowledgetransferparameterdistributioncommonanddistinctivesubspacesimportance-weightedELBO
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a Bayesian neural network can learn a sequence of tasks without catastrophic forgetting by replacing the standard KL-divergence regularizer with revised penalties that control how the mean and variance of each parameter move. It adds a small Parameter Learning Network that generates dense-layer weights from a shared distribution, plus a representation matrix for convolutional layers, so old-task parameters do not have to be stored in full. The parameter space is split into common and distinctive subspace bases via SVD, and an importance-weighted ELBO aligns parameters with data across sessions. If the claims hold, continual learners get higher average accuracy and backward knowledge transfer while using far less memory than standard Bayesian continual learning.

What carries the argument

The central objects are the Parameter Learning Network (PLN), a small network $G_\psi: \mathbb{R}^k \to \mathbb{R}^k$ that generates all dense-layer weight tensors from one learned distribution under the assumption that all $k$-dimensional tensors of size $m$ follow a shared parameter trajectory, and the revised regularizer that replaces the KL term with $\|\mu_t-\mu_{t-1}\|_2^2 + \left\|\frac{(\mu_t-\mu_{t-1})^2}{\sigma_t^2}\right\|_2^2 + \|\sigma_{t-1}\|_1 + \left\|\frac{\sigma_t-\sigma_{t-1}}{\sigma_{t-1}}\right\|_1$. For convolutional layers, flattened masks are stored in representation matrices, and SVD partitions the parameter space into common and distinctive subspace bases. The importance-weighted ELBO $L_{COR}$ uses the ratio $R_{t,M} = \frac{1}{M}\sum_i \frac{Q(W_{t-1}^i, D_{t-1})}{Q(W_t^i)}$ to correlate parameters with past data, and the correlation threshold decides when basis reuse is safe for backward knowledge transfer.

What would settle it

Measure, during actual continual training on one of the reported benchmarks, the pairwise distances or cosine similarities of the per-node $k$-dimensional weight vectors in a dense layer across sessions. If those vectors do not move as a tight cluster, with per-session displacement variance comparable to the mean displacement, then a single Parameter Learning Network cannot faithfully regenerate all tensors from one distribution, and the reported storage reduction would not transfer to settings where trajectories diverge.

Watch

Extended reading notes

Core claim

The central claim is that the proposed ECL-RR outperforms state-of-the-art Bayesian and non-Bayesian continual learning methods on Permuted MNIST (97.8% accuracy), Split MNIST (99.7%), alternating CIFAR 10/100 (80.16%), and the 8-mixture dataset (84.58%), while storing only the Parameter Learning Network and subspace bases instead of full per-session parameter tensors. The mechanism is a revised loss in which the KL term is replaced by four penalties: two on mean movement (raw magnitude and magnitude relative to current variance) and two on variance (sparsity of past variance and boundedness of relative variance change). A correlation-based importance-weighted ELBO then identifies which subspace bases are common across sessions and which are distinctive, enabling backward transfer: reusing common bases for the current task and adding new distinctive bases when the correlation check fails.

Load-bearing premise

The whole storage reduction rests on the assumption that all weight vectors in a layer drift in nearly the same way during training, so one small network can generate all of them; the paper offers visual evidence but no quantitative proof.

Editorial extensions

If this is right

  • If ECL-RR works as claimed, replacing KL with mean-variance penalties should reduce catastrophic forgetting under sparse past-data access, because the gradient issues tied to KL's scale factor are bypassed.
  • Parameter storage for Bayesian continual learning can fall from twice the network size plus per-session copies to the size of the PLN plus subspace bases, with little or no accuracy loss on the reported benchmarks.
  • The common and distinctive subspace partition provides backward knowledge transfer when the correlation between the current loss and the previous-session loss exceeds a threshold.
  • The importance-weighted ELBO gives a tighter lower bound than plain ELBO, offering a principled way to align parameters with earlier-session data without replaying all past samples.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The shared-trajectory assumption could be tested on transformer or large-scale vision continual learning, where nodes are less symmetric; if trajectories diverge per node, a single PLN per layer would need to be replaced by per-cluster generators, degrading storage savings gracefully.
  • The common and distinctive subspace idea can be recast as a low-rank plus sparse decomposition of the representation matrix, where backward transfer corresponds to keeping the common low-rank component fixed and appending sparse novel components.
  • The variance-sparsity regularizer implies an interpretable uncertainty budget: nodes that become certain early stop moving, while nodes that remain uncertain stay learnable; one could predict per-task forgetting by tracking which nodes stay in the high-variance regime.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 7 minor

Summary. The paper proposes a Bayesian variational-inference method for continual learning, called ECL-RR. Its components are: (i) a revised regularizer in Eq. (4) that replaces the KL term with positive penalties on the mean difference and on variance magnitudes; (ii) a Parameter Learning Network (PLN) that is claimed to store the dense-layer posterior compactly by exploiting a shared parameter trajectory across output nodes (Sec. 4.1); (iii) an SVD-based decomposition of representation matrices into common and distinctive subspaces with a claimed backward-knowledge-transfer condition (Eq. 7); and (iv) an importance-weighted LCOR term (Eq. 5) meant to capture parameter-data correspondence. Experiments report state-of-the-art average accuracy and backward transfer on Permuted MNIST, Split MNIST, alternating CIFAR 10/100, and an 8-mixture dataset (Table 1).

Significance. If the theoretical and architectural claims were correct, the paper would address two genuinely important problems in continual learning: compact storage of Bayesian posteriors and backward knowledge transfer. The broad benchmark comparison is a useful feature, and the GTM visualizations are a reasonable way to inspect parameter drift. However, the paper does not provide machine-checked proofs, reproducible code, or storage measurements, and several load-bearing derivations and architectural premises are incorrect. As it stands, the efficient-storage contribution and the regularization contribution are not established on the evidence in the manuscript.

major comments (6)
  1. [Gradient Analysis preceding Eq. (4)] The displayed gradient is not the gradient of KL(Q(W_t|theta_t) || Q(W_{t-1}|theta_{t-1})). For diagonal Gaussian variational distributions, the derivative with respect to mu_t is Sigma_{t-1}^{-1}(mu_t - mu_{t-1}), not sigma_t^{-1}(mu_t - mu_{t-1}); the variance derivative also does not contain the mean-difference term shown, because the mean term in the KL uses the previous covariance. The two regularization terms in Eq. (4) are therefore not derived from the stated KL gradient, and the paper's explanation that sigma_t controls the sensitivity of the mean update is based on a wrong derivative.
  2. [Paragraph after Eq. (4)] The claim that positive regularizers ensure the loss remains a lower bound of the log evidence is incorrect as stated. The objective in Eq. (3) is the negative ELBO, which is an upper bound on -log P(D), not a lower bound on log P(D). Adding arbitrary positive terms to Eq. (3) breaks any ELBO lower-bound interpretation unless a separate derivation is supplied. This invalidates the theoretical justification given for the revised objective.
  3. [Section 4.1] The premise that 'every k-dimensional tensor with size m shares a consistent trajectory in parameter space' is false for a standard fully connected layer. In a layer with output y_j = sum_i W_{ji} x_i, the gradient for row j is proportional to (dL/dy_j) x^T; the input x is shared, but dL/dy_j differs across output nodes, so rows receive different updates. A PLN G_psi: R^k -> R^k mapping one random sample to one parameter vector has no mechanism to generate m distinct row trajectories. Since the dense-layer storage claim depends entirely on this premise, the statement 'size of PLN << R^k' is unsupported, and the manuscript provides no parameter counts or quantitative trajectory-overlap statistics.
  4. [Section 4.2 and Eq. (7)] The backward-transfer evaluation is circular. The common and distinctive subspace bases are obtained by SVD on representation matrices built from the same session parameter histories that are later used to evaluate Eq. (7) and to compute BWT, and the correlation gate in Eq. (6) uses the LCOR objective that the method is optimizing. A positive BWT (Table 1d: 0.24) can therefore reflect self-selection rather than genuine cross-session transfer. The paper needs a held-out basis-selection procedure or an ablation with randomly chosen bases to support the inequality in Eq. (7).
  5. [Section 5 and Table 1] The headline efficient-storage contribution is not empirically validated. The tables report only ACC and BWT; they do not report the parameter count of the PLN, whether the evaluated networks were actually generated by the PLN, or whether the reported accuracy is obtained from the PLN samples. Without this information, the claim that ECL-RR 'significantly reduces storage requirements' is not supported by the experiments. The tables also omit standard deviations and the number of runs, so the margins over HAT and UCB (e.g., 0.38% on Permuted MNIST, 0.6% on the 8-mixture dataset) cannot be assessed.
  6. [Eq. (5)] The importance-weighted estimator R_{t,M} = (1/M) sum_i Q(W_{t-1}^i, D_{t-1}) / Q(W_t^i) is not a standard importance-weighted ELBO for the current session, and the paper does not define the joint proposal Q(W,D) or show that E[log R_{t,M}] is a lower bound for log P(D_t). The additional KL terms in Eq. (5) also appear without a derivation. As written, the LCOR objective is not a well-defined estimator of log evidence.
minor comments (7)
  1. [Eq. (2)] The equation reference is left as 'as shown in ??', which should be fixed.
  2. [Table 1a and Section 5.2] The reported HAT accuracy is inconsistent: the text first states HAT achieves 91.6%, while Table 1a lists 97.3% and the later sentence cites 97.34%.
  3. [Eq. (6)] The correlation coefficient rho and the threshold epsilon are not defined, and no values are given for epsilon in the experimental section.
  4. [Figure 1] Both subcaptions (a) and (b) describe 'convolutional neural network' even though panel (a) is described in the text as the fully connected case; this is confusing.
  5. [Section 5.1] The '8-mixture dataset' is not described; the reader cannot tell how many sessions it contains or how the mixture is generated.
  6. [Eq. (4)] The notation for elementwise square and division in terms such as (mu_t - mu_{t-1})^2 / sigma_t^2 is ambiguous for tensor parameters and should be defined explicitly.
  7. [Section 4] The statement that layer-wise correlation is 'independent of the dataset' is asserted without any proof or measurement; this is a strong claim that needs support.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: reported accuracies and BWT are held-out measurements, and the storage mechanism rests on an unproven but non-circular trajectory assumption.

full rationale

I found no step in which a claimed prediction or first-principles result reduces by construction to an input or to a self-citation. The headline results in Table 1 are final-session average accuracies and backward-transfer values computed from held-out test sets via Eq. 8; they are not fitted quantities. The revised regularizer in Eq. 4 is a modified objective, and its positivity argument is a standard bound property, not a tautology. The correlation gate in Eq. 6 is a heuristic control using the same LCOR objective, but the reported BWT is an external evaluation, not the value of that gate. The common/distinctive subspace construction uses SVD on stored parameter histories; using stored previous-task information to mitigate forgetting is the disclosed mechanism of the method, not a hidden refit of the evaluation metric. There are no load-bearing self-citations: the reference list contains no overlapping authors with the present paper. The main legitimate concern is correctness, not circularity: Section 4.1 asserts that 'every k-dimensional tensor with size m shares a consistent trajectory in parameter space' and Section 4.2 asserts that 'flattened convolutional masks shows overlapping pattern in the parameter space,' both without proof or quantitative support. If false, the PLN storage and subspace decomposition claims would fail, but that is an unsupported empirical assumption, not a circular derivation. I therefore assign score 0.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

No new physical entities are introduced; the novel components are algorithmic (the Parameter Learning Network and the common/distinctive subspaces) and are covered under axioms and free parameters. The main conceptual commitments are the Gaussian variational assumption and several unproven structural claims about parameter trajectories and loss bounds.

free parameters (7)
  • initial standard deviation for conv layers = 0.03, 0.02, 0.015 for layers 1-2, 3-4, 5-6
    Chosen by hand in Section 5; affects uncertainty dynamics and final accuracy.
  • regularizer coefficients (implicit) = 1 for each term in Eq. 4
    The loss in Eq. 4 has no explicit coefficients; this equal weighting is a design choice not justified or swept.
  • correlation threshold epsilon = not reported
    Used in Eq. 6 to gate parameter updates; value not specified.
  • learning rates lambda and alpha = not reported
    Two learning rates appear in Eq. 6; not reported.
  • number of importance samples M = not reported
    Appears in R_t,M in Section 4; not reported.
  • SVD rank / number of bases kept = not reported
    The common and distinctive subspaces are defined by top-eigenvalue bases; the rank is unspecified.
  • coreset size = not reported
    A coreset is used for backward transfer evaluation, akin to VCL; size not given.
assumptions (6)
  • domain assumption Gaussian posterior approximation via Bayes by Backprop is adequate for continual learning
    The whole method assumes Q(W|theta) is Gaussian parameterized by mean and variance (Section 3).
  • ad hoc to paper All parameter tensors in a layer share a common trajectory in parameter space
    Section 4.1 asserts this to justify the PLN compression; no proof cited, only visual intuition.
  • ad hoc to paper The importance-weighted estimator R_t,M is a valid approximation of log evidence
    Section 4 defines R_t,M = (1/M) sum Q(W_{t-1}^i, D_{t-1})/Q(W_t^i) without derivation from importance weighting of the likelihood; formula as written is dimensionally inconsistent.
  • ad hoc to paper Positive regularizers in Eq. 4 ensure the loss remains a lower bound of log evidence
    Stated in Section 4; the inference is incorrect because positivity alone is insufficient and the bound direction is reversed. This is an explicit unsupported claim.
  • ad hoc to paper Layer-wise correlations of losses are constant across layers and independent of dataset
    Section 4 asserts rho(L(W_t^{i,C},D_t), L(W_t^{j,C},D_t)) = const without evidence.
  • ad hoc to paper The inequality in Eq. 7 holds (projection onto common plus differentiated subspace reduces previous-task loss)
    Backward knowledge transfer argument uses this inequality; no proof is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Revised Regularization for Efficient Continual Learning through Correlation-Based Parameter Update in Bayesian Neural Networks." pith.science (2026). https://pith.science/paper/523G7N5V

@misc{pith2026241114202,
  author       = {Pith},
  title        = {Pith review of: Revised Regularization for Efficient Continual Learning through Correlation-Based Parameter Update in Bayesian Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/523G7N5V}},
  note         = {Machine review of arXiv:2411.14202}
}
read the original abstract

We propose a Bayesian neural network-based continual learning algorithm using Variational Inference, aiming to overcome several drawbacks of existing methods. Specifically, in continual learning scenarios, storing network parameters at each step to retain knowledge poses challenges. This is compounded by the crucial need to mitigate catastrophic forgetting, particularly given the limited access to past datasets, which complicates maintaining correspondence between network parameters and datasets across all sessions. Current methods using Variational Inference with KL divergence risk catastrophic forgetting during uncertain node updates and coupled disruptions in certain nodes. To address these challenges, we propose the following strategies. To reduce the storage of the dense layer parameters, we propose a parameter distribution learning method that significantly reduces the storage requirements. In the continual learning framework employing variational inference, our study introduces a regularization term that specifically targets the dynamics and population of the mean and variance of the parameters. This term aims to retain the benefits of KL divergence while addressing related challenges. To ensure proper correspondence between network parameters and the data, our method introduces an importance-weighted Evidence Lower Bound term to capture data and parameter correlations. This enables storage of common and distinctive parameter hyperspace bases. The proposed method partitions the parameter space into common and distinctive subspaces, with conditions for effective backward and forward knowledge transfer, elucidating the network-parameter dataset correspondence. The experimental results demonstrate the effectiveness of our method across diverse datasets and various combinations of sequential datasets, yielding superior performance compared to existing approaches.

Figures

Figures reproduced from arXiv: 2411.14202 by the authors.

Figure 1
Figure 1. (a) Efficient parameter updating for a convolutional neural network involves updating the model parameters with the revised loss function. (b) Efficient parameter updating for a convolutional neural network involves updating the model parameters with the revised loss function. Following this, the basis for the differentiated subspace is determined using SVD on the representation matrix. Upon establishing corresponde… view at source ↗
Figure 2
Figure 2. Illustration of backward knowledge transfer. (a) parameters at the beginning (b) parameters after training on task 1 (c) parameters after training on task 2. The parameters that exhibited uncertainty after task 1 and were subsequently learned during task 2 contribute to the model’s ability to approach a closer alignment between the log evidence curve and its surrogate. layer is deemed uncertain. Updating the uncerta… view at source ↗
Figure 3
Figure 3. Sigma at different convolutional layers for CIFAR100. (a) layers 1,2 (b) layers 3,4 (c) layers 5,6. Best viewed when zoomed in. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 26 canonical work pages

  1. [1]

    Hongjoon Ahn, Sungmin Cha, Donggyu Lee, and Taesup Moon. 2019. Uncertainty-based continual learning with adaptive regularization. Advances in neural information processing systems 32 (2019)

    work page 2019

  2. [2]

    Michael Arbel, Anna Korba, Adil Salim, and Arthur Gretton. 2019. Maximum mean discrepancy gradient flow. Advances in Neural Information Processing Systems 32 (2019)

    work page 2019

  3. [3]

    Christopher M Bishop, Markus Svensén, and Christopher KI Williams. 1998. GTM: The generative topographic mapping. Neural computation 10, 1 (1998), 215–234

    work page 1998

  4. [4]

    David M Blei, Alp Kucukelbir, and Jon D McAuliffe. 2017. Variational inference: A review for statisticians. Journal of the American statistical Association 112, 518 (2017), 859–877

    2017

  5. [5]

    Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. 2015. Weight uncertainty in neural network. In International conference on machine learning. PMLR, 1613–1622

    2015

  6. [6]

    Stephen Brooks. 1998. Markov chain Monte Carlo method and its application. Journal of the royal statistical society: series D (the Statistician)47, 1 (1998), 69–100

    1998

  7. [7]

    Francisco M Castro, Manuel J Marín-Jiménez, Nicolás Guil, Cordelia Schmid, and Karteek Alahari. 2018. End-to-end incremental learning. In Proceedings of the European conference on computer vision (ECCV) . 233–248

    work page 2018

  8. [8]

    Arslan Chaudhry, Marc’Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elho- seiny. 2018. Efficient lifelong learning with a-gem.arXiv preprint arXiv:1812.00420 (2018)

    arXiv 2018

Show all 46 references
  1. [9]

    Yu Chen, Tom Diethe, and Neil Lawrence. 2019. Facilitating bayesian continual learning by natural gradients and stein gradients. arXiv preprint arXiv:1904.10644 (2019)

    2019 arXiv

  2. [10]

    Justin Domke and Daniel R Sheldon. 2018. Importance weighting and variational inference. Advances in neural information processing systems 31 (2018)

    2018

  3. [11]

    Sayna Ebrahimi, Mohamed Elhoseiny, Trevor Darrell, and Marcus Rohrbach

  4. [12]

    Mehrdad Farajtabar, Navid Azizan, Alex Mott, and Ang Li. 2020. Orthogonal gradient descent for continual learning. In International Conference on Artificial Intelligence and Statistics. PMLR, 3762–3773

    2020

  5. [13]

    Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, An- drei A Rusu, Alexander Pritzel, and Daan Wierstra. 2017. Pathnet: Evolution chan- nels gradient descent in super neural networks. arXiv preprint arXiv:1701.08734 (2017)

    2017 arXiv

  6. [14]

    Pierre Glaser, Michael Arbel, and Arthur Gretton. 2021. Kale flow: A relaxed kl gradient flow for probabilities with disjoint support. Advances in Neural Information Processing Systems 34 (2021), 8018–8031

    2021

  7. [15]

    José Miguel Hernández-Lobato and Ryan Adams. 2015. Probabilistic backpropaga- tion for scalable learning of bayesian neural networks. InInternational conference on machine learning. PMLR, 1861–1869

    2015

  8. [16]

    Matthew D Hoffman, David M Blei, Chong Wang, and John Paisley. 2013. Sto- chastic variational inference. Journal of Machine Learning Research (2013)

    2013

  9. [17]

    Heechul Jung, Jeongwoo Ju, Minju Jung, and Junmo Kim. 2016. Less-forgetting learning in deep neural networks. arXiv preprint arXiv:1607.00122 (2016)

    2016 arXiv

  10. [18]

    Mohammad Khan, Didrik Nielsen, Voot Tangkaratt, Wu Lin, Yarin Gal, and Akash Srivastava. 2018. Fast and scalable bayesian deep learning by weight-perturbation in adam. In International conference on machine learning . PMLR, 2611–2620

    2018

  11. [19]

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of...

    2017

  12. [20]

    Kibok Lee, Kimin Lee, Jinwoo Shin, and Honglak Lee. 2019. Overcoming cata- strophic forgetting with unlabeled data in the wild. InProceedings of the IEEE/CVF International Conference on Computer Vision . 312–321

    2019

  13. [21]

    Zhizhong Li and Derek Hoiem. 2017. Learning without forgetting. IEEE transac- tions on pattern analysis and machine intelligence 40, 12 (2017), 2935–2947

    2017

  14. [22]

    Sen Lin, Li Yang, Deliang Fan, and Junshan Zhang. 2022. Beyond not-forgetting: Continual learning with backward knowledge transfer. Advances in Neural Information Processing Systems 35 (2022), 16165–16177

    2022

  15. [23]

    Qiang Liu and Dilin Wang. 2016. Stein variational gradient descent: A general purpose bayesian inference algorithm. Advances in neural information processing systems 29 (2016)

    2016

  16. [24]

    David Lopez-Paz and Marc’Aurelio Ranzato. 2017. Gradient episodic memory for continual learning. Advances in neural information processing systems 30 (2017)

    2017

  17. [25]

    David John Cameron Mackay. 1992. Bayesian methods for adaptive models . Cali- fornia Institute of Technology

    1992

  18. [26]

    Alfred Müller. 1997. Integral probability metrics and their generating classes of functions. Advances in applied probability 29, 2 (1997), 429–443

    1997

  19. [27]

    Cuong V Nguyen, Yingzhen Li, Thang D Bui, and Richard E Turner. 2017. Varia- tional continual learning. arXiv preprint arXiv:1710.10628 (2017)

    2017 arXiv

  20. [28]

    Trung V Nguyen and Edwin V Bonilla. 2014. Automated variational inference for Gaussian process models. Advances in Neural Information Processing Systems 27 (2014)

    2014

  21. [29]

    Rajesh Ranganath, Sean Gerrish, and David Blei. 2014. Black box variational inference. In Artificial intelligence and statistics. PMLR, 814–822

    2014

  22. [30]

    Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. 2018. Learning to learn without forgetting by maximizing transfer and minimizing interference. arXiv preprint arXiv:1810.11910 (2018)

    2018 arXiv

  23. [31]

    David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. 2019. Experience replay for continual learning. Advances in Neural Information Processing Systems 32 (2019)

    2019

  24. [32]

    Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. 2016. Pro- gressive neural networks. arXiv preprint arXiv:1606.04671 (2016)

    2016 arXiv

  25. [33]

    Gobinda Saha, Isha Garg, and Kaushik Roy. 2021. Gradient projection memory for continual learning. arXiv preprint arXiv:2103.09762 (2021)

    2021 arXiv

  26. [34]

    Jonathan Schwarz, Wojciech Czarnecki, Jelena Luketina, Agnieszka Grabska- Barwinska, Yee Whye Teh, Razvan Pascanu, and Raia Hadsell. 2018. Progress & compress: A scalable framework for continual learning. InInternational conference on machine learning. PMLR, 4528–4537

    2018

  27. [35]

    Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. 2018. Over- coming catastrophic forgetting with hard attention to the task. In International conference on machine learning . PMLR, 4548–4557

    2018

  28. [36]

    Kumar Shridhar, Felix Laumann, and Marcus Liwicki. 2019. A comprehensive guide to bayesian convolutional neural network with variational inference.arXiv preprint arXiv:1901.02731 (2019)

    2019 arXiv

  29. [37]

    Jasper Snoek, Hugo Larochelle, and Ryan P Adams. 2012. Practical bayesian optimization of machine learning algorithms. Advances in neural information processing systems 25 (2012)

    2012

  30. [38]

    Artem Sobolev and Dmitry P Vetrov. 2019. Importance weighted hierarchical variational inference. Advances in Neural Information Processing Systems 32 (2019)

    2019

  31. [39]

    Hanna Tseran, Mohammad Emtiyaz Khan, Tatsuya Harada, and Thang D Bui

  32. [40]

    Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. 2019. Large scale incremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 374–382

    2019

  33. [41]

    Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. 2017. Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017)

    2017 arXiv

  34. [42]

    Friedemann Zenke, Ben Poole, and Surya Ganguli. 2017. Continual learning through synaptic intelligence. In International Conference on Machine Learning . PMLR, 3987–3995

    2017

  35. [43]

    Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals

  36. [2018]

    In Continual Learning Workshop@ NeurIPS, Vol

    Natural variational continual learning. In Continual Learning Workshop@ NeurIPS, Vol. 2

  37. [2019]

    arXiv preprint arXiv:1906.02425 (2019)

    Uncertainty-guided continual learning with bayesian neural networks. arXiv preprint arXiv:1906.02425 (2019)

    2019 arXiv

  38. [2021]

    Understanding deep learning (still) requires rethinking generalization. Commun. ACM 64, 3 (2021), 107–115

    2021

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.