REVIEW 2 cited by
On Quadratic Penalties in Elastic Weight Consolidation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Elastic weight consolidation (EWC, Kirkpatrick et al, 2017) is a novel algorithm designed to safeguard against catastrophic forgetting in neural networks. EWC can be seen as an approximation to Laplace propagation (Eskin et al, 2004), and this view is consistent with the motivation given by Kirkpatrick et al (2017). In this note, I present an extended derivation that covers the case when there are more than two tasks. I show that the quadratic penalties in EWC are inconsistent with this derivation and might lead to double-counting data from earlier tasks.
Forward citations
Cited by 2 Pith papers
-
Online Curvature-Aware Replay: Leveraging $\mathbf{2^{nd}}$ Order Information for Online Continual Learning
OCAR combines experience replay with K-FAC Fisher preconditioning and scheduled damping to improve stability and plasticity in online continual learning.
-
Act Now: A Novel Online Forecasting Framework for Large-Scale Streaming Data
Act-Now combines random subgraph sampling, fast and slow stream buffers, and a label-decomposition forecaster, reporting large MSE reductions on three cellular traffic datasets.
Discussion (0). Continue with ORCID to comment.