Continual Learning and Catastrophic Forgetting
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:CMXRPNHWrecord.jsonopen to challenge →
read the original abstract
This book chapter delves into the dynamics of continual learning, which is the process of incrementally learning from a non-stationary stream of data. Although continual learning is a natural skill for the human brain, it is very challenging for artificial neural networks. An important reason is that, when learning something new, these networks tend to quickly and drastically forget what they had learned before, a phenomenon known as catastrophic forgetting. Especially in the last decade, continual learning has become an extensively studied topic in deep learning. This book chapter reviews the insights that this field has generated.
This paper has not been read by Pith yet.
Forward citations
Cited by 12 Pith papers
-
Gradient-Free Continual Learning in Spiking Neural Networks via Inter-Spike Interval Regularization
ISI-CV derives a synaptic importance score from the regularity of neuron firing intervals to enable continual learning without gradients or forgetting on SNNs.
-
Optimal L2 Regularization in High-dimensional Continual Linear Regression
In high-dimensional continual linear regression, optimal fixed L2 regularization strength scales as T/ln T with the number of tasks and mitigates label noise for arbitrary linear teachers.
-
McNdroid: A Longitudinal Multimodal Benchmark for Robust Drift Detection in Android Malware
McNdroid is a new longitudinal multimodal benchmark showing that Android malware detectors degrade over time but multimodal approaches maintain better performance across long temporal gaps.
-
Remembering More, Risking More: Longitudinal Safety Risks in Memory-Equipped LLM Agents
Memory-equipped LLM agents exhibit increasing safety violation rates as memory accumulates across independent tasks, termed temporal memory contamination, detected via a new trigger-probe protocol.
-
Geometry Conflict: Explaining and Controlling Forgetting in LLM Continual Post-Training
Forgetting in LLM continual post-training is a geometry conflict between task-induced covariance structures and the evolving model state, controlled by gating Wasserstein barycenter merging on measured conflict.
-
Activation- and Influence-Aware Ranks (AIR): Function-Preserving SVD Compression for LLMs
AIR augments activation-aware SVD compression of LLMs with an influence metric and a closed-form ALS update, claiming >18% perplexity improvement at 60% parameter retention and 90% less calibration data than SVD-LLM(W).
-
DyGRO-VLA: Cross-Task Scaling of Vision-Language-Action Models via Dynamic Grouped Residual Optimization
DyGRO-VLA is a two-stage optimization framework for cross-task scaling of Vision-Language-Action models via dynamic grouped residual optimization in RL.
-
HEDP: A Hybrid Energy-Distance Prompt-based Framework for Domain Incremental Learning
HEDP uses energy regularization inspired by Helmholtz free energy plus hybrid energy-distance weighting in prompts to improve domain selection and achieve a 2.57% accuracy gain on benchmarks like CORe50 while mitigati...
-
An empirical evaluation of the risks of AI model updates using clinical data: stability, arbitrariness, and fairness
Updating clinical AI models can cause prediction flips, arbitrariness, and unfair error rates across groups, requiring dedicated monitoring dimensions.
-
ALAS: Adaptive Long-Horizon Action Synthesis via Async-pathway Stream Disentanglement
ALAS disentangles environment and self-state streams via bio-inspired modules to deliver 23% higher subtask success and 29% better execution efficiency on long-horizon HSI tasks.
-
Task Switching Without Forgetting via Proximal Decoupling
Operator splitting separates task optimization from proximal stability enforcement to achieve forgetting-free continual learning with SOTA benchmark results.
-
Toward decision-aware AI for LSST-scale time-domain astronomy
Proposes foundation models and decision-theoretic policies to manage evolving source representations and optimize follow-up resource allocation in LSST-scale time-domain astronomy.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.