REVIEW 8 cited by
Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Deep representation learning methods struggle with continual learning, suffering from both catastrophic forgetting of useful units and loss of plasticity, often due to rigid and unuseful units. While many methods address these two issues separately, only a few currently deal with both simultaneously. In this paper, we introduce Utility-based Perturbed Gradient Descent (UPGD) as a novel approach for the continual learning of representations. UPGD combines gradient updates with perturbations, where it applies smaller modifications to more useful units, protecting them from forgetting, and larger modifications to less useful units, rejuvenating their plasticity. We use a challenging streaming learning setup where continual learning problems have hundreds of non-stationarities and unknown task boundaries. We show that many existing methods suffer from at least one of the issues, predominantly manifested by their decreasing accuracy over tasks. On the other hand, UPGD continues to improve performance and surpasses or is competitive with all methods in all problems. Finally, in extended reinforcement learning experiments with PPO, we show that while Adam exhibits a performance drop after initial learning, UPGD avoids it by addressing both continual learning issues.
Forward citations
Cited by 8 Pith papers
-
V-Simba: Unleashing the Architectural Potential of RL in Visual Continuous Control
V-Simba, a visual RL architecture combining layer normalization, weight decay, and a distributional critic, matches or outperforms complex baselines on 29 continuous control tasks while using less compute.
-
Mitigating Plasticity Loss in Continual Reinforcement Learning by Reducing Churn
Reducing churn in continual RL via C-CHAIN prevents NTK rank collapse and substantially improves learning across four benchmark suites.
-
Torque-Aware Momentum
Torque-Aware Momentum damps momentum updates by the alignment between new gradients and previous momentum, giving small gains on some benchmarks but mixed results on large model fine-tuning.
-
Sustaining Plasticity via Learnable Wavelet Activations in Continual Learning
A learnable wavelet activation with dynamic capacity injection and slope regularization improves plasticity retention in continual learning.
-
Relative Value Learning
A critic that learns antisymmetric value differences ∆(s_i,s_j)=V(s_i)−V(s_j) has a provably contracting Bellman operator and an unbiased advantage estimator, and PPO with this critic matches standard PPO on Atari.
-
In Praise of Stubbornness: An Empirical Case for Cognitive-Dissonance Aware Continual Update of Knowledge in LLMs
Fine-tuning LLMs on contradictory facts catastrophically corrupts unrelated knowledge, undeterred by selective neuron updates, while non-contradictory updates remain safe.
-
Activation by Interval-wise Dropout: A Simple Way to Prevent Neural Networks from Plasticity Loss
AID, a stochastic activation applying different dropout rates to positive and negative preactivations, mitigates plasticity loss and improves continual learning, reinforcement learning, and standard supervised learnin...
-
A Simple Baseline for Stable and Plastic Neural Networks
RDBP combines a new activation (ReLUDown) with decaying backpropagation to achieve strong stability and plasticity on Continual ImageNet with low overhead.
Discussion (0). Continue with ORCID to comment.