REVIEW 6 cited by
Neuroplastic Expansion in Deep Reinforcement Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
The loss of plasticity in learning agents, analogous to the solidification of neural pathways in biological brains, significantly impedes learning and adaptation in reinforcement learning due to its non-stationary nature. To address this fundamental challenge, we propose a novel approach, {\it Neuroplastic Expansion} (NE), inspired by cortical expansion in cognitive science. NE maintains learnability and adaptability throughout the entire training process by dynamically growing the network from a smaller initial size to its full dimension. Our method is designed with three key components: (\textit{1}) elastic topology generation based on potential gradients, (\textit{2}) dormant neuron pruning to optimize network expressivity, and (\textit{3}) neuron consolidation via experience review to strike a balance in the plasticity-stability dilemma. Extensive experiments demonstrate that NE effectively mitigates plasticity loss and outperforms state-of-the-art methods across various tasks in MuJoCo and DeepMind Control Suite environments. NE enables more adaptive learning in complex, dynamic environments, which represents a crucial step towards transitioning deep reinforcement learning from static, one-time training paradigms to more flexible, continually adapting models.
Forward citations
Cited by 6 Pith papers
-
NeuMoSync: End-to-End Neuromodulatory Control for Plasticity and Adaptability in Continual Learning
A global neuromodulatory controller with per-neuron weight, activation, and offset modulation preserves plasticity and improves forward and backward adaptation in continual learning.
-
Network Sparsity Unlocks the Scaling Potential of Deep Reinforcement Learning
One-shot random pruning at initialization lets deep RL networks keep improving at model sizes where dense networks collapse in performance.
-
Silent Neuron Theory and Plasticity Preservation for Deep Reinforcement Learning in Adaptive Video Streaming
Silent Neuron theory provides a framework for plasticity degradation in deep RL, and ReSiN preserves it via forward-backward guided resets, yielding up to 168% higher bitrate and 108% better QoE in video streaming.
-
A Forget-and-Grow Strategy for Deep Reinforcement Learning Scaling in Continuous Control
Forget and Grow (FoG) combines decaying replay weights for old experiences with progressive critic-network expansion to improve continuous-control reinforcement learning, beating BRO, SimBa, and TD-MPC2 on most of 41 ...
-
A Simple Baseline for Stable and Plastic Neural Networks
RDBP combines a new activation (ReLUDown) with decaying backpropagation to achieve strong stability and plasticity on Continual ImageNet with low overhead.
-
Growing with Experience: Growing Neural Networks in Deep Reinforcement Learning
A simple PPO training scheme that adds hidden layers over time via function-preserving Net2Net morphisms outperforms static networks of the same final depth on MiniHack Room and MuJoCo Ant.
Discussion (0). Continue with ORCID to comment.