REVIEW 2 cited by
Replay in Deep Learning: Current Approaches and Missing Biological Elements
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Replay is the reactivation of one or more neural patterns, which are similar to the activation patterns experienced during past waking experiences. Replay was first observed in biological neural networks during sleep, and it is now thought to play a critical role in memory formation, retrieval, and consolidation. Replay-like mechanisms have been incorporated into deep artificial neural networks that learn over time to avoid catastrophic forgetting of previous knowledge. Replay algorithms have been successfully used in a wide range of deep learning methods within supervised, unsupervised, and reinforcement learning paradigms. In this paper, we provide the first comprehensive comparison between replay in the mammalian brain and replay in artificial neural networks. We identify multiple aspects of biological replay that are missing in deep learning systems and hypothesize how they could be utilized to improve artificial neural networks.
Forward citations
Cited by 2 Pith papers
-
Consolidator: Learning Persistent Routed Memory Across Context Boundaries
A learned slot-local transform lets a frozen backbone update and retrieve a persistent memory after context reset, and using that memory to guide routing nearly doubles updated-mapping recall.
-
Scalable Strategies for Continual Learning with Replay
A replay-based continual learning toolkit that combines low-rank adaptation, a post-task consolidation phase, and sequential weight merging to cut replay sample usage by up to 65% at matched accuracy.
Discussion (0). Continue with ORCID to comment.