REVIEW 6 cited by
Towards Continual Reinforcement Learning: A Review and Perspectives
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
In this article, we aim to provide a literature review of different formulations and approaches to continual reinforcement learning (RL), also known as lifelong or non-stationary RL. We begin by discussing our perspective on why RL is a natural fit for studying continual learning. We then provide a taxonomy of different continual RL formulations by mathematically characterizing two key properties of non-stationarity, namely, the scope and driver non-stationarity. This offers a unified view of various formulations. Next, we review and present a taxonomy of continual RL approaches. We go on to discuss evaluation of continual RL agents, providing an overview of benchmarks used in the literature and important metrics for understanding agent performance. Finally, we highlight open problems and challenges in bridging the gap between the current state of continual RL and findings in neuroscience. While still in its early days, the study of continual RL has the promise to develop better incremental reinforcement learners that can function in increasingly realistic applications where non-stationarity plays a vital role. These include applications such as those in the fields of healthcare, education, logistics, and robotics.
Forward citations
Cited by 6 Pith papers
-
Uncertainty Prioritized Experience Replay
UPER uses ensemble-based epistemic and aleatoric uncertainty to compute an information gain priority for experience replay, outperforming TD-error prioritization on Atari-57.
-
When Does Neuroevolution Outcompete Reinforcement Learning in Transfer Learning Tasks?
On two new curriculum benchmarks, direct-encoding neuroevolution (NEAT) transfers skills across levels better than PPO reinforcement learning, while indirect encodings like HyperNEAT transfer poorly.
-
Position: Theory of Mind Benchmarks are Broken for Large Language Models
The paper proposes that LLM theory-of-mind evaluation should measure functional adaptation to partners, not just literal prediction of their behavior, and shows the two can diverge sharply in simple games.
-
Enabling Realtime Reinforcement Learning at Scale with Staggered Asynchronous Inference
Staggered asynchronous inference lets reinforcement learning agents with large, slow models act at every time step in realtime environments, at the cost of delay regret that grows with environment stochasticity.
-
Parseval Regularization for Continual Reinforcement Learning
Parseval regularization, a cheap orthogonality-preserving penalty, improves continual RL agents' success on new tasks across gridworld, CARL and MetaWorld benchmarks.
-
An Arbitration Control for an Ensemble of Diversified DQN variants in Continual Reinforcement Learning
ACED-DQN combines heterogeneous DQN variants with loss-based reliability weighting and experience assignment, but the paper's own ablation indicates that arbitration control is not the key factor behind the performance gain.
Discussion (0). Continue with ORCID to comment.