Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Advancements and Challenges in Continual Reinforcement Learning: A Comprehensive Review

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A survey maps continual reinforcement learning into five method families and the open problems around them.

desk verdict A useful newcomer's map of continual RL, but the paper's own Figure 8 omits regularization as a family—contradicting its own Table 4—so 'comprehensive' overpromises. read the letter →

arxiv 2506.21899 v1 pith:5KM5FDGJ submitted 2025-06-27 cs.LG

Amara Zuffer , Michael Burke , Mehrtash Harandi This is my paper
classification cs.LG
keywords continualreinforcementlearningcatastrophicforgettinglifelongsurveyroboticsbenchmarkstask-agnosticknowledgeretention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This survey tries to give newcomers a single readable path into continual reinforcement learning (CRL), the setting where an RL agent must learn a sequence of tasks without forgetting earlier ones. It assembles background on brain-inspired continual learning and reinforcement learning, then organizes CRL algorithms into families such as memory consolidation, modular architectures, parameter isolation, distillation, and hybrid methods. It also catalogs common evaluation environments, benchmarks, and metrics, and reviews recent CRL work in robotics. The paper's goal is descriptive: to map the field and its open problems so that a new researcher can find methods, benchmarks, and gaps quickly.

What carries the argument

The organizing device is the taxonomy of CRL knowledge-retention methods (Figure 8), together with the tables of RL environments, benchmarks, and metrics. The taxonomy splits CRL into memory consolidation techniques (rehearsal, generative replay, dual-memory systems, distillation, hybrids), modular architecture techniques (fixed and dynamic networks), and parameter isolation techniques (pruning and masking), and the tables attach concrete algorithms, environments, and evaluation formulas to those categories. This machinery carries the survey's contribution by turning a scattered literature into a map that a newcomer can navigate by method family, application domain, and measurement choice.

What would settle it

A reader could test the comprehensiveness claim by taking the proceedings of the last few major reinforcement learning conferences, listing every continual RL method introduced, and checking whether each one fits into a branch of Figure 8 or appears in the survey's tables; any prominent missing algorithm or category that does not fit would show that the map is incomplete.

Watch

Extended reading notes

Core claim

The paper's central claim is that continual reinforcement learning can be presented as a structured landscape with a manageable number of knowledge-retention strategies, and that this organization, paired with a summary of environments and metrics, is enough to orient a newcomer. On its own terms, the survey establishes a vertical taxonomy in which CRL approaches branch into memory consolidation techniques, modular architecture techniques, parameter isolation techniques, distillation-based approaches, and hybrid techniques, and it maps prominent algorithms and benchmarks onto these branches. It further claims that most CRL evaluations rely on a small set of environments, with Atari historically dominant and rehearsal-based methods most common there, and that robotics is a growing application area with its own evaluation conventions. The discovery, if accepted, is that the field is mature enough to be surveyed in one accessible document yet still open on issues such as task-agnostic learning, backward transfer, and realistic benchmarks.

Load-bearing premise

The review's usefulness rests on the assumption that the selected references and the Figure 8 categories faithfully represent the current CRL field, since the paper does not describe a systematic literature search or inclusion criteria.

Editorial extensions

If this is right

  • A newcomer can use the Figure 8 taxonomy to place any CRL algorithm into one of five method families and then find representative work and environments for that family.
  • Researchers comparing methods can use the survey's metric summaries (average performance, forward transfer, catastrophic forgetting, backward transfer, continual evaluation) to align their evaluation with common practice in non-robotic and robotic settings.
  • The review's environment table identifies Atari, Continual World, Minigrid, Procgen, and Meta-World as common testbeds, and shows that rehearsal-based algorithms dominate Atari evaluations.
  • The robotics section implies that sim-to-real transfer, wear-and-tear non-stationarity, and limited data are the main obstacles that CRL must solve before robots can master multiple tasks on one system.
  • The future-directions discussion identifies task-agnostic learning, backward transfer, and benchmark realism as the open problems most likely to define the next phase of CRL research.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the survey's coverage is representative, then the field's heavy focus on catastrophic forgetting means that other desiderata, such as backward transfer and skill composition, are under-measured; adding standardized backward-transfer metrics to existing benchmarks would likely reshuffle current method rankings.
  • Because the survey does not describe a systematic search or inclusion criteria, its map is best read as a curated entry point; a living, community-maintained version would be needed to keep the title's promise of completeness true as the field grows.
  • The proposed skill re-usability and task-identification-efficiency metrics could be operationalized directly in hierarchical and task-agnostic benchmarks, giving reviewers a quantitative way to compare methods that currently report only average reward.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This manuscript is a survey of continual reinforcement learning (CRL). It introduces biological inspiration and continual learning (CL) background, reviews RL basics, categorizes CRL approaches into modular architectures, parameter isolation, memory consolidation, distillation, and hybrids, discusses robotic CRL, summarizes evaluation metrics and benchmark tools, and lists open challenges. The paper states its core goal as providing an easily accessible resource for new researchers and a comprehensive overview of recent CRL developments, rather than presenting a formal taxonomy.

Significance. If the survey were internally consistent and verifiably comprehensive, it would be a useful entry point for newcomers: it compiles a large bibliography, includes pseudo-codes for several standard CL algorithms, and tabulates environments, metrics, and benchmark platforms. The paper's central claim, however, depends on the accuracy and completeness of its organizing map and on a defensible selection of references. The taxonomy in Figure 8 omits regularization as a CRL family even though the paper's own text and Table 4 discuss EWC, Online EWC, SI, MAS, RWalk, and VCL, and the paper reports no systematic literature-search protocol to support the 'comprehensive' title. These issues weaken the accessibility claim but are correctable within the manuscript's scope.

major comments (4)
  1. [3.2, Figure 8] Figure 8 is presented as the paper's map of CRL families, but it omits regularization-based approaches entirely. EWC, Online EWC, SI, MAS, RWalk, and VCL are discussed in Sections 3.2.1 and 3.2.5 and appear in the benchmark rows of Table 4, yet a newcomer using Figure 8 cannot place any of them. This is load-bearing for the 'easily accessible resource' claim in Section 1. The figure should either include a regularization branch or the paper should explicitly justify why its taxonomy departs from standard CRL taxonomies such as Khetarpal et al. (2022), which is cited but not followed.
  2. [3.2.1 vs 3.2.4] Progress and Compress (P&C) is described in Section 3.2.1 as 'based on knowledge distillation' and using an Online EWC component, but it is categorized in Figure 8 under Memory Consolidation, while distillation is a separate branch (Section 3.2.4). This internal inconsistency makes the taxonomy harder to use. The authors should either cross-reference P&C as a hybrid or explain the assignment rule they use when an algorithm spans multiple families.
  3. [1, Scope] The title and Scope promise a 'comprehensive review' and coverage of 'the latest advancements', but the paper does not report a systematic literature-search procedure, inclusion/exclusion criteria, or any coverage assessment. Without such a methodology, a reader cannot distinguish deliberate pruning from accidental omission, and the comprehensiveness claim is not verifiable. Adding a short methodology paragraph or an explicit coverage discussion would substantiate the paper's central claim.
  4. [5.2] The proposed metrics in Section 5.2 are not validated. Transfer efficiency in Eq. (12) uses an undefined performance measure P and is presented without any experimental or theoretical demonstration, and the task-identification efficiency metric is only described verbally. For a paper intended as an accessible resource for newcomers, presenting these as established metrics is misleading. They should be moved to the future-directions section or provided with precise definitions and illustrative validation.
minor comments (5)
  1. [Abstract] The abstract uses the term 'continuous reinforcement learning', while the rest of the paper uses 'continual reinforcement learning'; since 'continuous' also refers to continuous action/state spaces in RL, this inconsistency will confuse newcomers and should be fixed.
  2. [2.3.5] There are several copyediting errors in this section, including 'the the need' and 'it's policy' instead of 'its policy'; these should be corrected throughout.
  3. [2.3.4, References] The citation 'Gulcehre et al., 2006' for RL Unplugged is incorrect; the actual work was published in 2020. The reference should be updated to avoid propagating a wrong year.
  4. [5.1, Table 3] The forward-transfer formula in Eq. (8) for robotic environments uses AUC values but does not define AUC or explain the normalization denominator (1 - AUC_Tr(i)); a brief definition would make the metric reproducible for readers new to the area.
  5. [4] The phrase '2x improvement' should be written as '2× improvement' for consistency with standard notation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the review's claims are descriptive and grounded in external literature; the few self-citations are background references and not load-bearing.

full rationale

This manuscript is a literature review. Its central assertions are that it provides an accessible entry point to continual reinforcement learning and that it surveys recent advancements; these claims are supported by citations to external papers, environments, benchmarks, and metrics. No parameter is fitted to a subset of data and then renamed as a prediction, and no derived result is equivalent by construction to an input assumption. The mathematical content (Equations 1-12) restates standard definitions from the cited literature, such as the continual learning loss, Fisher Information Matrix, and common forward-transfer and forgetting metrics; none of these derivations reduce to the paper's own conclusions. The authors' self-citations (Asenov et al. 2019, Davchev et al. 2022, Carreno et al. 2023) appear as ordinary background references in discussions of robotics and adaptation; they are not used to justify the paper's organizational framework or to exclude alternative taxonomies. Concerns about the completeness of Figure 8's taxonomy or the absence of a systematic search protocol are accuracy and coverage issues, not circularity. The review is self-contained as a survey: its value depends on faithful reporting of external work, and no load-bearing step is circular.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

The survey introduces no free parameters or invented entities. Its only domain assumption is the POMDP framing of CRL tasks, which is standard in the literature.

assumptions (1)
  • domain assumption CRL tasks are modeled as POMDPs
    Section 3 states 'Each of these tasks is presumed to follow the framework of a POMDP', which shapes the review's framing of CRL.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Advancements and Challenges in Continual Reinforcement Learning: A Comprehensive Review." pith.science (2026). https://pith.science/paper/5KM5FDGJ

@misc{pith2026250621899,
  author       = {Pith},
  title        = {Pith review of: Advancements and Challenges in Continual Reinforcement Learning: A Comprehensive Review},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5KM5FDGJ}},
  note         = {Machine review of arXiv:2506.21899}
}
read the original abstract

The diversity of tasks and dynamic nature of reinforcement learning (RL) require RL agents to be able to learn sequentially and continuously, a learning paradigm known as continuous reinforcement learning. This survey reviews how continual learning transforms RL agents into dynamic continual learners. This enables RL agents to acquire and retain useful and reusable knowledge seamlessly. The paper delves into fundamental aspects of continual reinforcement learning, exploring key concepts, significant challenges, and novel methodologies. Special emphasis is placed on recent advancements in continual reinforcement learning within robotics, along with a succinct overview of evaluation environments utilized in prominent research, facilitating accessibility for newcomers to the field. The review concludes with a discussion on limitations and promising future directions, providing valuable insights for researchers and practitioners alike.

Figures

Figures reproduced from arXiv: 2506.21899 by the authors.

Figure 1
Figure 1. The learning schema within the brain. (a) Complementary Learning System [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Key features/Desiderata of Lifelong Learning Systems (Kudithipudi et al., 2022). [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Diverse learning paradigms in contrast to CL. Each human figure represents a [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Classification of CL methodologies with their sub-categories. The blue text [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Interaction between an RL agent and MDP/POMDP based environment. (a) The [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: A summary on data modality-based inputs for an RL agent. [PITH_FULL_IMAGE:figures/full_fig_p023_6.png]
Figure 7
Figure 7. Figure 7: An illustrative representation showing a change-point (highlighted in red) resulting [PITH_FULL_IMAGE:figures/full_fig_p027_7.png]
Figure 8
Figure 8. Figure 8: A vertical tree-based representation of the various families or methods in CRL [PITH_FULL_IMAGE:figures/full_fig_p028_8.png]
Figure 9
Figure 9. Figure 9: The continual training curve(in green) across 2 environments, each trained for 5M [PITH_FULL_IMAGE:figures/full_fig_p036_9.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PATH-Bench: Path-Dependent Evaluation of Lifelong Agents

    cs.AI 2026-08 conditional novelty 6.0 of 10

    A benchmark that controls task order to measure transfer, retention, and path-dependence in lifelong LLM agents, applied to code and tool-use tasks.

Reference graph

Works this paper leans on

36 extracted references · 16 canonical work pages · cited by 1 Pith paper

  1. [1]

    Abbas, Z., Zhao, R., Modayil, J., White, A., and Machado, M. C. (2023). Loss of plasticity in continual deep reinforcement learning. InConference on Lifelong Learning Agents, pages 620–636. PMLR. Abraham, W. C. and Bear, M. F. (1996). Metaplasticity: the plasticity of synaptic plasticity. Trends in neurosciences, 19(4):126–130. Agarwal, R., Schuurmans, D....

    arXiv 2023

  2. [4]

    Shim, D., Mai, Z., Jeong, J., Sanner, S., Kim, H., and Jang, J. (2021). Online class- incremental continual learning with adversarial shapley value. InProceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 9630–9638. Shin, H., Lee, J. K., Kim, J., and Kim, J. (2017). Continual learning with deep generative replay.Advances in neura...

    work page 2021

  3. [7]

    Chevalier-Boisvert, M., Dai, B., Towers, M., de Lazcano, R., Willems, L., Lahlou, S., Pal, S., Castro, P

    Springer. Chevalier-Boisvert, M., Dai, B., Towers, M., de Lazcano, R., Willems, L., Lahlou, S., Pal, S., Castro, P. S., and Terry, J. (2023). Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks.CoRR, abs/2306.13831. Clavera, I., Rothfuss, J., Schulman, J., Fujita, Y., Asfour, T., and Abbeel, P. (2018). ...

    arXiv 2023

  4. [10]

    M., Mathieu, M., Dudzik, A., Chung, J., Choi, D

    Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019). Grandmaster level in starcraft ii using multi-agent reinforcement learning.Nature, 575(7782):350–354. 62 V¨ odisch, N., Cattaneo, D., Burgard, W., and Valada, A. (2023). Covio: Online continual learning for v...

    arXiv 2019

  5. [11]

    Hessel, M., Modayil, J., Van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., Horgan, D., Piot, B., Azar, M., and Silver, D. (2018). Rainbow: Combining improvements in deep reinforcement learning. InProceedings of the AAAI conference on artificial intelligence, volume

    work page 2018

  6. [12]

    Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531. Hoi, S. C., Sahoo, D., Lu, J., and Zhao, P. (2021). Online learning: A comprehensive survey. Neurocomputing, 459:249–289. Hospedales, T., Antoniou, A., Micaelli, P., and Storkey, A. (2021). Meta-learning in neural networks: A survey...

    arXiv 2015

  7. [13]

    Jedlicka, P., Tomko, M., Robins, A., and Abraham, W. C. (2022). Contributions by metaplasticity to solving the catastrophic forgetting problem.Trends in Neurosciences, 45(9):656–666. 50 Johnson, E. C., Nguyen, E. Q., Schreurs, B., Ewulum, C. S., Ashcraft, C., Fendley, N. M., Baker, M. M., New, A., and Vallabha, G. K. (2022). L2explorer: A lifelong reinfor...

    arXiv 2022

  8. [18]

    52 Kumaran, D., Hassabis, D., and McClelland, J. L. (2016). What learning systems do intelligent agents need? complementary learning systems theory updated.Trends in cognitive sciences, 20(7):512–534. Kurniawati, H. (2022). Partially observable markov decision processes and robotics.Annual Review of Control, Robotics, and Autonomous Systems, 5:253–277. La...

    work page 2016

Show all 36 references
  1. [19]

    S., and Lim, J

    Lee, Y., Sun, S.-H., Somasundaram, S., Hu, E. S., and Lim, J. J. (2018). Composing complex skills by learning transition policies. InInternational Conference on Learning Representations. Lesort, T., D ´ ıaz-Rodr ´ ıguez, N., Goudou, J.-F., and Filliat, D. (2018). State represe...

    2018 arXiv

  2. [20]

    Lin, S., Yang, L., Fan, D., and Zhang, J. (2022). Beyond not-forgetting: Continual learning with backward knowledge transfer.Advances in Neural Information Processing Systems, 35:16165–16177. Liu, B., Liu, Q., and Stone, P. (2022). Continual learning and private unlearning. In...

    2022 arXiv

  3. [21]

    Liu, Y., Su, Y., Liu, A.-A., Schiele, B., and Sun, Q. (2020). Mnemonics training: Multi-class incremental learning without forgetting. InProceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pages 12245–12254. Lomonaco, V., Pellegrini, L., Cossu, A...

    2020 arXiv

  4. [22]

    Konidaris, G., Kuindersma, S., Grupen, R., and Barto, A. (2012). Robot learning from demonstration by constructing skill trees.The International Journal of Robotics Research, 31(3):360–375. Konishi, T., Kurokawa, M., Ono, C., Ke, Z., Kim, G., and Liu, B. (2023). Parameter-leve...

    2012 arXiv

  5. [23]

    L., Kafle, K., Shrestha, R., Acharya, M., and Kanan, C

    Hayes, T. L., Kafle, K., Shrestha, R., Acharya, M., and Kanan, C. (2020). Remind your neural network to prevent catastrophic forgetting. InEuropean Conference on Computer Vision, pages 466–483. Springer. Hayes, T. L., Krishnan, G. P., Bazhenov, M., Siegelmann, H. T., Sejnowski...

    2020

  6. [24]

    A., van Seijen, H., and Eaton, E

    Mendez, J. A., van Seijen, H., and Eaton, E. (2022). Modular lifelong reinforcement learning via neural composition.arXiv preprint arXiv:2207.00429. Merlin, G., Lomonaco, V., Cossu, A., Carta, A., and Bacciu, D. (2022). Practical recom- mendations for replay-based continual le...

    2022 arXiv

  7. [25]

    P., Chakravarthi Raja, S., Cheney, N., Clune, J., et al

    Kudithipudi, D., Aguilar-Simon, M., Babb, J., Bazhenov, M., Blackiston, D., Bongard, J., Brna, A. P., Chakravarthi Raja, S., Cheney, N., Clune, J., et al. (2022). Biological underpinnings for lifelong learning machines.Nature Machine Intelligence, 4(3):196–210. Kumar, A., Fu, ...

    2022

  8. [26]

    Nikishin, E., Schwarzer, M., D’Oro, P., Bacon, P.-L., and Courville, A. (2022). The primacy bias in deep reinforcement learning. InInternational conference on machine learning, pages 16828–16847. PMLR. Normandin, F., Golemo, F., Ostapenko, O., Rodriguez, P., Riemer, M. D., Hur...

    2022 arXiv

  9. [27]

    Grauman, K., Westbury, A., Byrne, E., Chavis, Z., Furnari, A., Girdhar, R., Hamburger, J., Jiang, H., Liu, M., Liu, X., et al. (2022). Ego4d: Around the world in 3,000 hours of egocentric video. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio...

    2022 arXiv

  10. [28]

    Rebuffi, S.-A., Kolesnikov, A., Sperl, G., and Lampert, C. H. (2017). icarl: Incremental classifier and representation learning. InProceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 2001–2010. Ren, J., Zeng, Y., Zhou, S., and Zhang, Y. (2021)....

    2017

  11. [29]

    58 Robins, A. (1993). Catastrophic forgetting in neural networks: the role of rehearsal mecha- nisms. InProceedings 1993 The First New Zealand International Two-Stream Conference on Artificial Neural Networks and Expert Systems, pages 65–68. IEEE. Robins, A. (1995). Catastroph...

    1993

  12. [30]

    Asenov, M., Burke, M., Angelov, D., Davchev, T., Subr, K., and Ramamoorthy, S. (2019). Vid2param: Modeling of dynamics parameters from video.IEEE Robotics and Automation Letters, 5(2):414–421. Atkinson, C., McCane, B., Szymanski, L., and Robins, A. (2021). Pseudo-rehearsal: Ac...

    2019

  13. [31]

    Klasson, M., Kjellstr¨ om, H., and Zhang, C. (2022). Learn the time to learn: Replay scheduling in continual learning.arXiv preprint arXiv:2209.08660. Klinzing, J. G., Niethard, N., and Born, J. (2019). Mechanisms of systems memory consolidation during sleep.Nature neuroscienc...

    2022 arXiv

  14. [32]

    and Precup, D

    Anand, N. and Precup, D. (2024). Prediction and control in continual reinforcement learning. Advances in Neural Information Processing Systems,

    2024

  15. [33]

    Rosenbaum, C., Klinger, T., and Riemer, M. (2017). Routing networks: Adaptive selection of non-linear functions for multi-task learning.arXiv preprint arXiv:1711.01239. Ross, S., Gordon, G., and Bagnell, D. (2011). A reduction of imitation learning and structured prediction to...

    2017 arXiv

  16. [34]

    R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., and Summers, R

    Shin, H.-C., Roth, H. R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., and Summers, R. M. (2016). Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning.IEEE transactions on medical imagin...

    2016 arXiv

  17. [35]

    Traor´ e, R., Caselles-Dupr´ e, H., Lesort, T., Sun, T., Cai, G., D ´ ıaz-Rodr ´ ıguez, N., and Filliat, D. (2019). Discorl: Continual reinforcement learning via policy distillation.arXiv preprint arXiv:1907.05855. Tse, D., Langston, R. F., Kakeyama, M., Bethus, I., Spooner, P...

    2019 arXiv

  18. [36]

    Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Pieter Abbeel, O., and Zaremba, W. (2017). Hindsight experience replay. Advances in neural information processing systems,

    2017

  19. [37]

    Verma, T., Jin, L., Zhou, J., Huang, J., Tan, M., Choong, B. C. M., Tan, T. F., Gao, F., Xu, X., Ting, D. S., et al. (2023). Privacy-preserving continual learning methods for medical image classification: a comparative analysis.Frontiers in Medicine,

    2023

  20. [38]

    and Finn, C

    Xie, A. and Finn, C. (2022). Lifelong robotic reinforcement learning by retaining experiences. InConference on Lifelong Learning Agents, pages 838–855. PMLR. Xu, H., Liu, B., Shu, L., and Yu, P. S. (2018). Lifelong domain word embedding via meta-learning.arXiv preprint arXiv:1...

    2022 arXiv

  21. [39]

    Yan, S., Xie, J., and He, X. (2021). Der: Dynamically expandable representation for class incremental learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3014–3023. Yang, R., Xu, H., Wu, Y., and Wang, X. (2020). Multi-task reinf...

    2021 arXiv

  22. [40]

    and Tian, T

    Young, K. and Tian, T. (2019). Minatar: An atari-inspired testbed for thorough and reproducible reinforcement learning experiments.arXiv preprint arXiv:1903.03176. Yu, T., Quillen, D., He, Z., Julian, R., Hausman, K., Finn, C., and Levine, S. (2020). Meta-world: A benchmark an...

    2019 arXiv

  23. [82]

    Tulving, E. (2002). Episodic memory: From mind to brain.Annual review of psychology, 53(1):1–25. Van de Ven, G. M., Siegelmann, H. T., and Tolias, A. S. (2020). Brain-inspired replay for continual learning with artificial neural networks.Nature communications, 11(1):4069. Van ...

    2002 arXiv

  24. [307]

    Auddy, S., Hollenstein, J., Saveriano, M., Rodr ´ ıguez-S´ anchez, A., and Piater, J. (2023). Continual learning from demonstration of robotics skills.Robotics and Autonomous Systems, 165:104427. Baevski, A., Hsu, W.-N., Xu, Q., Babu, A., Gu, J., and Auli, M. (2022). Data2vec:...

    2023 arXiv

  25. [389]

    Riemer, M., Cases, I., Ajemian, R., Liu, M., Rish, I., Tu, Y., and Tesauro, G. (2018). Learning to learn without forgetting by maximizing transfer and minimizing interference. arXiv preprint arXiv:1810.11910. Ritter, H., Botev, A., and Barber, D. (2018). Online structured lapl...

    2018 arXiv

  26. [1476]

    D., Jeong, J., and Kim, G

    Kim, C. D., Jeong, J., and Kim, G. (2020). Imbalanced continual learning with partitioning reservoir sampling. InComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIII 16, pages 411–428. Springer. Kirkpatrick, J., Pascanu, ...

    2020

  27. [1724]

    C., Fishman, S., Korattikara, A., Canny, J., and Guadarrama, S

    Chan, S. C., Fishman, S., Korattikara, A., Canny, J., and Guadarrama, S. (2020). Measuring the reliability of reinforcement learning algorithms. InInternational Conference on Learning Representations. Chaudhry, A., Dokania, P. K., Ajanthan, T., and Torr, P. H. (2018a). Riemann...

    2020 arXiv

  28. [4309]

    Gaya, J.-B., Doan, T., Caccia, L., Soulier, L., Denoyer, L., and Raileanu, R. (2022). Building a subspace of policies for scalable continual learning.arXiv preprint arXiv:2211.10445. Gaya, J.-B., Soulier, L., and Denoyer, L. (2021). Learning a subspace of policies for online a...

    2022 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.