Continual Learning in Transition

Dan Zhang; Haiyun Guo; Haokai Ma; Hongyan An; Jinqiao Wang; Junfeng Fang; Liyuan Wang; Tao Feng; Tat-Seng Chua; Wei Li

REVIEW 2 major objections 6 minor 221 references

Continual Learning in Transition

T0 review · 2 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Continual learning is transitioning from parameter-centric learning to system-level adaptation.

desk verdict Genuinely useful When-Where-How reframing of continual learning, with a How axis that overextends one RL finding and a method selection tilted toward the authors' own work. read the letter →

arxiv 2608.06216 v2 pith:OBYZRM7G submitted 2026-08-06 cs.LG cs.AI

Zhiyan Hou , Dan Zhang , Tao Feng , Liyuan Wang , Wei Li , Xiangzhao Hao , Hongyan An , Junfeng Fang

show 6 more authors

Haokai Ma Zhaohui Xu Xinyu Tang Haiyun Guo Jinqiao Wang Tat-Seng Chua

This is my paper · ORCID

classification cs.LGcs.AI

keywords continuallearninglargelanguagemodelsagenticAIcatastrophicforgettingtest-timetrainingmodelmergingmemorysystemsskilllibraries

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This survey argues that continual learning is outgrowing its classical definition as the problem of suppressing catastrophic forgetting in a single model during sequential training. It claims the field is instead becoming a question of how a whole system — model plus external memory, skills, and interaction protocols — keeps gaining capability over its full lifecycle. The paper organizes that transition along three axes: when capability evolves, where it is carried, and how updates are driven. Classical continual learning appears as one specific point on these axes. If the framing is right, forgetting is one narrow sub-problem inside a larger design question: deciding how a system should store, update, combine, and release capability across its components.

What carries the argument

The central machinery is the When–Where–How profile. When names the lifecycle stage at which capability changes, spanning pre-training, post-training, and inference-time; Where names the carrier, either model parameters or the external harness of memory, skills, and protocols; How names the update mechanism, from off-policy gradient updates and on-policy gradient updates to learning beyond gradients, such as model merging, zeroth-order optimization, heuristic learning, and prompt evolution. This profile does the work of placing each surveyed method at a coordinate and making dense and empty regions visible, so the framework both organizes existing methods and points to underexplored combinations.

What would settle it

Run the same sequence of diverse post-training tasks twice, once with off-policy supervised fine-tuning on static data and once with on-policy reinforcement learning over current-policy rollouts, holding final-task performance equal and measuring both KL divergence from the reference policy and retention on earlier tasks; if any off-policy method constrained to the same small KL forgets as little as on-policy learning, the claimed structural advantage of the How axis is refuted.

Watch

Extended reading notes

Core claim

The paper's central claim is that the field of continual learning is transitioning from parameter-centric learning to system-level adaptation. The evidence cited is the convergence of several recently developed method families: continued pre-training, multi-stage post-training, test-time training, reinforcement-learning-based alignment, model merging, zeroth-order optimization, and harness-level accumulation through memory, skills, and protocols. Each family relaxes one of three assumptions the classical setting held fixed: learning happens only in a dedicated training stage; capability is carried only in model parameters; and updates are driven only by gradients over externally supplied data. The paper therefore recasts continual learning as continual capability evolution, characterized by when, where, and how learning occurs, with classical continual learning as one coordinate point in this larger space.

Load-bearing premise

The How axis rests on a single cited result — on-policy reinforcement learning forgets less than off-policy fine-tuning because it implicitly makes updates with minimal KL divergence from the reference policy — and if that result does not hold in the broader post-training and inference-time settings surveyed here, the off-policy/on-policy distinction is not the fundamental axis the paper claims.

Editorial extensions

If this is right

Forgetting becomes multi-faceted: not just parameter overwriting, but retrieval decay in memory, context-window overflow, and capability mismatch in skills and protocols.
Evaluation should move toward long-horizon agent trajectories and per-carrier metrics, since short static benchmarks measure shipped capability rather than capability gained during operation.
Hand-tuned harness engineering can look like continual learning but does not mechanistically constitute it, because the agent of revision is a person rather than the system's own closed loop.
A key open mechanism is coordinated evolution of model and harness: bidirectional transfer between parameters and external carriers, scheduling across carriers, and active forgetting as capability release.
The survey prioritizes continual learning for the frontier model over domain-specific continual learning on the path to AGI.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the tri-axial frame is adopted, single-number forgetting scores become less informative; a natural next step is to require each method to report a (when, where, how) profile and separate retention metrics per carrier.
The survey's open scheduling question — what to keep in context, write to memory, promote to skills, and consolidate into parameters — has a formal analog in memory-hierarchy management and could be made testable by measuring the cost and retention of different carrier assignments on long-horizon agent tasks.
The paper does not run this comparison, but its KL-based account implies that an off-policy update with an explicit small-KL constraint should close much of the forgetting gap with on-policy learning; if it does, the on-policy/off-policy axis is a proxy for update conservatism rather than a distinct mechanism.
The paper's position that harness engineering is not yet a learning mechanism implies a testable criterion: a harness-based system counts as continually learning only if it improves without human edits to its memory, skills, or protocols over a held-out horizon.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Hou et al.'s survey is the most serious attempt I've seen to re-organize continual learning around something other than the classical replay/regularization/architecture/gradient taxonomy. The When-Where-How frame (lifecycle stage, capability carrier, update mechanism) genuinely brings test-time training, memory/skill/protocol harnesses, model merging, and RL-based post-training into one coordinate system. That is a real service. The paper is also careful about its own limits: figures are labeled non-exhaustive, profiles non-exclusive, and the limitations section is honest.

The soft spot is the How axis. Section 3.4.1 asks a single finding—Shenfeld et al.'s small-KL property of on-policy RL—to carry the claim that on-policy is structurally aligned with forgetting prevention. That is a lot of weight for one result, and the paper generalizes it to test-time RL without showing the KL-minimization mechanism survives noisy rewards or weak verifiers. More basically, the off-policy/on-policy distinction is really doing two jobs: data source and update mechanism. SDFT trains on the model's own generations but is classed off-policy because it lacks reward conditioning; that is defensible, but it means 'on-policy' is really 'reward-conditioned self-generation,' not a fundamental axis of update mechanisms. I'd want that reframed or hedged.

Also fair to flag: the representative-method selection leans heavily on the authors' own groups (SRPO, ZeroFlow, SEEKR, ReCALL, MLLM-CTBench all appear with co-authors here). The paper says the figures aren't exhaustive but never explains the selection procedure, and the self-bias reduces the utility of the density/emptiness claims. Minor: ref [40] is a federated scheduling paper, irrelevant where cited. Easily fixed.

The structural-ceiling discussion in Section 4.1 is more essay than proof, but it is clearly labeled as a position, so I don't hold it against the taxonomy. The core taxonomy is internally consistent and non-circular—it is an interpretation, not a derived result. I would send this to peer review. A good referee can get the authors to rework the How axis, add selection criteria, and fix the citation. The When/Where/How framing will be useful even if the specific on-policy claim does not generalize.

Referee Report

2 major / 6 minor

Summary. This survey argues that continual learning (CL) in the era of large language models and agentic AI is transitioning from parameter-centric sequential weight updates to system-level adaptation across the full model lifecycle. The authors propose a tri-axial taxonomy: When capability evolves (pre-training, post-training, inference time), Where capability is carried (parameters versus the external harness of memory, skills, and protocols), and How updates are driven (off-policy gradients, on-policy learning, and learning beyond gradients). Classical CL is positioned as one point in this space, and representative LLM-era methods are profiled along the axes, with a benchmark catalogue and cross-dimensional profiles. The paper also discusses structural ceilings of context, why harness engineering alone is insufficient, coordinated model-harness evolution, long-horizon agents as testbeds, and AGI priorities.

Significance. If the taxonomy is accepted, it provides a useful common language for organizing a fragmented literature and broadens CL's problem statement from forgetting suppression to lifecycle-level system design. The paper is honest about its scope: profiles are non-exclusive, figures are non-exhaustive, and limitations are explicitly stated. Its synthesis of memory, skills, protocols, inference-time updates, and gradient-free mechanisms in one framework is a genuine contribution, and the benchmark catalogue (Table B.1) and cross-dimensional profiles (Table C.1) are useful reference material. The main weakness is the How axis, where the on-policy/off-policy distinction is made to carry more weight than the cited evidence supports; this does not invalidate the taxonomy, but the structural claim needs revision before it can be accepted as stated.

major comments (2)

[Section 3.4.1] The paragraph beginning 'The genuinely new mechanism that the LLM era brings to the How axis is on-policy learning' elevates a single empirical result, Shenfeld et al. [27], to a structural property: it asserts that on-policy RL implicitly prefers the minimum-KL solution and therefore forgets less, while off-policy supervised fine-tuning can drift arbitrarily far. The paper then extends this to all on-policy post-training and to TTRL [164] at test time without showing that the KL-minimization mechanism survives noisy rewards, weak verifiers, or settings without explicit KL penalties. This is load-bearing because the How axis is one of the paper's three pillars; if the claim is only an empirical regularity, the axis should be presented as such, with stated boundary conditions or additional independent evidence.
[Section 3.4.1, Table C.1, and Section 5] The off-policy/on-policy distinction is operationalized as 'training data is generated by the very policy being updated, conditioned on reward signals,' which conflates data source with reward conditioning. This makes the placement of SDFT [161] as off-policy appear contradictory: its data are self-generated, but collected before training from a fixed reference policy, whereas on-policy self-distillation [29] uses the current policy. The reader is left to infer that reward conditioning, not the update mechanism, is the actual differentiator. Please give a single precise definition and apply it uniformly, explicitly distinguishing the two SDFT methods, or separate 'data source' and 'reward conditioning' into independent properties; the paper's own limitation statement about boundary crossings concerns cross-axis methods and does not resolve this definitional ambiguity.

minor comments (6)

[Title and Section 5] The title contains an unintended space in 'T ransition,' and the first sentence of Section 5 reads 'his survey reframes' instead of 'This survey reframes.'
[Section 3.5 and Figure 2 caption] The sentence 'Figures 2 contains selected representative methods and do not result from an exhaustive literature-enumeration protocol' is ungrammatical; please revise to 'Figure 2 contains selected representative methods and does not result from an exhaustive enumeration protocol.'
[Section 3.4.1 and Table A.2] The distinction between SDFT [161] and the on-policy self-distillation method [29] is made only in Table A.2; readers of Section 3.4.1 will benefit from an explicit note in the main text that the two methods differ in whether the self-generated data come from a fixed reference policy or from the policy being updated.
[Section 4.1] The 'needle-in-a-haystack' phenomenon is invoked without a citation; please include the original or a representative reference.
[Section 3.4.2] The phrase describing ZeroFlow as having an 'intrinsic stability property paralleling the small-KL behavior of on-policy RL' is stronger than the summarized evidence; if the on-policy claim is softened in Section 3.4.1, this parallel should be softened accordingly.
[References] References [32], [188], and [191] are blog posts or essays; they are labeled in the bibliography, but the in-text citations should also identify them as non-archival commentary at first mention.

Circularity Check

0 steps flagged · score 0.0 of 10

Survey taxonomy is interpretive; no derivation reduces to its inputs—no significant circularity.

full rationale

This manuscript is a survey and taxonomy rather than a derivation: it proposes a When/Where/How framing of continual learning, and its claims are classifications of existing methods, not predictions derived from equations. There are no fitted parameters, no experiments, and no first-principles result that is claimed to follow from an input. The load-bearing empirical anchor of the How axis—that on-policy RL forgets less because it implicitly minimizes KL to the reference policy—is explicitly attributed to an external source, Shenfeld et al. [27], and the paper presents it as a cited finding ('Shenfeld et al. [27] show that ...'), not as a result derived within this paper. The harness taxonomy (memory, skills, protocols) is likewise adopted from external [36]. The paper does contain self-citations to representative methods from the authors' groups (SRPO, SEEKR, ReCALL, MLLM-CTBench, ZeroFlow, C-Flat series), but these serve as examples placed in the taxonomy, not as premises that force the taxonomy; Section 3.5 and Appendix C explicitly disclaim exhaustive enumeration and state that region density should not be read as a statistical estimate. The apparent SDFT inconsistency is resolved by the paper's own note that Figure 2's SDFT label refers to the on-policy method of [29], while Section 3.4.1's off-policy SDFT cites [161]; this is a labeling clarification, not a definitional reduction. The remaining concern—whether the KL-minimization result generalizes to noisy rewards, weak verifiers, and test-time RL—is an empirical-robustness question, not a circularity. Under the requirement to exhibit a specific reduction rather than infer intent, no circular step can be quoted, so the honest finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The survey's central claim is a taxonomy, so it has no fitted numerical parameters. It does inherit several substantive assumptions from the surrounding literature, the most load-bearing being the on-policy KL-forgetting result and the partition of the harness into memory/skills/protocols. No new entities are invented; the When/Where/How axes are analytical dimensions, not postulated objects.

assumptions (5)

domain assumption Classical continual learning is fully characterized by three implicit assumptions: training-stage updates, parameter carriers, off-policy gradient updates.
Section 2.1 presents this as a reading of the classical literature; the survey's whole framing depends on this historical claim.
domain assumption On-policy RL implicitly selects minimal-KL solutions and therefore forgets less than off-policy SFT.
Section 3.4.1 treats the result of Shenfeld et al. [27] as the structural basis of the How axis; this is a prior empirical/theoretical result, not derived here.
domain assumption Memory, skills, and protocols are a legitimate partition of the harness and count as carriers of continual learning.
Section 3.3.2 follows [36]; the classification of harness edits as continual learning is definitional.
domain assumption The context window has hard and soft ceilings (length limit, attention dilution) so context-only adaptation cannot replace continual learning.
Section 4.1 argues this from needle-in-a-haystack observations; no formal proof is provided.
domain assumption AGI requires continual learning and verifiable rewards are a key bridge to autonomous improvement.
Section 4.5 states this as a priority judgment, consistent with the authors' position but not established by the survey.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Continual Learning in Transition." pith.science (2026). https://pith.science/paper/OBYZRM7G

@misc{pith2026260806216,
  author       = {Pith},
  title        = {Pith review of: Continual Learning in Transition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OBYZRM7G}},
  note         = {Machine review of arXiv:2608.06216}
}

read the original abstract

Classical continual learning (CL) has primarily focused on enabling models to update and retain knowledge through parameter-centric mechanisms, e.g., training strategies, architectural designs, and weight adaptation. However, emerging paradigms are reshaping the scope of CL beyond this traditional model adaptation view. For instance, on-policy learning broadens the space of update mechanisms; test-time training extends CL from the training phase to inference; and external harness components such as memory, skill libraries, and interaction protocols extend the evolutionary boundaries of model capabilities far beyond the static parameter space. Collectively, these developments indicate a transition from parameter-centric learning toward system-level adaptation. To characterize this transition, we examine the evolution of continual learning through three dimensions: When, How, and Where learning occurs. The How dimension encompasses off-policy, on-policy, and beyond-gradient optimization mechanics. The When dimension captures evolution across pre-training, post-training, and inference-time stages. The Where dimension delineates updates occurring within internal parameters versus external structural constraints. Anchored by this tri-axial framework, we systematically survey representative methods, trace the ongoing transition of continual learning, and discuss the key challenges, broader implications, and future directions arising from this paradigm shift.

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

221 extracted references · 10 canonical work pages

[40]

Papka, and Venkatram Vishwanath

Aditya Tanikanti, Benoit Côté, Yanfei Guo, Le Chen, Nickolaus Saint, Ryan Chard, Ken Raffenetti, Rajeev Thakur, Thomas Uram, Ian Foster, Michael E. Papka, and Venkatram Vishwanath. FIRST: Federated inference resource scheduling toolkit for scientific ai model access. InProceedings of the SC ’25 Workshops of the Inter- national Conference for High Performa...
[27]

RL’s razor: Why online reinforcement learning forgets less,

Idan Shenfeld, Jyothish Pari, and Pulkit Agrawal. RL’s razor: Why online reinforcement learning forgets less,
[164]

TTRL: Test-time reinforcement learning, 2025

Yuxin Zuo, Kaiyan Zhang, Li Sheng, Shang Qu, Ganqu Cui, Xuekai Zhu, Haozhan Li, Yuchen Zhang, Xinwei Long, Ermo Hua, Biqing Qi, Youbang Sun, Zhiyuan Ma, Lifan Yuan, Ning Ding, and Bowen Zhou. TTRL: Test-time reinforcement learning, 2025. URLhttps://arxiv.org/abs/2504.16084

arXiv 2025
[161]

Self-distillation bridges distribution gap in language model fine-tuning

Zhaorui Yang, Qian Liu, Tianyu Pang, Han Wang, Haozhe Feng, Minfeng Zhu, and Wei Chen. Self-distillation bridges distribution gap in language model fine-tuning. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024. URLhttps://arxiv.org/abs/2402.13669

arXiv 2024
[29]

Self-distillation enables continual learning,

Idan Shenfeld, Mehul Damani, Jonas Hübotter, and Pulkit Agrawal. Self-distillation enables continual learning,
[1]

GPT-4 technical report, 2023

OpenAI. GPT-4 technical report, 2023. URLhttps://arxiv.org/abs/2303.08774

arXiv 2023
[2]

ChatGLM: A family of large language models from GLM-130B to GLM-4 all tools, 2024

GLM Team. ChatGLM: A family of large language models from GLM-130B to GLM-4 all tools, 2024. URL https://arxiv.org/abs/2406.12793

arXiv 2024
[3]

Qwen technical report, 2023

Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report, 2023. URLhttps://arxiv.org/abs/2309.16609

arXiv 2023

Show all 221 references

[4]

DeepSeek-R1 incentivizes reasoning in LLMs through reinforce- ment learning.Nature, 645:633–638, 2025

Daya Guo, Dejian Yang, Haowei Zhang, et al. DeepSeek-R1 incentivizes reasoning in LLMs through reinforce- ment learning.Nature, 645:633–638, 2025. doi: 10.1038/s41586-025-09422-z. URLhttps://doi.org/10.1038/ s41586-025-09422-z

2025 doi
[5]

Kimi K2.5: Visual agentic intelligence, 2026

Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao, SH Cai, Yuan Cao, Y Charles, HS Che, Cheng Chen, Guanduo Chen, et al. Kimi K2.5: Visual agentic intelligence, 2026. URLhttps://arxiv.org/abs/2602.02276

2026 arXiv
[6]

GLM-4.5: Agentic, reasoning, and coding (ARC) foundation models, 2025

Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, et al. GLM-4.5: Agentic, reasoning, and coding (ARC) foundation models, 2025. URL https://arxiv.org/abs/2508.06471

2025 arXiv
[7]

GLM-5: From vibe coding to agentic engineering, 2026

Aohan Zeng, Xin Lv, Zhenyu Hou, Zhengxiao Du, Qinkai Zheng, Bin Chen, Da Yin, Chendi Ge, Chengxing Xie, Cunxiang Wang, et al. GLM-5: From vibe coding to agentic engineering, 2026. URLhttps://arxiv.org/ abs/2602.15763

2026 arXiv
[8]

ReST-MCTS*: LLM self- training via process-reward-guided tree search

Dan Zhang, Sining Zhoubian, Ziniu Hu, Yisong Yue, Yuxiao Dong, and Jie Tang. ReST-MCTS*: LLM self- training via process-reward-guided tree search. InNeurIPS, 2024

2024
[9]

TDRM: Smooth reward models with temporal difference for LLM RL and inference, 2025

Dan Zhang, Min Cai, Jonathan Light, Ziniu Hu, Yisong Yue, and Jie Tang. TDRM: Smooth reward models with temporal difference for LLM RL and inference, 2025. URLhttps://arxiv.org/abs/2509.15110

2025
[10]

ReST-RL: Achieving accurate code reasoning of LLMs with optimized self-training and decoding, 2025

Sining Zhoubian, Dan Zhang, and Jie Tang. ReST-RL: Achieving accurate code reasoning of LLMs with optimized self-training and decoding, 2025. URLhttps://arxiv.org/abs/2508.19576

2025 arXiv
[11]

Scenegenagent: Precise industrial scene generation with coding agent.arXiv preprint arXiv:2410.21909, 2024

Xiao Xia, Dan Zhang, Zibo Liao, Zhenyu Hou, Tianrui Sun, Jing Li, Ling Fu, and Yuxiao Dong. Scenegenagent: Precise industrial scene generation with coding agent.arXiv preprint arXiv:2410.21909, 2024

2024 arXiv
[12]

MemoryLLM: Towards self-updatable large language models

Yu Wang, Yifan Gao, Xiusi Chen, Haoming Jiang, Shiyang Li, Jingfeng Yang, Qingyu Yin, Zheng Li, Xian Li, Bing Yin, Jingbo Shang, and Julian McAuley. MemoryLLM: Towards self-updatable large language models. arXiv preprint arXiv:2402.04624, 2024. 17

2024 arXiv
[13]

Voyager: An open-ended embodied agent with large language models, 2023

Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models, 2023. URLhttps:// arxiv.org/abs/2305.16291

2023 arXiv
[14]

Reflexion: Language agents with verbal reinforcement learning, 2023

Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning, 2023. URLhttps://arxiv.org/abs/2303. 11366

2023
[15]

Patil, Ion Stoica, and Joseph E

Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. MemGPT: Towards LLMs as operating systems.arXiv preprint arXiv:2310.08560, 2023

2023 arXiv
[16]

MemoryBank: Enhancing large language models with long-term memory

Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. MemoryBank: Enhancing large language models with long-term memory. InProceedings of the AAAI Conference on Artificial Intelligence, 2024. URL https://arxiv.org/abs/2305.10250

2024 arXiv
[17]

AgentEvolver: Towards efficient self-evolving agent system, 2025

Yunpeng Zhai, Shuchang Tao, Cheng Chen, Anni Zou, Ziqian Chen, Qingxu Fu, Shinji Mai, Li Yu, Jiaji Deng, Zouying Cao, Zhaoyang Liu, Bolin Ding, and Jingren Zhou. AgentEvolver: Towards efficient self-evolving agent system, 2025. URLhttps://arxiv.org/abs/2511.10395

2025
[18]

Catastrophic interference in connectionist networks: The sequential learning problem

Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. InPsychology of learning and motivation, volume 24, pages 109–165. Elsevier, 1989

1989
[19]

Towards continual reinforcement learning: A review and perspectives.Journal of Artificial Intelligence Research, 75:1401–1476, 2022

Khimya Khetarpal, Matthew Riemer, Irina Rish, and Doina Precup. Towards continual reinforcement learning: A review and perspectives.Journal of Artificial Intelligence Research, 75:1401–1476, 2022

2022
[20]

A comprehensive survey of continual learning: Theory, method and application, 2024

Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application, 2024. URLhttps://arxiv.org/abs/2302.00487

2024 arXiv
[21]

Continual learning for large language models: A survey.arXiv preprint arXiv:2402.01364, 2024

Tongtong Wu, Linhao Luo, Yuan-Fang Li, Shirui Pan, Thuy-Trang Vu, and Gholamreza Haffari. Continual learning for large language models: A survey.arXiv preprint arXiv:2402.01364, 2024

2024 arXiv
[22]

Training language models to follow instructions with human feedback.Advances in Neural Information Processing Systems, 35:27730–27744, 2022

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback.Advances in Neural Information Processing Systems, 35:27730...

2022
[23]

DeepSeekMath: Pushing the limits of mathematical reasoning in open language models

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

2024 arXiv
[24]

Unifying group-relative and self-distillation policy optimization via sample routing

Gengsheng Li, Tianyu Yang, Junfeng Fang, Mingyang Song, Mao Zheng, Haiyun Guo, Dan Zhang, Jinqiao Wang, and Tat-Seng Chua. Unifying group-relative and self-distillation policy optimization via sample routing. arXiv preprint arXiv:2604.02288, 2026

2026
[25]

KnowRL: Boosting LLM reasoning via reinforcement learning with minimal-sufficient knowledge guidance, 2026

Linhao Yu, Tianmeng Yang, Siyu Ding, Renren Jin, Naibin Gu, Xiangzhao Hao, Shuaiyi Nie, Deyi Xiong, Weichong Yin, Yu Sun, and Hua Wu. KnowRL: Boosting LLM reasoning via reinforcement learning with minimal-sufficient knowledge guidance, 2026. URLhttps://arxiv.org/abs/2604.12627

2026 arXiv
[26]

CLEAR: Unlocking generative potential for degraded image understanding in unified multimodal models, 2026

Xiangzhao Hao, Zefeng Zhang, Zhenyu Zhang, Linhao Yu, Yao Chen, Yiqian Zhang, Haiyun Guo, Shuohuan Wang, and Yu Sun. CLEAR: Unlocking generative potential for degraded image understanding in unified multimodal models, 2026. URLhttps://arxiv.org/abs/2604.04780

2026 arXiv
[28]

On-policy distillation of language models: Learning from self-generated mistakes

Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self-generated mistakes. InInternational Conference on Learning Representations (ICLR), 2024. URLhttps://a...

2024 arXiv
[30]

Model merging in LLMs, MLLMs, and beyond: Methods, theories, applications and opportunities.arXiv preprint arXiv:2408.07666, 2024

Enneng Yang, Li Shen, Guibing Guo, Xingwei Wang, Xiaochun Cao, Jie Zhang, and Dacheng Tao. Model merging in LLMs, MLLMs, and beyond: Methods, theories, applications and opportunities.arXiv preprint arXiv:2408.07666, 2024. 18

2024 arXiv
[31]

Lee, Danqi Chen, and Sanjeev Arora

Sadhika Malladi, Tianyu Gao, Eshaan Nichani, Alex Damian, Jason D. Lee, Danqi Chen, and Sanjeev Arora. Fine-tuning language models with just forward passes. InAdvances in Neural Information Processing Systems (NeurIPS), 2023. URLhttps://arxiv.org/abs/2305.17333

2023 arXiv
[32]

Learning beyond gradients

Jiayi Weng. Learning beyond gradients. Blog post, 2026. URLhttps://trinkle23897.github.io/ learning-beyond-gradients/. Contextual overview rather than a primary technical source

2026
[33]

Prompt- breeder: Self-referential self-improvement via prompt evolution, 2023

Chrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero, and Tim Rocktäschel. Prompt- breeder: Self-referential self-improvement via prompt evolution, 2023. URLhttps://arxiv.org/abs/2309. 16797

2023
[34]

Tent: Fully test-time adaptation by entropy minimization

Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. InInternational Conference on Learning Representations, 2021. URL https://arxiv.org/abs/2006.10726

2021 arXiv
[35]

Learning to (learn at test time): RNNs with expressive hidden states

YuSun, XinhaoLi, KaranDalal, JiaruiXu, ArjunVikram, GenghanZhang, YannDubois, XinleiChen, Xiaolong Wang, Sanmi Koyejo, Tatsunori Hashimoto, and Carlos Guestrin. Learning to (learn at test time): RNNs with expressive hidden states. InProceedings of the 42nd International Confer...

2025
[36]

Externalization in LLM agents: A unified review of memory, skills, protocols and harness engineering, 2026

Chenyu Zhou, Huacan Chai, Wenteng Chen, Zihan Guo, Rong Shan, Yuanyi Song, Tianyi Xu, Yingxuan Yang, Aofan Yu, Weiming Zhang, Congming Zheng, Jiachen Zhu, Zeyu Zheng, Zhuosheng Zhang, Xingyu Lou, Changwang Zhang, Zhihui Fu, Jun Wang, Weiwen Liu, Jianghao Lin, and Weinan Zhang....

2026 arXiv
[37]

An empirical investigation of catastrophic forgetting in gradient-based neural networks.arXiv preprint arXiv:1312.6211, 2013

Ian J Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empirical investigation of catastrophic forgetting in gradient-based neural networks.arXiv preprint arXiv:1312.6211, 2013

2013 arXiv
[38]

icarl: Incremental classifier and representation learning

Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. InProceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 2001–2010, 2017

2001
[39]

Experience replay for continual learning.Advances in neural information processing systems, 32, 2019

David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning.Advances in neural information processing systems, 32, 2019

2019
[41]

Gradient episodic memory for continual learning

David Lopez-Paz and Marc’Aurelio Ranzato. Gradient episodic memory for continual learning. InAdvances in Neural Information Processing Systems, 2017

2017
[42]

Orthogonal gradient descent for continual learning

Mehrdad Farajtabar, Navid Azizan, Alex Mott, and Ang Li. Orthogonal gradient descent for continual learning. InInternational conference on artificial intelligence and statistics, pages 3762–3773. PMLR, 2020

2020
[43]

URLhttps://doi.org/10.1145/3731599.3767346

doi: 10.1145/3731599.3767346. URLhttps://doi.org/10.1145/3731599.3767346
[44]

A faster path to continual learning

Wei Li, Hangjie Yuan, Zixiang Zhao, Borui Kang, Ziwei Liu, and Tao Feng. A faster path to continual learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 25088–25098, June 2026

2026
[45]

Rethinking the stability-plasticity trade-off in continual learning from an architectural perspective

Aojun Lu, Hangjie Yuan, Tao Feng, and Yanan Sun. Rethinking the stability-plasticity trade-off in continual learning from an architectural perspective. In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu...

2025
[46]

Make continual learning stronger via c-flat.Advances in Neural Information Processing Systems, 37: 7608–7630, 2024

Ang Bian, Wei Li, Hangjie Yuan, Chengrong Yu, Mang Wang, Zixiang Zhao, Aojun Lu, Pengliang Ji, and Tao Feng. Make continual learning stronger via c-flat.Advances in Neural Information Processing Systems, 37: 7608–7630, 2024

2024
[47]

Packnet: Adding multiple tasks to a single network by iterative pruning

Arun Mallya and Svetlana Lazebnik. Packnet: Adding multiple tasks to a single network by iterative pruning. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018

2018
[48]

Overcoming catastrophic forgetting with hard attention to the task

Joan Serrà, Dídac Surís, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. InInternational Conference on Machine Learning, 2018

2018
[49]

Revisiting neural networks for continual learning: An architectural perspective

Aojun Lu, Tao Feng, Hangjie Yuan, Xiaotian Song, and Yanan Sun. Revisiting neural networks for continual learning: An architectural perspective. In Kate Larson, editor,Proceedings of the Thirty-Third International 19 Joint Conference on Artificial Intelligence, IJCAI-24, pages...

2024 doi
[50]

Overcoming catastrophic forgetting in incremental object detection via elastic response distillation

Tao Feng, Mang Wang, and Hangjie Yuan. Overcoming catastrophic forgetting in incremental object detection via elastic response distillation. InCVPR, pages 9417–9426. IEEE, 2022

2022
[51]

Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, DharshanKumaran, andRaiaHadsell

James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, DharshanKumaran, andRaiaHadsell. Overcomingcatastrophicforgettinginneura...

2017 doi
[52]

Rusu, Neil C

Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks.arXiv preprint arXiv:1606.04671, 2016

2016 arXiv
[53]

Memory aware synapses: Learning what (not) to forget, 2018

Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget, 2018. URLhttps://arxiv.org/abs/1711.09601

2018 arXiv
[54]

Learning without forgetting.IEEE transactions on pattern analysis and machine intelligence, 40(12):2935–2947, 2017

Zhizhong Li and Derek Hoiem. Learning without forgetting.IEEE transactions on pattern analysis and machine intelligence, 40(12):2935–2947, 2017

2017
[55]

Continual learning through synaptic intelligence

Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. InInter- national conference on machine learning, pages 3987–3995. PMLR, 2017

2017
[56]

Richter, Quentin Anthony, Eugene Belilovsky, Irina Rish, and Timothée Lesort

Kshitij Gupta, Benjamin Thérien, Adam Ibrahim, Mats L. Richter, Quentin Anthony, Eugene Belilovsky, Irina Rish, and Timothée Lesort. Continual pre-training of large language models: How to (re)warm your model?,
[57]

Towards continual knowledge learning of language models

Joel Jang, Seonghyeon Ye, Sohee Yang, Joongbo Shin, Janghoon Han, Gyeonghun Kim, Stanley Jungkyu Choi, and Minjoon Seo. Towards continual knowledge learning of language models. InInternational Conference on Learning Representations (ICLR), 2022

2022
[58]

Richter, Quentin Anthony, Timothée Lesort, Eugene Belilovsky, and Irina Rish

Adam Ibrahim, Benjamin Thérien, Kshitij Gupta, Mats L. Richter, Quentin Anthony, Timothée Lesort, Eugene Belilovsky, and Irina Rish. Simple and scalable strategies to continually pre-train large language models, 2024. URLhttps://arxiv.org/abs/2403.08763

2024 arXiv
[59]

TimeLMs: Diachronic language models from twitter

Daniel Loureiro, Francesco Barbieri, Leonardo Neves, Luis Espinosa Anke, and Jose Camacho-Collados. TimeLMs: Diachronic language models from twitter. InProceedings of ACL 2022: System Demonstrations, 2022

2022
[60]

Large language model empowered recommendation meets all-domain continual pre-training.IEEE Transactions on Knowledge and Data Engineering, pages 1–14, 2026

Haokai Ma, Yunshan Ma, Ruobing Xie, Lei Meng, Jialie Shen, Xingwu Sun, Zhanhui Kang, and Tat-Seng Chua. Large language model empowered recommendation meets all-domain continual pre-training.IEEE Transactions on Knowledge and Data Engineering, pages 1–14, 2026. doi: 10.1109/TKD...

2026
[61]

SAPT: A shared attention framework for parameter-efficient continual learning of large language models

Weixiang Zhao, Shilong Wang, Yulin Hu, Yanyan Zhao, Bing Qin, Xuanyu Zhang, Qing Yang, Dongliang Xu, and Wanxiang Che. SAPT: A shared attention framework for parameter-efficient continual learning of large language models. InProceedings of the 62nd Annual Meeting of the Associ...

2024
[62]

ELLE: Efficient lifelong pre-training for emerging data

Yujia Qin, Jiajie Zhang, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. ELLE: Efficient lifelong pre-training for emerging data. InFindings of the Association for Computational Linguistics: ACL 2022, 2022

2022
[63]

Mitigating catastrophic forgetting in large language models with self-synthesized rehearsal

Jianheng Huang, Leyang Cui, Ante Wang, Chengyi Yang, Xinting Liao, Linfeng Song, Junfeng Yao, and Jinsong Su. Mitigating catastrophic forgetting in large language models with self-synthesized rehearsal. InProceedings of the 62nd Annual Meeting of the Association for Computatio...

2024
[64]

Seekr: Selective attention- guided knowledge retention for continual learning of large language models.arXiv preprint arXiv:2411.06171, 2024

Jinghan He, Haiyun Guo, Kuan Zhu, Zihan Zhao, Ming Tang, and Jinqiao Wang. Seekr: Selective attention- guided knowledge retention for continual learning of large language models.arXiv preprint arXiv:2411.06171, 2024

2024 arXiv
[65]

Yu, and Xiao-Ming Wu

Yujie Feng, Hao Wang, Jian Li, Xu Chu, Zhaolu Kang, Yiran Liu, Yasha Wang, Philip S. Yu, and Xiao-Ming Wu. FOREVER: Forgetting curve-inspired memory replay for language model continual learning, 2026

2026
[66]

InsCL: A data-efficient continual learning paradigm for fine-tuning large language models with instructions

Yifan Wang, Yafei Liu, Chufan Shi, Haoling Li, Chen Chen, Haonan Lu, and Yujiu Yang. InsCL: A data-efficient continual learning paradigm for fine-tuning large language models with instructions. InProceedings of the 2024 Conference of the North American Chapter of the Associati...

2024
[67]

Why reinforcement fine-tuning enables MLLMs preserve prior knowledge better: A data perspective.arXiv preprint arXiv:2506.23508, 2025

Zhihao Zhang, Qiaole Dong, Qi Zhang, Jun Zhao, Enyu Zhou, Zhiheng Xi, Senjie Jin, Xiaoran Fan, Yuhao Zhou, Mingqi Wu, et al. Why reinforcement fine-tuning enables MLLMs preserve prior knowledge better: A data perspective.arXiv preprint arXiv:2506.23508, 2025

2025
[68]

Skill neologisms: Towards skill-based continual learning, 2026

Antonin Berthon, Nicolas Astorga, and Mihaela van der Schaar. Skill neologisms: Towards skill-based continual learning, 2026. URLhttps://arxiv.org/abs/2605.04970

2026 arXiv
[69]

End- to-end test-time training for long context.arXiv preprint arXiv:2512.23675, 2025

Arnuv Tandon, Karan Dalal, Xinhao Li, Daniel Koceja, Marcel Rød, Sam Buchanan, Xiaolong Wang, Jure Leskovec, Sanmi Koyejo, Tatsunori Hashimoto, Carlos Guestrin, Jed McCaleb, Yejin Choi, and Yu Sun. End- to-end test-time training for long context.arXiv preprint arXiv:2512.23675, 2025

2025
[70]

KeepLoRA: Continual learning with residual gradient adaptation, 2026

Mao-Lin Luo, Zi-Hao Zhou, Yi-Lin Zhang, Yuanyu Wan, Tong Wei, and Min-Ling Zhang. KeepLoRA: Continual learning with residual gradient adaptation, 2026. URLhttps://arxiv.org/abs/2601.19659. Accepted at ICLR 2026

2026
[71]

Orthogonal subspace learning for language model continual learning

Xiao Wang, Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, and Xuan-Jing Huang. Orthogonal subspace learning for language model continual learning. InFindings of the Association for Computational Linguistics: EMNLP 2023, pages 10658–10671, 2023

2023
[72]

Progres- sive prompts: Continual learning for language models

Anastasia Razdaibiedina, Yuning Mao, Rui Hou, Madian Khabsa, Mike Lewis, and Amjad Almahairi. Progres- sive prompts: Continual learning for language models. InThe Eleventh International Conference on Learning Representations, 2023

2023
[73]

LoRAMoE: Alleviating world knowledge forgetting in large language models via MoE-style plugin

Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Wei Shen, et al. LoRAMoE: Alleviating world knowledge forgetting in large language models via MoE-style plugin. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 2024

2024
[74]

Titans: Learning to memorize at test time, 2025

Ali Behrouz, Peilin Zhong, and Vahab Mirrokni. Titans: Learning to memorize at test time, 2025

2025
[75]

Rehearsal-free modular and compositional continual learning for language models

Mingyang Wang, Heike Adel, Lukas Lange, Jannik Strötgen, and Hinrich Schütze. Rehearsal-free modular and compositional continual learning for language models. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics (N...

2024
[76]

AlphaEdit: Null-space constrained knowledge editing for language models

Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Shi Jie, Xiang Wang, Xiangnan He, and Tat-Seng Chua. AlphaEdit: Null-space constrained knowledge editing for language models. InInternational Conference on Learning Representations (ICLR), 2025

2025
[77]

Norm anchors make model edits last, 2026

Mingda Liu, Zhenghan Zhu, Ze’an Miao, and Katsuki Fujisawa. Norm anchors make model edits last, 2026

2026
[78]

SLIM: Let LLMs learn more and forget less with soft LoRA and identity mixture

Jiayi Han, Liang Du, Hongwei Du, Xiangguo Zhou, Yiwen Wu, Yuanfang Zhang, Weibo Zheng, and Donghong Han. SLIM: Let LLMs learn more and forget less with soft LoRA and identity mixture. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Associatio...

2025
[79]

CRAM: Centroid-routing and adaptive MoE for multimodal continual instruction tuning, 2026

Jun-Tao Tang, Zhen-Hao Xie, Yu-Cheng Shi, and Da-Wei Zhou. CRAM: Centroid-routing and adaptive MoE for multimodal continual instruction tuning, 2026

2026
[80]

Ziqi Wang, Chang Che, Qi Wang, Hui Ma, Zenglin Shi, Cees G. M. Snoek, and Meng Wang. Harmonious parameter adaptation in continual visual instruction tuning for safety-aligned MLLMs, 2025

2025
[81]

Hidden forgetting in continual multimodal learning: When accuracy survives but grounding fails, 2026

Qianyu Chen, Canran Xiao, and Runxuan Tang. Hidden forgetting in continual multimodal learning: When accuracy survives but grounding fails, 2026

2026
[82]

Dynamic cross-modal prompt generation for multimodal continual instruction tuning, 2026

Tao Hu and Da-Wei Zhou. Dynamic cross-modal prompt generation for multimodal continual instruction tuning, 2026

2026
[83]

Language models need sleep: Learning to self-modify and consolidate memories, 2026

Ali Behrouz, Farnoosh Hashemi, and Vahab Mirrokni. Language models need sleep: Learning to self-modify and consolidate memories, 2026

2026
[84]

Peam: Parametric embodied agent memory through contrastive internalization of experience in minecraft, 2026

Yuchen Guo, Junli Gong, Weicheng Wang, Hongmin Cai, Yiu ming Cheung, and Weifeng Su. Peam: Parametric embodied agent memory through contrastive internalization of experience in minecraft, 2026

2026
[85]

Evolving-rl: End-to-end optimization of experience-driven self-evolving capability within agents, 2026

Zhiyuan Fan, Wenwei Jin, Feng Zhang, Bin Li, Yihong Dong, Yao Hu, and Jiawei Li. Evolving-rl: End-to-end optimization of experience-driven self-evolving capability within agents, 2026

2026
[86]

Rethinking continual experience internalization for self-evolving LLM agents, 2026

Jingwen Chen, Wenkai Yang, Shengda Fan, Wenbo Nie, Chenxing Sun, Shaodong Zheng, Yangen Hu, Lu Pan, Ke Zeng, and Yankai Lin. Rethinking continual experience internalization for self-evolving LLM agents, 2026. 21

2026
[87]

Scaling self-evolving agents via parametric memory, 2026

Tao Ren, Weiyao Luo, Hui Yang, Rongzhi Zhu, Xiang Huang, Yuchuan Wu, Bingxue Chou, Jieping Ye, Jiafeng Liang, Yongbin Li, and Yijie Peng. Scaling self-evolving agents via parametric memory, 2026. URLhttps: //arxiv.org/abs/2606.04536

2026 arXiv
[88]

Aging with GRACE: Lifelong model editing with discrete key-value adaptors

Tom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. Aging with GRACE: Lifelong model editing with discrete key-value adaptors. InAdvances in Neural Information Processing Systems (NeurIPS), 2023

2023
[89]

A-MEM: Agentic memory for LLM agents, 2025

Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. A-MEM: Agentic memory for LLM agents, 2025. URLhttps://arxiv.org/abs/2502.12110

2025 arXiv
[90]

WISE: Rethinking the knowledge memory for lifelong model editing of large language models

Peng Wang, Zexi Li, Ningyu Zhang, Ziwen Xu, Yunzhi Yao, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. WISE: Rethinking the knowledge memory for lifelong model editing of large language models. InAdvances in Neural Information Processing Systems (NeurIPS), 2024

2024
[91]

ExpeL: LLM agents are experiential learners

Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. ExpeL: LLM agents are experiential learners. InProceedings of the AAAI Conference on Artificial Intelligence, 2024

2024
[92]

Agent workflow memory

Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. Agent workflow memory. InInternational Conference on Machine Learning (ICML), 2025

2025
[93]

Mem0: Building production- ready AI agents with scalable long-term memory, 2025

Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. Mem0: Building production- ready AI agents with scalable long-term memory, 2025

2025
[94]

HippoRAG: Neurobiologically inspired long-term memory for large language models

Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su. HippoRAG: Neurobiologically inspired long-term memory for large language models. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

2024
[95]

Forget to improve: On-device LLM-agent continual learning via budget-curated memory, 2026

Beining Wu, Zihao Ding, Jun Huang, and Yanxiao Zhao. Forget to improve: On-device LLM-agent continual learning via budget-curated memory, 2026

2026
[96]

Collaborative multi-agent test-time reinforcement learning for reasoning, 2026

Zhiyuan Hu, Yunhai Hu, Juncheng Liu, Shuyue Stella Li, Yucheng Wang, Zhen Xu, Xinxing Xu, See-Kiong Ng, Anh Tuan Luu, Bryan Hooi, Cynthia Breazeal, and Hae Won Park. Collaborative multi-agent test-time reinforcement learning for reasoning, 2026

2026
[97]

MemRL: Self-evolving agents via runtime reinforcement learning on episodic memory, 2026

Shengtao Zhang, Jiaqian Wang, Ruiwen Zhou, Junwei Liao, Yuchen Feng, Weinan Zhang, Ying Wen, Zhiyu Li, Feiyu Xiong, Yutao Qi, Bo Tang, and Muning Wen. MemRL: Self-evolving agents via runtime reinforcement learning on episodic memory, 2026. URLhttps://arxiv.org/abs/2601.03192

2026 arXiv
[98]

Towards scalable lifelong knowledge editing with selective knowledge suppression, 2026

Dahyun Jung, Jaewook Lee, and Heuiseok Lim. Towards scalable lifelong knowledge editing with selective knowledge suppression, 2026

2026
[99]

Mem- α: Learning memory construction via reinforcement learning, 2025

Yu Wang, Ryuichi Takanobu, Zhiqi Liang, Yuzhen Mao, Yuanzhe Hu, Julian McAuley, and Xiaojian Wu. Mem- α: Learning memory construction via reinforcement learning, 2025

2025
[100]

Memory-R2: Fair credit assignment for long-horizon memory-augmented LLM agents, 2026

Sikuan Yan, Ahmed Bahloul, Ercong Nie, Susanna Schwarzmann, Riccardo Trivisonno, Volker Tresp, and Yunpu Ma. Memory-R2: Fair credit assignment for long-horizon memory-augmented LLM agents, 2026

2026
[101]

MemBuilder: Reinforcing LLMs for long-term memory construction via attributed dense rewards, 2026

Zhiyu Shen, Ziming Wu, Fuming Lai, Shaobing Lian, and Yanghui Rao. MemBuilder: Reinforcing LLMs for long-term memory construction via attributed dense rewards, 2026

2026
[102]

Pan, Hinrich Schütze, Volker Tresp, and Yunpu Ma

Sikuan Yan, Xiufeng Yang, Zuchao Huang, Ercong Nie, Zifeng Ding, Zonggen Li, Xiaowen Ma, Jinhe Bi, Kristian Kersting, Jeff Z. Pan, Hinrich Schütze, Volker Tresp, and Yunpu Ma. Memory-r1: Enhancing large language model agents to manage and utilize memories via reinforcement lea...

2025
[103]

Marginal advantage accumulation for memory-driven agent self-evolution, 2026

Mingyu Yang, Keye Zheng, Congchao Cheng, Yujie Liu, Xingkang Lu, Fan Jiang, and Yefei Zheng. Marginal advantage accumulation for memory-driven agent self-evolution, 2026

2026
[104]

Just-in-time reinforce- ment learning: Continual learning in LLM agents without gradient updates, 2026

Yibo Li, Zijie Lin, Ailin Deng, Xuan Zhang, Yufei He, Shuo Ji, Tri Cao, and Bryan Hooi. Just-in-time reinforce- ment learning: Continual learning in LLM agents without gradient updates, 2026

2026
[105]

Le, Samira Daruki, Xiangru Tang, Vishy Tirumalashetty, George Lee, Mahsan Rofouei, Hangfei Lin, Jiawei Han, Chen-Yu Lee, and Tomas Pfister

Siru Ouyang, Jun Yan, I-Hung Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T. Le, Samira Daruki, Xiangru Tang, Vishy Tirumalashetty, George Lee, Mahsan Rofouei, Hangfei Lin, Jiawei Han, Chen-Yu Lee, and Tomas Pfister. Reasoningbank: Scaling agent self-evolving with ...

2025
[106]

Memq: Integrating q-learning into self-evolving memory agents over provenance dags, 2026

Junwei Liao, Haoting Shi, Ruiwen Zhou, Jiaqian Wang, Shengtao Zhang, Wei Zhang, Ying Wen, Zhiyu Li, Feiyu Xiong, Bo Tang, Weinan Zhang, and Muning Wen. Memq: Integrating q-learning into self-evolving memory agents over provenance dags, 2026. 22

2026
[107]

Exg: Self-evolving agents with experience graphs, 2026

Yuxin Jin, Siyuan Zhang, Hanchen Wang, Lu Qin, Ying Zhang, and Wenjie Zhang. Exg: Self-evolving agents with experience graphs, 2026

2026
[108]

The past is prologue: A plug-in controller for selective updates in sequentially evolving LLM memory, 2026

Zihan Chen, Songwei Dong, Chengshuai Shi, Peng Wang, Song Wang, Cong Shen, and Jundong Li. The past is prologue: A plug-in controller for selective updates in sequentially evolving LLM memory, 2026

2026
[109]

Simple- Mem: Efficient lifelong memory for LLM agents, 2026

Jiaqi Liu, Yaofeng Su, Peng Xia, Siwei Han, Zeyu Zheng, Cihang Xie, Mingyu Ding, and Huaxiu Yao. Simple- Mem: Efficient lifelong memory for LLM agents, 2026

2026
[110]

Learning on the job: An experience-driven self-evolving agent for long-horizon tasks, 2025

Cheng Yang, Xuemeng Yang, Licheng Wen, Daocheng Fu, Jianbiao Mei, Rong Wu, Pinlong Cai, Yufan Shen, Nianchen Deng, Botian Shi, Yu Qiao, and Haifeng Li. Learning on the job: An experience-driven self-evolving agent for long-horizon tasks, 2025

2025
[111]

Autoagent: Evolving cognition and elastic memory orchestration for adaptive agents, 2026

Xiaoxing Wang, Ning Liao, Shikun Wei, Chen Tang, and Feiyu Xiong. Autoagent: Evolving cognition and elastic memory orchestration for adaptive agents, 2026

2026
[112]

Toward generalist autonomous research via hypothesis-tree refinement, 2026

Jiajie Jin, Yuyang Hu, Kai Qiu, Qi Dai, Chong Luo, Guanting Dong, Xiaoxi Li, Tong Zhao, Xiaolong Ma, Gongrui Zhang, Zhirong Wu, Bei Liu, Zhengyuan Yang, Linjie Li, Lijuan Wang, Hongjin Qian, Yutao Zhu, and Zhicheng Dou. Toward generalist autonomous research via hypothesis-tree...

2026
[113]

Evoclinician: A self-evolving agent for multi-turn medical diagnosis via test-time evolutionary learning, 2026

Yufei He, Juncheng Liu, Zhiyuan Hu, Yulin Chen, Yue Liu, Yuan Sui, Yibo Li, Nuo Chen, Jun Hu, Bryan Hooi, Xinxing Xu, and Jiang Bian. Evoclinician: A self-evolving agent for multi-turn medical diagnosis via test-time evolutionary learning, 2026

2026
[114]

Automem: Automated learning of memory as a cognitive skill, 2026

Shengguang Wu, Hao Zhu, Yuhui Zhang, Xiaohan Wang, and Serena Yeung-Levy. Automem: Automated learning of memory as a cognitive skill, 2026

2026
[115]

SkillRL: Evolving agents via recursive skill- augmented reinforcement learning, 2026

Peng Xia, Jianwen Chen, Hanyang Wang, Jiaqi Liu, Kaide Zeng, Yu Wang, Siwei Han, Yiyang Zhou, Xujiang Zhao, Haifeng Chen, Zeyu Zheng, Cihang Xie, and Huaxiu Yao. SkillRL: Evolving agents via recursive skill- augmented reinforcement learning, 2026. URLhttps://arxiv.org/abs/2602.08234

2026 arXiv
[116]

Reinforcement learning for self-improving agent with skill library.arXiv preprint arXiv:2512.17102, 2025

Jiongxiao Wang, Qiaojing Yan, Yawei Wang, Yijun Tian, Soumya Smruti Mishra, Zhichao Xu, Megha Gandhi, and Panpan Xu. Reinforcement learning for self-improving agent with skill library.arXiv preprint arXiv:2512.17102, 2025

2025 arXiv
[117]

AutoSkill: Experience-driven lifelong learning via skill self-evolution, 2026

Yutao Yang, Junsong Li, Qianjun Pan, Bihao Zhan, Yuxuan Cai, Lin Du, Jie Zhou, Kai Chen, Qin Chen, Xin Li, Bo Zhang, and Liang He. AutoSkill: Experience-driven lifelong learning via skill self-evolution, 2026

2026
[118]

TroVE: Inducing verifiable and efficient toolboxes for solving programmatic tasks

Zhiruo Wang, Daniel Fried, and Graham Neubig. TroVE: Inducing verifiable and efficient toolboxes for solving programmatic tasks. InInternational Conference on Machine Learning (ICML), 2024

2024
[119]

Arise: Agent reasoning with intrinsic skill evolution in hierarchical reinforcement learning, 2026

Yu Li, Rui Miao, Zhengling Qi, and Tian Lan. Arise: Agent reasoning with intrinsic skill evolution in hierarchical reinforcement learning, 2026

2026
[120]

Skillgraph: Skill- augmented reinforcement learning for agents via evolving skill graphs, 2026

Xiaoyuan Li, Moxin Li, Keqin Bao, Yubo Ma, Wenjie Wang, Dayiheng Liu, and Fuli Feng. Skillgraph: Skill- augmented reinforcement learning for agents via evolving skill graphs, 2026

2026
[121]

Skill1: Unified evolution of skill-augmented agents via reinforcement learning, 2026

Yaorui Shi, Yuxin Chen, Zhengxi Lu, Yuchun Miao, Shugui Liu, Qi Gu, Xunliang Cai, Xiang Wang, and An Zhang. Skill1: Unified evolution of skill-augmented agents via reinforcement learning, 2026

2026
[122]

Reskill: Reconciling skill creation with policy optimization in agentic rl, 2026

Zelin He, Haotian Lin, Boran Han, Wei Zhu, Haoyang Fang, Bernie Wang, Xuan Zhu, Runze Li, and Matthew Reimherr. Reskill: Reconciling skill creation with policy optimization in agentic rl, 2026

2026
[123]

From atomic actions to standard operating procedures: Iterative tool optimization for self-evolving LLM agents, 2026

Haipeng Ding, Yuexiang Xie, Zhewei Wei, Yaliang Li, and Bolin Ding. From atomic actions to standard operating procedures: Iterative tool optimization for self-evolving LLM agents, 2026

2026
[124]

Agentfactory: A self-evolving framework through executable subagent accumulation and reuse, 2026

Zhang Zhang, Shuqi Lu, Hongjin Qian, Di He, and Zheng Liu. Agentfactory: A self-evolving framework through executable subagent accumulation and reuse, 2026

2026
[125]

Tool-making and self-evolving LLM agents in low-latency systems, 2026

Kalle Kujanpää, Ning Liu, Shahnawaz Alam, Yeshwanth Reddy Sura, Tianyu Yang, Kristina Klinkner, and Shervin Malmasi. Tool-making and self-evolving LLM agents in low-latency systems, 2026

2026
[126]

Skillcomposer: Learning to evolve agent skills for specification and generalization, 2026

Qi Zhang, Zhaopeng Feng, Xiaonan Shi, Xiaomeng Hu, Chu Liu, Pengjun Xie, Xiaobin Wang, Jieping Ye, Bryan Hooi, Haobo Wang, and Junbo Zhao. Skillcomposer: Learning to evolve agent skills for specification and generalization, 2026. 23

2026
[127]

Ratchet: A minimal hygiene recipe for self-evolving LLM agents, 2026

Xing Zhang, Yanwei Cui, Guanghui Wang, Ziyuan Li, Wei Qiu, Bing Zhu, and Peiyang He. Ratchet: A minimal hygiene recipe for self-evolving LLM agents, 2026

2026
[128]

Fatemi, Xiaolong Jin, Zora Zhiruo Wang, Apurva Gandhi, Yueqi Song, Yu Gu, Jayanth Srinivasa, Gaowen Liu, Graham Neubig, and Yu Su

Boyuan Zheng, Michael Y. Fatemi, Xiaolong Jin, Zora Zhiruo Wang, Apurva Gandhi, Yueqi Song, Yu Gu, Jayanth Srinivasa, Gaowen Liu, Graham Neubig, and Yu Su. SkillWeaver: Web agents can self-improve by discovering and honing skills, 2025. URLhttps://arxiv.org/abs/2504.07079

2025 arXiv
[129]

Inducing programmatic skills for agentic tasks, 2025

Zora Zhiruo Wang, Apurva Gandhi, Graham Neubig, and Daniel Fried. Inducing programmatic skills for agentic tasks, 2025

2025
[130]

Dreamprover: Evolving transferable lemma libraries via a wake-sleep theorem-proving agent, 2026

Youyuan Zhang, Jialiang Sun, Hangrui Bi, Chuqin Geng, Wenjie Ma, Zhaoyu Li, and Xujie Si. Dreamprover: Evolving transferable lemma libraries via a wake-sleep theorem-proving agent, 2026

2026
[131]

Alita: Generalist agent enabling scalable agentic reasoning with minimal predefinition and maximal self-evolution, 2025

Jiahao Qiu, Xuan Qi, Tongcheng Zhang, Xinzhe Juan, Jiacheng Guo, Yifu Lu, Yimin Wang, Zixin Yao, Qihan Ren, Xun Jiang, Xing Zhou, Dongrui Liu, Ling Yang, Yue Wu, Kaixuan Huang, Shilong Liu, Hongru Wang, and Mengdi Wang. Alita: Generalist agent enabling scalable agentic reasoni...

2025
[132]

Memp: Exploring agent procedural memory, 2025

Runnan Fang, Yuan Liang, Xiaobin Wang, Jialong Wu, Shuofei Qiao, Pengjun Xie, Fei Huang, Huajun Chen, and Ningyu Zhang. Memp: Exploring agent procedural memory, 2025

2025
[133]

Skill-Pro: Learning reusable skills from experience via non-parametric PPO for LLM agents, 2026

Qirui Mi, Zhijian Ma, Mengyue Yang, Haoxuan Li, Yisen Wang, Haifeng Zhang, and Jun Wang. Skill-Pro: Learning reusable skills from experience via non-parametric PPO for LLM agents, 2026

2026
[134]

Gonzalez, and Bin Cui

Ling Yang, Zhaochen Yu, Tianjun Zhang, Shiyi Cao, Minkai Xu, Wentao Zhang, Joseph E. Gonzalez, and Bin Cui. Buffer of thoughts: Thought-augmented reasoning with large language models, 2024

2024
[135]

AutoRefine: From trajectories to reusable expertise for continual LLM agent refinement, 2026

Libin Qiu, Zhirong Gao, Junfu Chen, Yuhang Ye, Weizhi Huang, Xiaobo Xue, Wenkai Qiu, and Shuo Tang. AutoRefine: From trajectories to reusable expertise for continual LLM agent refinement, 2026

2026
[136]

Mem2evolve: Towards self-evolving agents via co-evolutionary capability expansion and experience distillation, 2026

Zihao Cheng, Zeming Liu, Yingyu Shan, Xinyi Wang, Xiangrong Zhu, Yunpu Ma, Hongru Wang, Yuhang Guo, Wei Lin, and Yunhong Wang. Mem2evolve: Towards self-evolving agents via co-evolutionary capability expansion and experience distillation, 2026

2026
[137]

Skillforge: Forging domain-specific, self-evolving agent skills in cloud technical support, 2026

Xingyan Liu, Xiyue Luo, Linyu Li, Ganghong Huang, Jianfeng Liu, and Honglin Qiao. Skillforge: Forging domain-specific, self-evolving agent skills in cloud technical support, 2026

2026
[138]

Muse-autoskill: Self-evolving agents via skill creation, memory, management, and evaluation, 2026

Huawei Lin, Peng Li, Jie Song, Fuxin Jiang, and Tieying Zhang. Muse-autoskill: Self-evolving agents via skill creation, memory, management, and evaluation, 2026

2026
[139]

Mem- skill: Learning and evolving memory skills for self-evolving agents, 2026

Haozhen Zhang, Quanyu Long, Jianzhu Bao, Tao Feng, Weizhi Zhang, Haodong Yue, and Wenya Wang. Mem- skill: Learning and evolving memory skills for self-evolving agents, 2026

2026
[140]

Agent0: Unleashing self-evolving agents from zero data via tool-integrated reasoning, 2025

Peng Xia, Kaide Zeng, Jiaqi Liu, Can Qin, Fang Wu, Yiyang Zhou, Caiming Xiong, and Huaxiu Yao. Agent0: Unleashing self-evolving agents from zero data via tool-integrated reasoning, 2025. URLhttps://arxiv.org/ abs/2511.16043

2025
[141]

Learning while acting: A skill-enhanced test-time co-evolution framework for online lifelong learning agents, 2026

Bo Mao, Jie Zhou, Yutao Yang, Xin Li, Xian Wei, Qin Chen, Xingjiao Wu, and Liang He. Learning while acting: A skill-enhanced test-time co-evolution framework for online lifelong learning agents, 2026

2026
[142]

Federatedskill: Federated learning for agentic skill evolution, 2026

Jingbo Yang, Guanyu Yao, Yang Zhang, Ramana Rao Kompella, Gaowen Liu, and Shiyu Chang. Federatedskill: Federated learning for agentic skill evolution, 2026

2026
[143]

Guanyu Jiang, Zhaochen Su, Xiaoye Qu, and Yi R. Fung. Xskill: Continual learning from experience and skills in multimodal agents, 2026

2026
[144]

Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J

Lakshya A. Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J. Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, and Omar Khattab. GEPA: Reflec...

2025 arXiv
[145]

Optima: Optimizing effectiveness and efficiency for LLM-based multi-agent systems, 2024

Weize Chen, Jiarui Yuan, Chen Qian, Cheng Yang, Zhiyuan Liu, and Maosong Sun. Optima: Optimizing effectiveness and efficiency for LLM-based multi-agent systems, 2024

2024
[146]

SKILL0: In-context agentic reinforcement learning for skill internalization, 2026

Zhengxi Lu, Zhiyuan Yao, Jinyang Wu, Chengcheng Han, Qi Gu, Xunliang Cai, Weiming Lu, Jun Xiao, Yueting Zhuang, and Yongliang Shen. SKILL0: In-context agentic reinforcement learning for skill internalization, 2026. URLhttps://arxiv.org/abs/2604.02268. 24

2026 arXiv
[147]

Agent-pro: Learning to evolve via policy-level reflection and optimization, 2024

Wenqi Zhang, Ke Tang, Hai Wu, Mengna Wang, Yongliang Shen, Guiyang Hou, Zeqi Tan, Peng Li, Yueting Zhuang, and Weiming Lu. Agent-pro: Learning to evolve via policy-level reflection and optimization, 2024

2024
[148]

Symbolic learning enables self-evolving agents, 2024

Wangchunshu Zhou, Yixin Ou, Shengwei Ding, Long Li, Jialong Wu, Tiannan Wang, Jiamin Chen, Shuai Wang, Xiaohua Xu, Ningyu Zhang, Huajun Chen, and Yuchen Eleanor Jiang. Symbolic learning enables self-evolving agents, 2024

2024
[149]

Autoguide: Automated generation and selection of context-aware guidelines for large language model agents, 2024

Yao Fu, Dong-Ki Kim, Jaekyeom Kim, Sungryull Sohn, Lajanugen Logeswaran, Kyunghoon Bae, and Honglak Lee. Autoguide: Automated generation and selection of context-aware guidelines for large language model agents, 2024

2024
[150]

AutoManual: Constructing instruction manuals by LLM agents via interactive environmental learning, 2024

Minghao Chen, Yihang Li, Yanting Yang, Shiyu Yu, Binbin Lin, and Xiaofei He. AutoManual: Constructing instruction manuals by LLM agents via interactive environmental learning, 2024

2024
[151]

Automated design of agentic systems, 2024

Shengran Hu, Cong Lu, and Jeff Clune. Automated design of agentic systems, 2024

2024
[152]

Aflow: Automating agentic workflow generation, 2024

Jiayi Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xionghui Chen, Jiaqi Chen, Mingchen Zhuge, Xin Cheng, Sirui Hong, Jinlin Wang, Bingnan Zheng, Bang Liu, Yuyu Luo, and Chenglin Wu. Aflow: Automating agentic workflow generation, 2024

2024
[153]

AgentSquare: Automatic LLM agent search in modular design space, 2024

Yu Shang, Yu Li, Keyu Zhao, Likai Ma, Jiahe Liu, Fengli Xu, and Yong Li. AgentSquare: Automatic LLM agent search in modular design space, 2024

2024
[154]

Pace: Two-timescale self-evolution for small language model agents, 2026

Chen Ling, Pei Chen, Albert Guan, Jiaming Qu, Shayan Ali Akbar, Madhu Gopinathan, and Erwin Cornejo. Pace: Two-timescale self-evolution for small language model agents, 2026

2026
[155]

AgentNet: Decentralized evolutionary coordination for LLM-based multi-agent systems, 2025

Yingxuan Yang, Huacan Chai, Shuai Shao, Yuanyi Song, Siyuan Qi, Renting Rui, and Weinan Zhang. AgentNet: Decentralized evolutionary coordination for LLM-based multi-agent systems, 2025

2025
[156]

Sew: Self-evolving agentic workflows for automated code generation, 2025

Siwei Liu, Jinyuan Fang, Han Zhou, Yingxu Wang, and Zaiqiao Meng. Sew: Self-evolving agentic workflows for automated code generation, 2025

2025
[157]

Sepo: Self-evolving prompt agent for system prompt optimiza- tion, 2026

Wangcheng Tao, Han Wu, and Weng-Fai Wong. Sepo: Self-evolving prompt agent for system prompt optimiza- tion, 2026

2026
[158]

Evoflow: Evolving diverse agentic workflows on the fly, 2025

Guibin Zhang, Kaijie Chen, Guancheng Wan, Heng Chang, Hong Cheng, Kun Wang, Shuyue Hu, and Lei Bai. Evoflow: Evolving diverse agentic workflows on the fly, 2025

2025
[159]

Darwin: Dynamic agentically rewriting self-improving network, 2026

Henry Jiang. Darwin: Dynamic agentically rewriting self-improving network, 2026

2026
[160]

Darwin godel machine: Open-ended evolution of self-improving agents, 2025

Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. Darwin godel machine: Open-ended evolution of self-improving agents, 2025

2025
[162]

Autogenesis: A self-evolving agent protocol, 2026

Wentao Zhang et al. Autogenesis: A self-evolving agent protocol, 2026

2026
[163]

CPPO: Continual learning for reinforcement learning with human feedback

Han Zhang, Yu Lei, Lin Gui, Min Yang, Yulan He, Hui Wang, and Ruifeng Xu. CPPO: Continual learning for reinforcement learning with human feedback. InInternational Conference on Learning Representations (ICLR), 2024. 25

2024
[165]

Denser̸=better: Limits of on-policy self-distillation for continual post-training, 2026

Meng Wang, Haohan Zhao, Wenzhuo Liu, Lu Yang, Geng Liu, Haiyang Guo, Guo-Sen Xie, Gaofeng Meng, Hongbin Liu, and Fei Zhu. Denser̸=better: Limits of on-policy self-distillation for continual post-training, 2026

2026
[166]

COPR: Continual human preference learning via optimal policy regularization

Han Zhang, Lin Gui, Yu Lei, Yuanzhao Zhai, Yehong Zhang, Zhuo Zhang, Yulan He, Hui Wang, Yue Yu, Kam-Fai Wong, Bin Liang, and Ruifeng Xu. COPR: Continual human preference learning via optimal policy regularization. InFindings of the Association for Computational Linguistics: A...

2025
[167]

Self-adapting language models

Adam Zweiger, Jyothish Pari, Han Guo, Ekin Akyürek, Yoon Kim, and Pulkit Agrawal. Self-adapting language models. InAdvances in Neural Information Processing Systems (NeurIPS), 2025

2025
[168]

AIMMerging: Adaptive iterative model merging using training trajectories for language model continual learning

Yujie Feng, Jian Li, Xiaoyu Dong, Pengfei Xu, Xiaohui Zhou, Yujia Zhang, Zexin Lu, Yasha Wang, Alan Zhao, Xu Chu, and Xiao-Ming Wu. AIMMerging: Adaptive iterative model merging using training trajectories for language model continual learning. InProceedings of the 2025 Confere...

2025
[169]

Mitigating catastrophic forgetting in language transfer via model merging

Anton Alexandrov, Veselin Raychev, Mark Niklas Müller, Ce Zhang, Martin Vechev, and Kristina Toutanova. Mitigating catastrophic forgetting in language transfer via model merging. InFindings of the Association for Computational Linguistics: EMNLP 2024, 2024

2024
[170]

Learning on the job: Test-time curricula for targeted reinforcement learning.arXiv preprint arXiv:2510.04786, 2025

Jonas Hübotter, Leander Diaz-Bone, Ido Hakimi, Andreas Krause, and Moritz Hardt. Learning on the job: Test-time curricula for targeted reinforcement learning.arXiv preprint arXiv:2510.04786, 2025

2025
[171]

ZeroFlow: Overcoming catastrophic forgetting is easier than you think

Tao Feng, Wei Li, Didi Zhu, Hangjie Yuan, Wendi Zheng, Dan Zhang, and Jie Tang. ZeroFlow: Overcoming catastrophic forgetting is easier than you think. InProceedings of the 42nd International Conference on Machine Learning, volume 267 ofProceedings of Machine Learning Research,...

2025
[172]

Merge before forget: A single LoRA continual learning via continual merging, 2025

Fuli Qiao and Mehrdad Mahdavi. Merge before forget: A single LoRA continual learning via continual merging, 2025

2025
[173]

Evolu- tionary strategies lead to catastrophic forgetting in LLMs, 2026

Immanuel Abdi, Akshat Gupta, Micah Mok, Alexander Lu, Nicholas Lee, and Gopala Anumanchipalli. Evolu- tionary strategies lead to catastrophic forgetting in LLMs, 2026

2026
[174]

Geometry conflict: Explaining and controlling forgetting in LLM continual post-training, 2026

Yuanyi Wang, Yifan Yang, Su Lu, Yanggan Gu, Pengkai Wang, Wenjun Wang, Zhaoyi Yan, Congkai Xie, Jianmin Wu, Jialun Cao, Shing-Chi Cheung, and Hongxia Yang. Geometry conflict: Explaining and controlling forgetting in LLM continual post-training, 2026

2026
[175]

In- vestigating continual pretraining in large language models: Insights and implications, 2024

Çağatay Yıldız, Nishaanth Kanna Ravichandran, Prishruit Punia, Matthias Bethge, and Beyza Ermis. In- vestigating continual pretraining in large language models: Insights and implications, 2024. URLhttps: //arxiv.org/abs/2402.17400

2024 arXiv
[176]

An empirical study of catastrophic forgetting in large language models during continual fine-tuning.ArXiv, abs/2308.08747, 2023

Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yuechen Zhang. An empirical study of catastrophic forgetting in large language models during continual fine-tuning.ArXiv, abs/2308.08747, 2023. URLhttps: //api.semanticscholar.org/CorpusID:261031244

2023 arXiv
[177]

TRACE: A comprehensive benchmark for continual learning in large language models, 2023

Xiao Wang, Yuansen Zhang, Tianze Chen, Songyang Gao, Senjie Jin, Xianjun Yang, Zhiheng Xi, Rui Zheng, Yicheng Zou, Tao Gui, Qi Zhang, and Xuanjing Huang. TRACE: A comprehensive benchmark for continual learning in large language models, 2023. URLhttps://arxiv.org/abs/2310.06762

2023 arXiv
[178]

Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. Don’t stop pretraining: Adapt language models to domains and tasks. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020

2020
[179]

ReCALL: Recalibrating capability degradation for MLLM-based composed image retrieval,

Tianyu Yang, Chenwei He, Xiangzhao Hao, Tianyue Wang, Jiarui Guo, Haiyun Guo, Leigang Qu, Jinqiao Wang, and Tat-Seng Chua. ReCALL: Recalibrating capability degradation for MLLM-based composed image retrieval,
[180]

Continual GUI agents

Ziwei Liu, Borui Kang, Hangjie Yuan, Zixiang Zhao, Wei Li, Yifan Zhu, and Tao Feng. Continual GUI agents. InProceedings of the 43rd International Conference on Machine Learning, 2026. 26

2026
[181]

LoRA: Low-rank adaptation of large language models

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. InInternational Conference on Learning Repre- sentations, 2022

2022
[182]

Spurious forgetting in continual learning of language models, 2025

Junhao Zheng, Xidi Cai, Shengjie Qiu, and Qianli Ma. Spurious forgetting in continual learning of language models, 2025. URLhttps://arxiv.org/abs/2501.13453

2025 arXiv
[183]

Editing models with task arithmetic.arXiv preprint arXiv:2212.04089, 2022

Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic.arXiv preprint arXiv:2212.04089, 2022

2022 arXiv
[184]

Accepted at CVPR 2026

URLhttps://arxiv.org/abs/2602.01639. Accepted at CVPR 2026

2026
[185]

Evolutionary optimization of model merging recipes.arXiv preprint arXiv:2403.13187, 2024

Takuya Akiba, Makoto Shing, Yujin Tang, Qi Sun, and David Ha. Evolutionary optimization of model merging recipes.arXiv preprint arXiv:2403.13187, 2024

2024 arXiv
[186]

Language models are super mario: Absorbing abilities from homologous models as a free lunch

Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. Language models are super mario: Absorbing abilities from homologous models as a free lunch. InProceedings of the 41st International Conference on Machine Learning, 2024. URLhttps://arxiv.org/abs/2311.03099

2024 arXiv
[187]

Harness handbook: Making evolving agent harnesses readable, navigable, and editable

Ruhan Wang, Yucheng Shi, Zongxia Li, Zhongzhi Li, Yue Yu, Junyao Yang, Kishan Panaganti, Haitao Mi, Dongruo Zhou, et al. Harness handbook: Making evolving agent harnesses readable, navigable, and editable. arXiv preprint arXiv:2607.13285, 2026

2026 arXiv
[188]

Why I do not think AGI is right around the corner.https://www.dwarkesh.com/p/ timelines-june-2025, 2025

Dwarkesh Patel. Why I do not think AGI is right around the corner.https://www.dwarkesh.com/p/ timelines-june-2025, 2025. URLhttps://www.dwarkesh.com/p/timelines-june-2025. Essay, Dwarkesh Podcast; contextual commentary rather than a primary technical source

2025
[189]

TIES-merging: Resolving interference when merging models

Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. TIES-merging: Resolving interference when merging models. InThirty-seventh Conference on Neural Information Processing Systems,
[190]

URLhttps://openreview.net/forum?id=xtaX3WyCj1
[191]

Contra dwarkesh on continual learning, 2025

Nathan Lambert. Contra dwarkesh on continual learning, 2025. URLhttps://www.interconnects.ai/p/ contra-dwarkesh-on-continual-learning. Essay, Interconnects; contextual commentary rather than a pri- mary technical source

2025
[192]

Revisiting replay and gradient alignment for continual pre-training of large language models

Istabrak Abbes, Gopeshh Subbaraj, Matthew Riemer, Nizar Islah, Tsuguchika Tabaru, Hiroaki Kingetsu, Sarath Chandar, and Irina Rish. Revisiting replay and gradient alignment for continual pre-training of large language models. InProceedings of The 4th Conference on Lifelong Lea...

2026
[193]

Twin-merging: Dynamic integration of modular expertise in model merging

Zhenyi Lu, Chenghao Fan, Wei Wei, Xiaoye Qu, Dangyang Chen, and Yu Cheng. Twin-merging: Dynamic integration of modular expertise in model merging. InAdvances in Neural Information Processing Systems,
[194]

Agentic memory: Learning unified long-term and short-term memory management for large language model agents

Yi Yu, Liuyi Yao, Yuexiang Xie, Qingquan Tan, Jiaqi Feng, Yaliang Li, and Libing Wu. Agentic memory: Learning unified long-term and short-term memory management for large language model agents. InProceedings of the 64th Annual Meeting of the Association for Computational Lingu...

2026 doi
[195]

From player to master: Enhancing test-time learning of LLM agents via reinforcement learning over memory, 2026

Yishuo Cai, Xingyu Guo, Xuancheng Huang, Jinhua Du, Can Huang, Wenxuan Huang, Wenhan Ma, Yuyang Hu, Aohan Zeng, Jie Tang, and Xu Sun. From player to master: Enhancing test-time learning of LLM agents via reinforcement learning over memory, 2026. URLhttps://arxiv.org/abs/2606.0...

2026 arXiv
[196]

From storage to experience: A survey on the evolution of LLM agent memory mechanisms, 2026

Jinghao Luo, Yuchen Tian, Chuxue Cao, Ziyang Luo, Hongzhan Lin, Kaixin Li, Chuyi Kong, Ruichao Yang, and Jing Ma. From storage to experience: A survey on the evolution of LLM agent memory mechanisms, 2026. Findings of ACL 2026

2026
[197]

The future of continual learning in the era of foundation models: Three key directions,

Jack Bell, Luigi Quarantiello, Eric Nuertey Coleman, Lanpei Li, Malio Li, Mauro Madeddu, Elia Piccoli, and Vincenzo Lomonaco. The future of continual learning in the era of foundation models: Three key directions,
[198]

arXiv:2506.03320; accepted at the TCAI workshop, 2025

2025 arXiv
[199]

TemporalWiki: A lifelong benchmark for training and evaluating ever-evolving language models

Joel Jang, Seonghyeon Ye, Changho Lee, Sohee Yang, Joongbo Shin, Janghoon Han, Gyeonghun Kim, and Minjoon Seo. TemporalWiki: A lifelong benchmark for training and evaluating ever-evolving language models. InProceedings of the 2022 Conference on Empirical Methods in Natural Lan...

2022
[200]

CITB: A benchmark for continual instruction tuning

Zihan Zhang, Meng Fang, Ling Chen, and Mohammad-Reza Namazi-Rad. CITB: A benchmark for continual instruction tuning. InFindings of the Association for Computational Linguistics: EMNLP 2023, pages 9443–

2023
[201]

Self-updatable large language models by integrating context into model parameters, 2024

Yu Wang, Xinshuang Liu, Xiusi Chen, Sean O’Brien, Junda Wu, and Julian McAuley. Self-updatable large language models by integrating context into model parameters, 2024. URLhttps://arxiv.org/abs/2410. 00487

2024
[202]

MLLM-CL: Continual learning for multimodal large language models, 2025

Hongbo Zhao, Fei Zhu, Haiyang Guo, Meng Wang, Rundong Wang, Gaofeng Meng, and Zhaoxiang Zhang. MLLM-CL: Continual learning for multimodal large language models, 2025. URLhttps://arxiv.org/abs/ 2506.05453

2025
[203]

MLLM-CTBench: A benchmark for continual instruction tuning with reasoning process diagnosis,

Haiyun Guo, Zhiyan Hou, Yandu Sun, Jinghan He, Yu Chen, Yuzhe Zhou, Yuheng Jia, Jinqiao Wang, and Tat- Seng Chua. MLLM-CTBench: A benchmark for continual instruction tuning with reasoning process diagnosis,
[204]

Agentic 27 context engineering: Evolving contexts for self-improving language models

Qizheng Zhang, Changran Hu, Shubhangi Upasani, Boyuan Ma, Fenglu Hong, Vamsidhar Kamanuru, Jay Rainton, Chen Wu, Mengmeng Ji, Hanchen Li, Urmish Thakker, James Zou, and Kunle Olukotun. Agentic 27 context engineering: Evolving contexts for self-improving language models. InThe ...

2026
[205]

Dynamic cheatsheet: Test- time learning with adaptive memory, 2025

Mirac Suzgun, Mert Yuksekgonul, Federico Bianchi, Dan Jurafsky, and James Zou. Dynamic cheatsheet: Test- time learning with adaptive memory, 2025. URLhttps://arxiv.org/abs/2504.07952

2025 arXiv
[206]

TiC-LM: A web-scale benchmark for time-continual LLM pretraining

Jeffrey Li, Mohammadreza Armandpour, Seyed Iman Mirzadeh, Sachin Mehta, Vaishaal Shankar, Raviteja Vemulapalli, Samy Bengio, Oncel Tuzel, Mehrdad Farajtabar, Hadi Pouransari, and Fartash Faghri. TiC-LM: A web-scale benchmark for time-continual LLM pretraining. InProceedings of...

2025 doi
[207]

Evomembench: Benchmarking agent memory from a self-evolving perspective, 2026

Yuyao Wang, Zhongjian Zhang, Mo Chi, Kaichi Yu, Yuhan Li, Miao Peng, Bing Tong, Chen Zhang, Yan Zhou, and Jia Li. Evomembench: Benchmarking agent memory from a self-evolving perspective, 2026

2026
[208]

Shanshan Zhong, Yi Lu, Jingjie Ning, Yibing Wan, Lihan Feng, Yuyi Ao, Leonardo F. R. Ribeiro, Markus Dreyer, Sean Ammirati, and Chenyan Xiong. Skilllearnbench: Benchmarking continual learning methods for agent skill generation on real-world tasks, 2026

2026
[209]

Skillflow: Bench- marking lifelong skill discovery and evolution for autonomous agents, 2026

Ziao Zhang, Kou Shi, Shiting Huang, Avery Nie, Yu Zeng, Yiming Zhao, Zhen Fang, Qishen Su, Haibo Qiu, Wei Yang, Qingnan Ren, Shun Zou, Wenxuan Huang, Lin Chen, Zehui Chen, and Feng Zhao. Skillflow: Bench- marking lifelong skill discovery and evolution for autonomous agents, 2026

2026
[210]

SEAGym: An evaluation environ- ment for self-evolving LLM agents, 2026

Congjie Zheng, Chuanyi Xue, Bin Liang, Jun Yang, and Changshui Zhang. SEAGym: An evaluation environ- ment for self-evolving LLM agents, 2026. 28 A Glossary of Abbreviations For readability, Table A.1 summarizes the abbreviations used throughout this survey. Table A.1Abbreviati...

2026
[211]

CoIN: A benchmark of continual instruction tuning for multimodel large language models

Cheng Chen, Junchen Zhu, Xu Luo, Heng Tao Shen, Jingkuan Song, and Lianli Gao. CoIN: A benchmark of continual instruction tuning for multimodel large language models. InAdvances in Neural Information Processing Systems, volume 37, pages 57817–57840, 2024. doi: 10.52202/079017-...

2024 doi
[214]

URLhttps://arxiv.org/abs/2508.08275
[215]

Continual test-time domain adaptation

Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai. Continual test-time domain adaptation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022. URLhttps://arxiv.org/ abs/2203.13591

2022 arXiv
[216]

Can large language models keep up? benchmarking online adaptation to continual knowledge streams

Jiyeon Kim, Hyunji Lee, Dylan Zhou, Sue Hyun Park, Seunghyun Yoon, Trung Bui, Franck Dernoncourt, Sungmin Cha, and Minjoon Seo. Can large language models keep up? benchmarking online adaptation to continual knowledge streams. InProceedings of the 64th Annual Meeting of the Ass...

2026 doi
[217]

MemoryBench: A benchmark for memory and continual learning in LLM systems, 2025

Qingyao Ai, Yichen Tang, Changyue Wang, Jianming Long, Weihang Su, and Yiqun Liu. MemoryBench: A benchmark for memory and continual learning in LLM systems, 2025

2025
[2023]

URLhttps://arxiv.org/abs/2308.04014

arXiv
[2024]

URLhttps://arxiv.org/abs/2406.15479

arXiv
[2025]

URLhttps://arxiv.org/abs/2509.04259

arXiv
[2026]

URLhttps://arxiv.org/abs/2601.19897

arXiv
[6250]

doi: 10.18653/v1/2022.emnlp-main.418

Association for Computational Linguistics, 2022. doi: 10.18653/v1/2022.emnlp-main.418. URLhttps: //aclanthology.org/2022.emnlp-main.418/

2022 doi
[9455]

doi: 10.18653/v1/2023.findings-emnlp.633

Association for Computational Linguistics, 2023. doi: 10.18653/v1/2023.findings-emnlp.633. URLhttps: //aclanthology.org/2023.findings-emnlp.633/

2023 doi

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [40]

Papka, and Venkatram Vishwanath

Aditya Tanikanti, Benoit Côté, Yanfei Guo, Le Chen, Nickolaus Saint, Ryan Chard, Ken Raffenetti, Rajeev Thakur, Thomas Uram, Ian Foster, Michael E. Papka, and Venkatram Vishwanath. FIRST: Federated inference resource scheduling toolkit for scientific ai model access. InProceedings of the SC ’25 Workshops of the Inter- national Conference for High Performa...

[2] [27]

RL’s razor: Why online reinforcement learning forgets less,

Idan Shenfeld, Jyothish Pari, and Pulkit Agrawal. RL’s razor: Why online reinforcement learning forgets less,

[3] [164]

TTRL: Test-time reinforcement learning, 2025

Yuxin Zuo, Kaiyan Zhang, Li Sheng, Shang Qu, Ganqu Cui, Xuekai Zhu, Haozhan Li, Yuchen Zhang, Xinwei Long, Ermo Hua, Biqing Qi, Youbang Sun, Zhiyuan Ma, Lifan Yuan, Ning Ding, and Bowen Zhou. TTRL: Test-time reinforcement learning, 2025. URLhttps://arxiv.org/abs/2504.16084

arXiv 2025

[4] [161]

Self-distillation bridges distribution gap in language model fine-tuning

Zhaorui Yang, Qian Liu, Tianyu Pang, Han Wang, Haozhe Feng, Minfeng Zhu, and Wei Chen. Self-distillation bridges distribution gap in language model fine-tuning. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024. URLhttps://arxiv.org/abs/2402.13669

arXiv 2024

[5] [29]

Self-distillation enables continual learning,

Idan Shenfeld, Mehul Damani, Jonas Hübotter, and Pulkit Agrawal. Self-distillation enables continual learning,

[6] [1]

GPT-4 technical report, 2023

OpenAI. GPT-4 technical report, 2023. URLhttps://arxiv.org/abs/2303.08774

arXiv 2023

[7] [2]

ChatGLM: A family of large language models from GLM-130B to GLM-4 all tools, 2024

GLM Team. ChatGLM: A family of large language models from GLM-130B to GLM-4 all tools, 2024. URL https://arxiv.org/abs/2406.12793

arXiv 2024

[8] [3]

Qwen technical report, 2023

Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report, 2023. URLhttps://arxiv.org/abs/2309.16609

arXiv 2023