Pith. sign in

REVIEW 2 major objections 3 minor 2 cited by

Continual Learning for Generative AI: From LLMs to MLLMs and Beyond

T0 review · 2 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This survey organizes continual learning for generative AI into three paradigms—architecture-based, regularization-based, and replay-based—spanning LLMs, MLLMs, vision-language-action models, and diffusion models.

desk verdict Useful first broad survey of continual learning for generative models, but its own method tables undercut the comprehensiveness claim and need fixing before this becomes a reference-quality resource. read the letter →

arxiv 2506.13045 v4 pith:EUITVQY3 submitted 2025-06-16 cs.LG cs.CV

Haiyang Guo , Fanhu Zeng , Fei Zhu , Jiayi Wang , Xukai Wang , Jingang Zhou , Hongbo Zhao , Wenzhuo Liu
show 4 more authors
Shijie Ma Da-Han Wang Xu-Yao Zhang Cheng-Lin Liu
This is my paper
classification cs.LGcs.CV
keywords ContinualLearningGenerativeModelsLargeLanguageMultimodalVision-Language-ActionDiffusionCatastrophicForgetting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that catastrophic forgetting in generative AI is a coherent, surveyable problem and that the many proposed fixes can be mapped onto one shared taxonomy. It reviews methods for four model families—large language models, multimodal large language models, vision-language-action models, and diffusion models—and classifies each into architecture-based, regularization-based, or replay-based approaches, mirroring human brain memory mechanisms. If this map is correct, researchers gain a common language for comparing methods, benchmarks, and open problems across families that are usually treated separately. The payoff would be faster transfer of continual learning ideas between domains and a clearer view of what remains unsolved.

What carries the argument

The organizing device is a three-paradigm taxonomy borrowed from complementary learning systems in the brain. Architecture-based methods mimic modular organization by freezing old parameters and adding task-specific modules, typically LoRA adapters or prompts; regularization-based methods mimic synaptic consolidation by penalizing changes to important parameters or features; replay-based methods mimic hippocampal replay by storing raw data, features, or generated pseudo-samples from previous tasks. The survey applies this taxonomy consistently across the four model families, using each family's training objective to define what forgetting means in that setting and its benchmarks (e.g., TRACE, CoIN, LIBERO, CLoG) to make the map concrete.

What would settle it

Have independent annotators reassign every method listed in the survey's tables to one of the three paradigms: if a large share of methods cannot be assigned a primary category with high agreement, or if a coherent cluster of continual learning methods for generative models (for example, audio or video generation) falls outside all three categories, the claimed taxonomy would fail as an organizing scheme.

Watch

Extended reading notes

Core claim

The central claim is that, despite large differences in objectives, benchmarks, and backbones, continual learning methods for mainstream generative models share a small set of underlying strategies, and a three-paradigm taxonomy captures the field. The survey formalizes each family's training setup—autoregressive next-token prediction for LLMs and MLLMs, behavioral cloning for vision-language-action models, and denoising losses for diffusion models—then assigns the existing literature to architecture-based methods that isolate task knowledge in new modules, regularization-based methods that constrain important parameters or features, and replay-based methods that store or regenerate past data. The paper positions itself as the first comprehensive review spanning all four families, with the implication that continual learning techniques developed for one family may transfer to others.

Load-bearing premise

The taxonomy assumes every surveyed method can be assigned a primary category among architecture-based, regularization-based, or replay-based, even though the paper itself notes that methods often combine several modules; if the categories overlap too heavily or miss important dimensions, the survey's organization could misrepresent the literature.

Editorial extensions

If this is right

  • A method developed for one generative family can be placed in the same conceptual slot as a method in another family, making cross-family transfer of continual learning ideas a natural next step.
  • The observed prevalence of architecture-based and replay-based methods in LLMs and MLLMs, versus regularization in diffusion models, indicates where each strategy is currently most mature.
  • The survey's unified evaluation metrics—Last, Average Accuracy, Forgetting Measure, Backward Transfer, and Zero-shot Transfer—give researchers a common yardstick for comparing continual learning methods across generative models.
  • The identified open problems (efficient mechanisms, RL-based learning, self-generation, scale, new modalities, unified optimization) constitute a concrete research agenda that follows directly from the survey's map.
  • The paper's own discussions suggest that regularization methods remain underused in LLMs because of tuning and stability difficulties, pointing to a specific gap where future work could concentrate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the taxonomy holds, replay methods validated in LLM continual instruction tuning should transfer to multimodal large language models with relatively small changes, since the paper notes that existing MLLM continual learning methods are not yet substantially different from language-only LLM methods.
  • The paper's emphasis on self-generation suggests a testable extension: use a generative model's own outputs as replay data and compare forgetting, data efficiency, and distributional fidelity against stored raw data in a controlled benchmark.
  • The survey's observation that vision-language-action action policies tend to forget faster than vision-language alignment suggests a concrete experiment: measure forgetting separately for action heads versus perception modules when a new task arrives.
  • Because the taxonomy assigns each surveyed method one primary paradigm while acknowledging multi-module designs, a quantitative study of how many methods genuinely span paradigms would test whether the map is a useful simplification or an oversimplification.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. This manuscript surveys continual learning methods for four families of generative models—large language models (LLMs), multimodal large language models (MLLMs), vision-language-action (VLA) models, and diffusion models—and organizes the methods into three paradigms derived from complementary learning systems theory: architecture-based, regularization-based, and replay-based. For each family, the paper gives the problem setup (training objectives, backbones, benchmarks), describes representative methods in the text, and summarizes them in a taxonomy table (Tables 1–3). The paper claims to be the first comprehensive review covering these four generative model families together and to provide a unified three-paradigm map. It also includes a GitHub repository for tracking ongoing work.

Significance. If the survey's taxonomy and tables are reliable, the paper would provide a useful structured entry point to a rapidly growing and fragmented literature, connecting continual learning research across language, multimodal, robotic, and generative-image domains. The mathematical formulations of the continual learning setups (Eqs. 1–18) are standard and appear correctly presented. The accompanying GitHub repository and the explicit discussion of open problems are practical strengths. However, the survey's central value depends on the accuracy and readability of the summary tables, which currently contain substantive errors. These issues are fixable, but they must be addressed before the paper can serve as a trustworthy reference.

major comments (2)
  1. [Table 3 (Section 5.2)] Table 3 misclassifies LIBERO as a continual learning method and is missing several methods that the text discusses. The row "LIBERO [126]" carries three taxonomy checkmarks, yet Section 5.1 identifies LIBERO as an imitation-learning benchmark, and the same table's "Benchmark / Dataset" column lists LIBERO as a dataset for other rows. Conversely, IsCiL [113], RLCM [149], PolyTask [72], ELIRL [148], and Decision-RWKV [47], all of which are described in Sections 5.2–5.4, do not appear in Table 3. As a result, the table does not faithfully represent the VLA continual learning literature, which directly undermines the paper's claim of a comprehensive and systematically organized review.
  2. [Tables 1–3 captions] The taxonomy legend in Tables 1–3 is unreadable: the caption states "✔ indicates the primary solution, and ✔ indicates the supporting method," using the same glyph for both categories. Consequently, the primary/supporting distinction that is central to the taxonomic organization cannot be recovered from the tables, and the categorization is not verifiable. Please use distinct symbols (or textual labels) and ensure they render consistently in the final version.
minor comments (3)
  1. [Eq. (2)] The definition of Avg_t as the average of Last_j is non-standard and seems inconsistent with the usual average-accuracy metric; please clarify whether Avg_t is intended to be the average accuracy over all seen tasks at time t, and if so, simplify the definition.
  2. [Section 3.3] The text refers to "NaCL [214]" but Table 1 and the reference list identify the same work as "DaCL [214]"; please reconcile the naming.
  3. [Section 4.2.2] MR-LoRA [256] is described twice with different mechanisms in the same paragraph; please merge the two descriptions or clearly distinguish them to avoid duplication and confusion.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the survey organizes existing literature and derives no predictions that reduce to its inputs.

full rationale

The paper is a survey: its central artifacts are a three-paradigm taxonomy, per-domain summaries, and method tables, not derived predictions. The equations reproduced (Eqs. 1-18) are standard continual-learning objectives, behavioral-cloning losses, and diffusion losses taken from the literature; no parameter is fitted to one subset of data and then reported as a prediction of a closely related quantity. The taxonomy is explicitly acknowledged in Section 2.3 as a primary-category assignment for methods that often combine multiple modules, so the classification is presented as an analytical choice rather than a forced consequence. Self-citations such as HiDe-LLaVA [64], DISCO [65], LLaVA-c [132], ModalPrompt [240], and MLLM-CL [256] appear as ordinary literature entries in the survey tables and corresponding sections; they are not invoked as load-bearing justifications for the survey's scope or organization, and they are independently published works with their own stated setups. The 'first comprehensive review' claim is a scope assertion, not a derivation. The skeptic's concerns about Table 3 (LIBERO listed as a method, some discussed methods absent, ambiguous legend glyphs) are accuracy and completeness issues for a survey artifact, not circularity, because the tables do not function as premises from which the paper's conclusions are deduced. No circular step can be exhibited with a specific reduction, so the honest finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The central claim is a survey claim, so it rests on the accuracy of the cited literature and the adequacy of the taxonomy. No free parameters are fitted, and no new entities are introduced.

assumptions (2)
  • domain assumption The cited papers are correctly and faithfully represented in the survey's summaries and classifications.
    The survey's descriptions of each method rest on the accuracy of the authors' reading of the primary literature; any misreading would propagate through the review.
  • domain assumption The three-category taxonomy (architecture, regularization, replay) is exhaustive and each method can be assigned a primary category.
    Section 2.3 assigns every method to one paradigm, but the authors note that methods often combine multiple strategies. The survey's organization depends on this assumption being reasonable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Continual Learning for Generative AI: From LLMs to MLLMs and Beyond." pith.science (2026). https://pith.science/paper/EUITVQY3

@misc{pith2026250613045,
  author       = {Pith},
  title        = {Pith review of: Continual Learning for Generative AI: From LLMs to MLLMs and Beyond},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EUITVQY3}},
  note         = {Machine review of arXiv:2506.13045}
}
read the original abstract

The rapid advancement of generative models has empowered modern AI systems to comprehend and produce highly sophisticated content, even achieving human-level performance in specific domains. However, these models are fundamentally constrained by \emph{catastrophic forgetting}, \ie~a persistent challenge where models experience performance degradation on previously learned tasks when adapting to new tasks. To address this practical limitation, numerous approaches have been proposed to enhance the adaptability and scalability of generative AI in real-world applications. In this work, we present a comprehensive survey of continual learning methods for mainstream generative AI models, encompassing large language models, multimodal large language models, vision-language-action models, and diffusion models. Drawing inspiration from the memory mechanisms of the human brain, we systematically categorize these approaches into three paradigms: architecture-based, regularization-based, and replay-based methods, while elucidating their underlying methodologies and motivations. We further analyze continual learning setups for different generative models, including training objectives, benchmarks, and core backbones, thereby providing deeper insights into the field. The project page of this paper is available at https://github.com/Ghy0501/Awesome-Continual-Learning-in-Generative-Models.

Figures

Figures reproduced from arXiv: 2506.13045 by the authors.

Figure 1
Figure 1. (a) Continual learning lets humans learn lifelong without forgetting; AI models aim to do the same while retaining old [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Illustrations of continual learning for generative AI models: Adapting to real-world demands by learning new [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Taxonomy of Continual Learning for Generative AI. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Overview of continual learning approaches in Large Language Models. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Three types of continual learning approaches in Multimodal Large Language Models. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Illustration of a continual learning framework applied to Vision-Language-Action Models. [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: Illustration of a continual learning framework applied to Diffusion Models. [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Agentic Software Engineering: Foundational Pillars and a Research Roadmap

    cs.SE 2025-09 conditional novelty 5.0 of 10

    SASE introduces a dual-modality framework (SE for Humans and SE for Agents) with workbenches and structured artifacts to turn agentic coding into a disciplined engineering practice.

  2. Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Training

    cs.LG 2025-07 conditional novelty 5.0 of 10

    Reinforcement fine-tuning largely prevents catastrophic forgetting during continual post-training of a multimodal LLM, while supervised fine-tuning degrades both task and general performance.

Reference graph

Works this paper leans on

278 extracted references · 14 canonical work pages · cited by 2 Pith papers

  1. [126]

    Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. 2023. Libero: Benchmarking knowledge transfer for lifelong robot learning. In NeurIPS, V ol. 36. 44776–44791

    2023

  2. [113]

    Daehee Lee, Minjong Yoo, Woo Kyung Kim, Wonje Choi, and Honguk Woo. 2024. Incremental learning of retrievable skills for efficient continual task adaptation. In NeurIPS, V ol. 37. 17286–17312

    2024

  3. [149]

    Michael McCloskey and Neal J Cohen. 1989. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation. V ol. 24. Elsevier, 109–165

    1989

  4. [72]

    Siddhant Haldar and Lerrel Pinto. 2023. Polytask: Learning unified policies through behavior distillation. arXiv preprint arXiv:2310.08573 (2023)

    arXiv 2023

  5. [148]

    Jorge Mendez, Shashank Shivkumar, and Eric Eaton. 2018. Lifelong inverse reinforcement learning. In NeurIPS, V ol. 31

    2018

  6. [47]

    Yujian Dong, Tianyu Wu, and Chaoyang Song. 2025. Optimizing Robotic Manipulation with Decision-RWKV: A Recurrent Sequence Modeling Approach for Lifelong Learning. JCISE 25, 3 (2025)

    2025

  7. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    arXiv 2023

  8. [2]

    Yongqi An, Xu Zhao, Tao Yu, Ming Tang, and Jinqiao Wang. 2024. Fluctuation-based adaptive structured pruning for large language models. In AAAI, V ol. 38. 10865–10873

    2024

Show all 278 references
  1. [3]

    Nguyen Hoang Anh, Quyen Tran, Thanh Xuan Nguyen, Nguyen Thi Ngoc Diep, Linh Ngo Van, Thien Huu Nguyen, and Trung Le. 2025. Mutual-pairing Data Augmentation for Fewshot Continual Relation Extraction. In NAACL. 4057–4075

    2025

  2. [4]

    Armen C Arevian, Vikrant Kapoor, and Nathaniel N Urban. 2008. Activity-dependent gating of lateral inhibition in the mouse olfactory bulb. Nature neuroscience 11, 1 (2008), 80–87

    2008

  3. [5]

    Yoshinori Aso and Gerald M Rubin. 2016. Dopaminergic neurons write and update memories with cell-type-specific rules. Elife 5 (2016), e16135

    2016

  4. [6]

    Yoshinori Aso, Divya Sitaraman, Toshiharu Ichinose, Karla R Kaun, Katrin V ogt, Ghislain Belliart-Guérin, Pierre-Yves Plaçais, Alice A Robie, Nobuhiro Yamagata, Christopher Schnaitmann, et al. 2014. Mushroom body output neurons encode valence and guide memory-based action sele...

    2014

  5. [7]

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-VL: A Frontier Large Vision-Language Model with Versatile Abilities. arXiv preprint arXiv:2308.12966 (2023)

    2023 arXiv

  6. [8]

    Long Bai, Mobarakol Islam, and Hongliang Ren. 2023. Revisiting distillation for continual learning on visual question localized-answering in robotic surgery. In International Conference on Medical Image Computing and Computer-Assisted Intervention . Springer, 68–78

    2023

  7. [9]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923 (2025)

    2025 arXiv

  8. [10]

    Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang, and Mike Zheng Shou. 2024. Hallucination of multimodal large language models: A survey. arXiv preprint arXiv:2404.18930 (2024)

    2024 arXiv

  9. [11]

    Michael Bain and Claude Sammut. 1995. A Framework for Behavioural Cloning.. In Machine intelligence 15. 103–129

    1995

  10. [12]

    Jihwan Bang, Heesu Kim, YoungJoon Yoo, Jung-Woo Ha, and Jonghyun Choi. 2021. Rainbow memory: Continual learning with a memory of diverse samples. In CVPR. 8218–8227

    2021

  11. [13]

    Leonard Bärmann, Rainer Kartmann, Fabian Peller-Konrad, Jan Niehues, Alex Waibel, and Tamim Asfour. 2024. Incremental learning of humanoid robot behavior from natural interaction and large language models. Frontiers in Robotics and AI 11 (2024), 1455375

    2024

  12. [14]

    Eseoghene Ben-Iwhiwhu, Saptarshi Nath, Praveen K Pilly, Soheil Kolouri, and Andrea Soltoggio. 2022. Lifelong reinforcement learning with modulating masks. arXiv preprint arXiv:2212.11110 (2022)

    2022 arXiv

  13. [15]

    Gagan Bhatia, El Moatez Billah Nagoudi, Hasan Cavusoglu, and Muhammad Abdul-Mageed. 2024. Fintral: A family of gpt-4 level multimodal financial large language models. arXiv preprint arXiv:2402.10986 (2024)

    2024 arXiv

  14. [16]

    Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al . 2024. Deepseek llm: Scaling open-source language models with longtermism. arXiv preprint arXiv:2401.02954 (2024)

    2024 arXiv

  15. [17]

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. 2023. Rt-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818 (2023)

    2023 arXiv

  16. [18]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. In NeurIPS, V ol. 33. 1877–1901

    2020

  17. [19]

    Chen Cai, Zheng Wang, Jianjun Gao, Wenyang Liu, Ye Lu, Runzhong Zhang, and Kim-Hui Yap. 2024. Empowering Large Language Model for Continual Video Question Answering with Collaborative Prompting. arXiv preprint arXiv:2410.00771 (2024)

    2024 arXiv

  18. [20]

    Yuliang Cai and Mohammad Rostami. 2024. CluMo: Cluster-based Modality Fusion Prompt for Continual Learning in Visual Question Answering. arXiv preprint arXiv:2408.11742 (2024)

    2024 arXiv

  19. [21]

    Yuliang Cai, Jesse Thomason, and Mohammad Rostami. 2023. Task-attentive transformer architecture for continual learning of vision-and-language tasks using knowledge distillation. arXiv preprint arXiv:2303.14423 (2023)

    2023 arXiv

  20. [22]

    Meng Cao, Yuyang Liu, Yingfei Liu, Tiancai Wang, Jiahua Dong, Henghui Ding, Xiangyu Zhang, Ian Reid, and Xiaodan Liang. 2024. Continual llava: Continual instruction tuning in large vision-language models. arXiv preprint arXiv:2411.02564 (2024)

    2024 arXiv

  21. [23]

    Margaret F Carr, Shantanu P Jadhav, and Loren M Frank. 2011. Hippocampal replay in the awake state: a potential substrate for memory consolidation and retrieval. Nature neuroscience 14, 2 (2011), 147–153. Manuscript submitted to ACM Continual Learning for Generative AI: From L...

    2011

  22. [24]

    Thibault Castells, Hyoung-Kyu Song, Bo-Kyeong Kim, and Shinkook Choi. 2024. Ld-pruner: Efficient pruning of latent diffusion models using task-agnostic insights. In CVPR. 821–830

    2024

  23. [25]

    Fabio Cermelli, Massimiliano Mancini, Samuel Rota Bulo, Elisa Ricci, and Barbara Caputo. 2020. Modeling the background for incremental learning in semantic segmentation. In CVPR. 9233–9242

    2020

  24. [26]

    Hongyu Chang, Wenbo Tang, Annabella M Wulf, Thokozile Nyasulu, Madison E Wolf, Antonio Fernandez-Ruiz, and Azahara Oliva. 2025. Sleep microstructure organizes memory replay. Nature (2025), 1–9

    2025

  25. [27]

    Chen Chen, Ruizhe Li, Yuchen Hu, Yuanyuan Chen, Chengwei Qin, and Qiang Zhang. 2024. Overcoming Catastrophic Forgetting by Exemplar Selection in Task-oriented Dialogue System. arXiv preprint arXiv:2405.10992 (2024)

    2024 arXiv

  26. [28]

    Cheng Chen, Junchen Zhu, Xu Luo, Hengtao Shen, Jingkuan Song, and Lianli Gao. 2024. CoIN: A Benchmark of Continual Instruction Tuning for Multimodel Large Language Models. In NeurIPS, V ol. 37. 57817–57840

    2024

  27. [29]

    Jie Chen, Zhipeng Chen, Jiapeng Wang, Kun Zhou, Yutao Zhu, Jinhao Jiang, Yingqian Min, Wayne Xin Zhao, Zhicheng Dou, Jiaxin Mao, et al. 2024. Towards effective and efficient continual pre-training of large language models. arXiv preprint arXiv:2407.18743 (2024)

    2024 arXiv

  28. [30]

    Jinpeng Chen, Runmin Cong, Yuzhi Zhao, Hongzheng Yang, Guangneng Hu, Horace Ho Shing Ip, and Sam Kwong. 2025. SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuning. arXiv preprint arXiv:2505.02486 (2025)

    2025 arXiv

  29. [31]

    Junying Chen, Chi Gui, Ruyi Ouyang, Anningzhe Gao, Shunian Chen, Guiming Hardy Chen, Xidong Wang, Ruifei Zhang, Zhenyang Cai, Ke Ji, et al

  30. [32]

    Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny. 2023. Minigpt-v2: large language model as a unified interface for vision-language multi-task learning. arXiv preprint arXi...

    2023 arXiv

  31. [33]

    Kexin Chen, Yuyang Du, Tao You, Mobarakol Islam, Ziyu Guo, Yueming Jin, Guangyong Chen, and Pheng-Ann Heng. 2024. LLM-assisted multi-teacher continual learning for visual question answering in robotic surgery. In ICRA. IEEE, 10772–10778

    2024

  32. [34]

    Letian Chen, Sravan Jayanthi, Rohan R Paleja, Daniel Martin, Viacheslav Zakharov, and Matthew Gombolay. 2023. Fast lifelong adaptive inverse reinforcement learning from demonstrations. In CoRL. PMLR, 2083–2094

    2023

  33. [35]

    Yi Chen, Jian Xu, Xu-Yao Zhang, Wen-Zhuo Liu, Yang-Yang Liu, and Cheng-Lin Liu. 2025. Recoverable compression: A multimodal vision token recovery mechanism guided by text information. In AAAI, V ol. 39. 2293–2301

    2025

  34. [36]

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. 2024. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In CVPR. 24185–24198

    2024

  35. [37]

    Hongrong Cheng, Miao Zhang, and Javen Qinfeng Shi. 2024. A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommendations. IEEE TPAMI (2024)

    2024

  36. [38]

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. 2023. Diffusion policy: Visuomotor policy learning via action diffusion. IJRR (2023), 02783649241273668

    2023

  37. [39]

    Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. 2025. Sft memorizes, rl generalizes: A comparative study of foundation model post-training. arXiv preprint arXiv:2501.17161 (2025)

    2025 arXiv

  38. [40]

    Yulai Cong, Miaoyun Zhao, Jianqiao Li, Sijia Wang, and Lawrence Carin. 2020. Gan memory with no forgetting. InNeurIPS, V ol. 33. 16481–16494

    2020

  39. [41]

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. 2023. InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning. arXiv preprint arXiv:2305.06500 (2023)

    2023 arXiv

  40. [42]

    Thomas J Davidson, Fabian Kloosterman, and Matthew A Wilson. 2009. Hippocampal replay of extended experience.Neuron 63, 4 (2009), 497–507

    2009

  41. [43]

    Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. 2021. A continual learning survey: Defying forgetting in classification tasks. IEEE TPAMI 44, 7 (2021), 3366–3385

    2021

  42. [44]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL. 4171–4186

    2019

  43. [45]

    Jiahua Dong, Wenqi Liang, Hongliu Li, Duzhen Zhang, Meng Cao, Henghui Ding, Salman H Khan, and Fahad Shahbaz Khan. 2024. How to Continually Adapt Text-to-Image Diffusion Models for Flexible Customization?. In NeurIPS, V ol. 37. 130057–130083

    2024

  44. [46]

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, et al. 2022. A survey on in-context learning. arXiv preprint arXiv:2301.00234 (2022)

    2022 arXiv

  45. [48]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint...

    2020 arXiv

  46. [49]

    David R Euston, Aaron J Gruber, and Bruce L McNaughton. 2012. The role of medial prefrontal cortex in memory and decision making. Neuron 76, 6 (2012), 1057–1070

    2012

  47. [50]

    Yujie Feng, Xu Chu, Yongxin Xu, Guangyuan Shi, Bo Liu, and Xiao-Ming Wu. 2024. Tasl: Continual dialog state tracking via task skill localization and consolidation. arXiv preprint arXiv:2408.09857 (2024)

    2024 arXiv

  48. [51]

    Yujie Feng, Xujia Wang, Zexin Lu, Shenghong Fu, Guangyuan Shi, Yongxin Xu, Yasha Wang, Philip S Yu, Xu Chu, and Xiao-Ming Wu. 2025. Recurrent knowledge identification and fusion for language model continual learning. arXiv preprint arXiv:2502.17510 (2025). Manuscript submitted...

    2025 arXiv

  49. [52]

    Jared Fernandez, Yonatan Bisk, and Emma Strubell. 2024. Gradient Localization Improves Lifelong Pretraining of Language Models. arXiv preprint arXiv:2411.04448 (2024)

    2024 arXiv

  50. [53]

    Timo Flesch, Keno Juechems, Tsvetomira Dumbalska, Andrew Saxe, and Christopher Summerfield. 2022. Orthogonal representations for robust context-dependent task performance in brains and neural networks. Neuron 110, 7 (2022), 1258–1270

    2022

  51. [54]

    Paul W Frankland and Bruno Bontempi. 2005. The organization of recent and remote memories. Nature reviews neuroscience 6, 2 (2005), 119–130

    2005

  52. [55]

    Chongkai Gao, Haichuan Gao, Shangqi Guo, Tianren Zhang, and Feng Chen. 2021. CRIL: Continual robot imitation learning via generative and prediction model. In IROS. IEEE, 6747–5754

    2021

  53. [56]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yixin Dai, Jiawei Sun, Haofen Wang, and Haofen Wang. 2023. Retrieval- augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997 2 (2023), 1

    2023 arXiv

  54. [57]

    Chendi Ge, Xin Wang, Zeyang Zhang, Hong Chen, Jiapei Fan, Longtao Huang, Hui Xue, and Wenwu Zhu. 2025. Dynamic Mixture of Curriculum LoRA Experts for Continual Multimodal Instruction Tuning. arXiv preprint arXiv:2506.11672 (2025)

    2025 arXiv

  55. [58]

    Yuying Ge, Sijie Zhao, Jinguo Zhu, Yixiao Ge, Kun Yi, Lin Song, Chen Li, Xiaohan Ding, and Ying Shan. 2024. Seed-x: Multimodal models with unified multi-granularity comprehension and generation. arXiv preprint arXiv:2404.14396 (2024)

    2024 arXiv

  56. [59]

    Binzong Geng, Fajie Yuan, Qiancheng Xu, Ying Shen, Ruifeng Xu, and Min Yang. 2021. Continual learning for task-oriented dialogue system with iterative network pruning, expanding and masking. arXiv preprint arXiv:2107.08173 (2021)

    2021 arXiv

  57. [60]

    Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR. 580–587

    2014

  58. [61]

    Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In NeurIPS, V ol. 27

    2014

  59. [62]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024)

    2024 arXiv

  60. [63]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al . 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 (2025)

    2025 arXiv

  61. [64]

    Haiyang Guo, Fanhu Zeng, Ziwei Xiang, Fei Zhu, Da-Han Wang, Xu-Yao Zhang, and Cheng-Lin Liu. 2025. Hide-llava: Hierarchical decoupling for continual instruction tuning of multimodal large language model. arXiv preprint arXiv:2503.12941 (2025)

    2025 arXiv

  62. [65]

    Haiyang Guo, Fanhu Zeng, Fei Zhu, Wenzhuo Liu, Da-Han Wang, Jian Xu, Xu-Yao Zhang, and Cheng-Lin Liu. 2025. Federated continual instruction tuning. arXiv preprint arXiv:2503.12897 (2025)

    2025 arXiv

  63. [66]

    Haiyang Guo, Fei Zhu, Fanhu Zeng, Bing Liu, and Xu-Yao Zhang. 2024. DESIRE: Dynamic Knowledge Consolidation for Rehearsal-Free Continual Learning. arXiv preprint arXiv:2411.19154 (2024)

    2024 arXiv

  64. [67]

    Yiduo Guo, Jie Fu, Huishuai Zhang, Dongyan Zhao, and Yikang Shen. 2024. Efficient continual pre-training by mitigating the stability gap. arXiv preprint arXiv:2406.14833 (2024)

    2024 arXiv

  65. [68]

    Yanhui Guo, Shaoyuan Xu, Jinmiao Fu, Jia Liu, Chaosheng Dong, and Bryan Wang. 2024. Q-tuning: Queue-based prompt tuning for lifelong few-shot language learning. arXiv preprint arXiv:2404.14607 (2024)

    2024 arXiv

  66. [69]

    Yanjiang Guo, Jianke Zhang, Xiaoyu Chen, Xiang Ji, Yen-Jen Wang, Yucheng Hu, and Jianyu Chen. 2025. Improving Vision-Language-Action Model with Online Reinforcement Learning. arXiv preprint arXiv:2501.16664 (2025)

    2025 arXiv

  67. [70]

    Zirun Guo and Tao Jin. 2025. ConceptGuard: Continual Personalized Text-to-Image Generation with Forgetting and Confusion Mitigation. arXiv preprint arXiv:2503.10358 (2025)

    2025 arXiv

  68. [71]

    Anoopum S Gupta, Matthijs AA Van Der Meer, David S Touretzky, and A David Redish. 2010. Hippocampal replay is not a simple function of experience. Neuron 65, 5 (2010), 695–705

    2010

  69. [73]

    Jiayi Han, Liang Du, Hongwei Du, Xiangguo Zhou, Yiwen Wu, Weibo Zheng, and Donghong Han. 2024. Slim: Let llm learn more and forget less with soft lora and identity mixture. arXiv preprint arXiv:2410.07739 (2024)

    2024 arXiv

  70. [74]

    Xue Han, Yitong Wang, Junlan Feng, Qian Hu, Chao Deng, et al. 2025. LOIRE: LifelOng learning on Incremental data via pre-trained language model gRowth Efficiently. In ICLR

    2025

  71. [75]

    Nicklas Hansen, Xiaolong Wang, and Hao Su. 2022. Temporal difference learning for model predictive control. arXiv preprint arXiv:2203.04955 (2022)

    2022 arXiv

  72. [76]

    Akiko Hayashi-Takagi, Sho Yagishita, Mayumi Nakamura, Fukutoshi Shirai, Yi I Wu, Amanda L Loshbaugh, Brian Kuhlman, Klaus M Hahn, and Haruo Kasai. 2015. Labelling and optical erasure of synaptic memory traces in the motor cortex. Nature 525, 7569 (2015), 333–338

    2015

  73. [77]

    Jinghan He, Haiyun Guo, Ming Tang, and Jinqiao Wang. 2023. Continual instruction tuning for large multimodal models. arXiv preprint arXiv:2311.16206 (2023)

    2023 arXiv

  74. [78]

    Jinghan He, Haiyun Guo, Kuan Zhu, Zihan Zhao, Ming Tang, and Jinqiao Wang. 2024. SEEKR: Selective Attention-Guided Knowledge Retention for Continual Learning of Large Language Models. In EMNLP. 3254–3266

    2024

  75. [79]

    Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. Mask r-cnn. In ICCV. 2961–2969

    2017

  76. [80]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR. 770–778

    2016

  77. [81]

    Yongquan He, Wenyuan Zhang, Xuancheng Huang, Peng Zhang, Lingxun Meng, Xiang Zhou, Ke Zeng, and Xunliang Cai. 2024. Don’t Half-listen: Capturing Key-part Information in Continual Instruction Tuning. arXiv preprint arXiv:2403.10056 (2024). Manuscript submitted to ACM Continual ...

    2024 arXiv

  78. [82]

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In NeurIPS, V ol. 30

    2017

  79. [83]

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)

    2015 arXiv

  80. [84]

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. In NeurIPS, V ol. 33. 6840–6851

    2020

  81. [85]

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. 2022. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556 (2022)

    2022 arXiv

  82. [86]

    Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 2023. 3d-llm: Injecting the 3d world into large language models. NeurIPS 36 (2023), 20482–20494

    2023

  83. [87]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. Lora: Low-rank adaptation of large language models.. In ICLR, V ol. 1. 3

    2022

  84. [88]

    Tianyu Huai, Jie Zhou, Xingjiao Wu, Qin Chen, Qingchun Bai, Ze Zhou, and Liang He. 2025. CL-MoE: Enhancing Multimodal Large Language Model with Dual Momentum Mixture-of-Experts for Continual Visual Question Answering. arXiv preprint arXiv:2503.00413 (2025)

    2025 arXiv

  85. [89]

    Jianheng Huang, Leyang Cui, Ante Wang, Chengyi Yang, Xinting Liao, Linfeng Song, Junfeng Yao, and Jinsong Su. 2024. Mitigating catastrophic forgetting in large language models with self-synthesized rehearsal. arXiv preprint arXiv:2403.01244 (2024)

    2024 arXiv

  86. [90]

    Jerry Huang, Prasanna Parthasarathi, Mehdi Rezagholizadeh, and Sarath Chandar. 2024. Towards Practical Tool Usage for Continually Learning LLMs. arXiv preprint arXiv:2404.09339 (2024)

    2024 arXiv

  87. [91]

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. 2025. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM TOIS 43, 2 (2025), 1–55

    2025

  88. [92]

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. 2025. Vision-r1: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749 (2025)

    2025 arXiv

  89. [93]

    Wenke Huang, Jian Liang, Xianda Guo, Yiyang Fang, Guancheng Wan, Xuankun Rong, Chi Wen, Zekun Shi, Qingyun Li, Didi Zhu, et al. 2025. Keeping yourself is important in downstream tuning multimodal large language model. arXiv preprint arXiv:2503.04543 (2025)

    2025 arXiv

  90. [94]

    Wenke Huang, Jian Liang, Zekun Shi, Didi Zhu, Guancheng Wan, He Li, Bo Du, Dacheng Tao, and Mang Ye. 2024. Learn from Downstream and Be Yourself in Multimodal Large Language Model Fine-Tuning. arXiv preprint arXiv:2411.10928 (2024)

    2024 arXiv

  91. [95]

    Yukang Huo and Hao Tang. 2025. When Continue Learning Meets Multimodal Large Language Model: A Survey. arXiv preprint arXiv:2503.01887 (2025)

    2025 arXiv

  92. [96]

    Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and Joao Carreira. 2021. Perceiver: General perception with iterative attention. In ICML. PMLR, 4651–4664

    2021

  93. [97]

    Nuri Jeong, Xiao Zheng, Abigail L Paulson, Stephanie M Prince, Victor P Nguyen, Sherina R Thomas, Caroline E Gilpin, Matthew C Goodson, and Annabelle C Singer. 2025. Goal-specific hippocampal inhibition gates learning. Nature (2025), 1–9

    2025

  94. [98]

    Saurav Jha, Shiqi Yang, Masato Ishii, Mengjie Zhao, christian simon, Muhammad Jehanzeb Mirza, Dong Gong, Lina Yao, Shusuke Takahashi, and Yuki Mitsufuji. 2025. Mining your own secrets: Diffusion Classifier Scores for Continual Personalization of Text-to-Image Diffusion Models....

    2025

  95. [99]

    Ziqi Jia, Anmin Wang, Xiaoyang Qu, Xiaowen Yang, and Jianzong Wang. 2025. Hierarchical-Task-Aware Multi-modal Mixture of Incremental LoRA Experts for Embodied Continual Learning. arXiv preprint arXiv:2506.04595 (2025)

    2025 arXiv

  96. [100]

    Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen. 2023. Motiongpt: Human motion as a foreign language.NeurIPS 36 (2023), 20067–20079

    2023

  97. [101]

    Gangwei Jiang, Caigao Jiang, Zhaoyi Li, Siqiao Xue, Jun Zhou, Linqi Song, Defu Lian, and Yin Wei. 2025. Unlocking the power of function vectors for characterizing and mitigating catastrophic forgetting in continual instruction tuning. arXiv preprint arXiv:2502.11019 (2025)

    2025 arXiv

  98. [102]

    Zixuan Ke, Yijia Shao, Haowei Lin, Tatsuya Konishi, Gyuhak Kim, and Bing Liu. 2023. Continual pre-training of language models. arXiv preprint arXiv:2302.03241 (2023)

    2023 arXiv

  99. [103]

    Byeonghwi Kim, Minhyuk Seo, and Jonghyun Choi. 2024. Online Continual Learning for Interactive Instruction Following Agents. In ICLR

    2024

  100. [104]

    Gyuhak Kim, Changnan Xiao, Tatsuya Konishi, and Bing Liu. 2023. Learnability and algorithm for continual learning. In ICML. PMLR, 16877– 16896

    2023

  101. [105]

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. 2024. Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246 (2024)

    2024 arXiv

  102. [106]

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017. Overcoming catastrophic forgetting in neural networks. PNAS 114, 13 (2017), 3521–3526

    2017

  103. [107]

    Zhenglun Kong, Yize Li, Fanhu Zeng, Lei Xin, Shvat Messica, Xue Lin, Pu Zhao, Manolis Kellis, Hao Tang, and Marinka Zitnik. 2025. Token Reduction Should Go Beyond Efficiency in Generative Models–From Vision, Language to Multimodality. arXiv preprint arXiv:2505.18227 (2025)

    2025

  104. [108]

    Georg Krempl, Indre Žliobaite, Dariusz Brzezi´nski, Eyke Hüllermeier, Mark Last, Vincent Lemaire, Tino Noack, Ammar Shaker, Sonja Sievi, Myra Spiliopoulou, et al. 2014. Open challenges for data stream mining research. ACM SIGKDD explorations newsletter 16, 1 (2014), 1–10

    2014

  105. [109]

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. InNeurIPS, V ol. 25

    2012

  106. [110]

    Dharshan Kumaran, Demis Hassabis, and James L McClelland. 2016. What learning systems do intelligent agents need? Complementary learning systems theory updated. Trends in cognitive sciences 20, 7 (2016), 512–534. Manuscript submitted to ACM 30 H. Guo et al

    2016

  107. [111]

    Mingrui Lao, Nan Pu, Yu Liu, Zhun Zhong, Erwin M Bakker, Nicu Sebe, and Michael S Lew. 2023. Multi-domain lifelong visual question answering via self-critical distillation. In ACM MM. 4747–4758

    2023

  108. [112]

    Minh Le, Tien Ngoc Luu, Thanh-Thien Le, Trang Nguyen, Tung Thanh Nguyen, Linh Ngo Van, Thien Huu Nguyen, et al. 2025. Adaptive Prompting for Continual Relation Extraction: A Within-Task Variance Perspective. InAAAI, V ol. 39. 24384–24392

    2025

  109. [114]

    Minjae Lee, Minhyuk Seo, Tingyu Qu, Tinne Tuytelaars, and Jonghyun Choi. 2025. OASIS: Online Sample Selection for Continual Visual Instruction Tuning. arXiv preprint arXiv:2506.02011 (2025)

    2025

  110. [115]

    Stan Weixian Lei, Difei Gao, Jay Zhangjie Wu, Yuxuan Wang, Wei Liu, Mengmi Zhang, and Mike Zheng Shou. 2023. Symbolic replay: Scene graph as prompt for continual learning on vqa task. In AAAI, V ol. 37. 1250–1259

    2023

  111. [116]

    Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691 (2021)

    2021 arXiv

  112. [117]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. In NeurIPS, V ol. 33. 9459–9474

    2020

  113. [118]

    Wentong Li, Yuqian Yuan, Jian Liu, Dongqi Tang, Song Wang, Jie Qin, Jianke Zhu, and Lei Zhang. 2024. Tokenpacker: Efficient visual projector for multimodal llm. arXiv preprint arXiv:2407.02392 (2024)

    2024 arXiv

  114. [119]

    Yi Li, Yuquan Deng, Jesse Zhang, Joel Jang, Marius Memmel, Raymond Yu, Caelan Reed Garrett, Fabio Ramos, Dieter Fox, Anqi Li, et al. 2025. HAMSTER: Hierarchical action models for open-world robot manipulation. arXiv preprint arXiv:2502.05485 (2025)

    2025 arXiv

  115. [120]

    Zhizhong Li and Derek Hoiem. 2017. Learning without forgetting. IEEE TPAMI 40, 12 (2017), 2935–2947

    2017

  116. [121]

    Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, et al. 2025. From system 1 to system 2: A survey of reasoning large language models. arXiv preprint arXiv:2502.17419 (2025)

    2025 arXiv

  117. [122]

    Jian Liang, Wenke Huang, Guancheng Wan, Qu Yang, and Mang Ye. 2025. Lorasculpt: Sculpting lora for harmonizing general and specialized knowledge in multimodal large language models. In Proceedings of the Computer Vision and Pattern Recognition Conference . 26170–26180

    2025

  118. [123]

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han

  119. [124]

    Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. 2017. Feature pyramid networks for object detection. In CVPR. 2117–2125

    2017

  120. [125]

    MLSys 6 (2024), 87–100

    Awq: Activation-aware weight quantization for on-device llm compression and acceleration. MLSys 6 (2024), 87–100

    2024

  121. [127]

    Yuxin Lin, Mengshi Qi, Liang Liu, and Huadong Ma. 2025. VLM-Assisted Continual learning for Visual Question Answering in Self-Driving. arXiv preprint arXiv:2502.00843 (2025)

    2025

  122. [128]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. Improved baselines with visual instruction tuning. In CVPR. 26296–26306

    2024

  123. [129]

    Chenxi Liu, Gan Sun, Wenqi Liang, Jiahua Dong, Can Qin, and Yang Cong. 2024. Museummaker: Continual style customization without catastrophic forgetting. arXiv preprint arXiv:2404.16612 (2024)

    2024 arXiv

  124. [130]

    Songhua Liu, Kai Wang, Xingyi Yang, Jingwen Ye, and Xinchao Wang. 2022. Dataset distillation via factorization. InNeurIPS, V ol. 35. 1100–1113

    2022

  125. [131]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. In NeurIPS, V ol. 36. 34892–34916

    2023

  126. [132]

    Wenzhuo Liu, Fei Zhu, Haiyang Guo, Longhui Wei, and Cheng-Lin Liu. 2025. LLaV A-c: Continual Improved Visual Instruction Tuning.arXiv preprint arXiv:2506.08666 (2025)

    2025 arXiv

  127. [133]

    Wenzhuo Liu, Xin-Jian Wu, Fei Zhu, Ming-Ming Yu, Chuang Wang, and Cheng-Lin Liu. 2025. Class incremental learning with self-supervised pre-training and prototype learning. Pattern Recognition 157 (2025), 110943

    2025

  128. [134]

    Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. 2024. Spinquant: Llm quantization with learned rotations. arXiv preprint arXiv:2405.16406 (2024)

    2024 arXiv

  129. [135]

    Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. 2025. Visual-rft: Visual reinforcement fine-tuning. arXiv preprint arXiv:2503.01785 (2025)

    2025 arXiv

  130. [136]

    Adrià López Escoriza, Nicklas Hansen, Stone Tao, Tongzhou Mu, and Hao Su. 2025. Multi-Stage Manipulation with Demonstration-Augmented Reward, Policy, and World Model Learning. arXiv preprint arXiv:2503.01837 (2025)

    2025

  131. [137]

    Yayu Long, Kewei Chen, Long Jin, and Mingsheng Shang. 2025. DRAE: Dynamic Retrieval-Augmented Expert Networks for Lifelong Learning and Task Adaptation in Robotics. arXiv preprint arXiv:2507.04661 (2025)

    2025

  132. [138]

    Zheheng Luo, Xin Zhang, Xiao Liu, Haoling Li, Yeyun Gong, Chen Qi, and Peng Cheng. 2024. Velocitune: A velocity-based dynamic domain reweighting method for continual pre-training. arXiv preprint arXiv:2411.14318 (2024)

    2024 arXiv

  133. [139]

    David Lopez-Paz and Marc’Aurelio Ranzato. 2017. Gradient episodic memory for continual learning. In NeurIPS, V ol. 30

    2017

  134. [140]

    Andrea Madotto, Zhaojiang Lin, Zhenpeng Zhou, Seungwhan Moon, Paul Crook, Bing Liu, Zhou Yu, Eunjoon Cho, and Zhiguang Wang. 2020. Continual learning in task-oriented dialogue systems. arXiv preprint arXiv:2012.15504 (2020)

    2020 arXiv

  135. [141]

    Shijie Ma, Fei Zhu, Zhen Cheng, and Xu-Yao Zhang. 2025. Towards trustworthy dataset distillation. Pattern Recognition 157 (2025), 110875

    2025

  136. [142]

    Adyasha Maharana, Jaehong Yoon, Tianlong Chen, and Mohit Bansal. 2024. Adapt-infty: Scalable Lifelong Multimodal Instruction Tuning via Dynamic Data Selection. arXiv preprint arXiv:2410.10636 (2024)

    2024 arXiv

  137. [143]

    Aru Maekawa, Hidetaka Kamigaito, Kotaro Funakoshi, and Manabu Okumura. 2023. Generative replay inspired by hippocampal memory indexing for continual language learning. In EACL. 930–942. Manuscript submitted to ACM Continual Learning for Generative AI: From LLMs to MLLMs and Beyond 31

    2023

  138. [144]

    Marc Masana, Xialei Liu, Bartłomiej Twardowski, Mikel Menta, Andrew D Bagdanov, and Joost Van De Weijer. 2022. Class-incremental learning: survey and performance evaluation on image classification. IEEE TPAMI 45, 5 (2022), 5513–5533

    2022

  139. [145]

    Imad Eddine Marouf, Enzo Tartaglione, Stephane Lathuiliere, and Joost van de Weijer. 2025. No Images, No Problem: Retaining Knowledge in Continual VQA with Questions-Only Memory. arXiv preprint arXiv:2502.04469 (2025)

    2025 arXiv

  140. [146]

    James L McClelland, Bruce L McNaughton, and Randall C O’Reilly. 1995. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological review 102, 3 (1995), 419

    1995

  141. [147]

    James L McClelland and Nigel H Goddard. 1996. Considerations arising from a complementary learning systems perspective on hippocampus and neocortex. Hippocampus 6, 6 (1996), 654–665

    1996

  142. [150]

    Angelo G Menezes, Gustavo de Moura, Cézanne Alves, and André CPLF de Carvalho. 2023. Continual object detection: a review of definitions, strategies, and challenges. Neural networks 161 (2023), 476–493

    2023

  143. [151]

    Jorge A Mendez, Harm van Seijen, and ERIC EATON. 2022. Modular Lifelong Reinforcement Learning via Neural Composition. In ICLR

    2022

  144. [152]

    Zichong Meng, Jie Zhang, Changdi Yang, Zheng Zhan, Pu Zhao, and Yanzhi Wang. 2024. Diffclass: Diffusion-based class incremental learning. In ECCV. Springer, 142–159

    2024

  145. [153]

    Yuan Meng, Zhenshan Bing, Xiangtong Yao, Kejia Chen, Kai Huang, Yang Gao, Fuchun Sun, and Alois Knoll. 2025. Preserving and combining knowledge in robotic lifelong reinforcement learning. Nature Machine Intelligence (2025), 1–14

    2025

  146. [154]

    Fei Mi, Liangwei Chen, Mengjie Zhao, Minlie Huang, and Boi Faltings. 2020. Continual learning for natural language generation in task-oriented dialog systems. arXiv preprint arXiv:2010.00910 (2020)

    2020 arXiv

  147. [155]

    Atharva Mete, Haotian Xue, Albert Wilcox, Yongxin Chen, and Animesh Garg. 2024. Quest: Self-supervised skill abstractions for learning continuous control. In NeurIPS, V ol. 37. 4062–4089

    2024

  148. [156]

    Saleh Momeni, Sahisnu Mazumder, Zixuan Ke, and Bing Liu. 2024. In-context Continual Learning Assisted by an External Continual Learner. arXiv preprint arXiv:2412.15563 (2024)

    2024 arXiv

  149. [157]

    Jisoo Mok, Jaeyoung Do, Sungjin Lee, Tara Taghavi, Seunghak Yu, and Sungroh Yoon. 2023. Large-scale lifelong learning of in-context instructions and how to tackle it. In ACL. 12573–12589

    2023

  150. [158]

    Malvina Nikandrou, Georgios Pantazopoulos, Ioannis Konstas, and Alessandro Suglia. 2024. Enhancing Continual Learning in Visual Question Answering with Modality-Aware Feature Distillation. arXiv preprint arXiv:2406.19297 (2024)

    2024 arXiv

  151. [159]

    Morris Moscovitch, Roberto Cabeza, Gordon Winocur, and Lynn Nadel. 2016. Episodic memory and beyond: the hippocampus and neocortex in transformation. Annual review of psychology 67, 1 (2016), 105–134

    2016

  152. [160]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In ACL. 311–318

    2002

  153. [161]

    H Freyja Ólafsdóttir, Daniel Bush, and Caswell Barry. 2018. The role of hippocampal replay in memory and planning. Current Biology 28, 1 (2018), R37–R50

    2018

  154. [162]

    Jitendra Parmar, Satyendra Chouhan, Vaskar Raychoudhury, and Santosh Rathore. 2023. Open-world machine learning: applications, challenges, and opportunities. Comput. Surveys 55, 10 (2023), 1–37

    2023

  155. [163]

    Meenal Parakh, Alisha Fong, Anthony Simeonov, Tao Chen, Abhishek Gupta, and Pulkit Agrawal. 2024. Lifelong robot learning with human assisted language planners. In ICRA. IEEE, 523–529

    2024

  156. [164]

    Weiguo Pian, Shijian Deng, Shentong Mo, Yunhui Guo, and Yapeng Tian. 2024. Modality-Inconsistent Continual Learning of Multimodal Large Language Models. arXiv preprint arXiv:2412.13050 (2024)

    2024 arXiv

  157. [165]

    Yingzhe Peng, Gongrui Zhang, Miaosen Zhang, Zhiyuan You, Jie Liu, Qipeng Zhu, Kai Yang, Xingzhong Xu, Xin Geng, and Xu Yang. 2025. Lmm-r1: Empowering 3b lmms with strong reasoning abilities through two-stage rule-based rl. arXiv preprint arXiv:2503.07536 (2025)

    2025 arXiv

  158. [166]

    Sam Powers, Eliot Xing, Eric Kolve, Roozbeh Mottaghi, and Abhinav Gupta. 2022. Cora: Benchmarks, baselines, and metrics as a platform for continual reinforcement learning agents. In CoLLAs. PMLR, 705–743

    2022

  159. [167]

    Weiguo Pian, Yiyang Nan, Shijian Deng, Shentong Mo, Yunhui Guo, and Yapeng Tian. 2024. Continual audio-visual sound separation. InNeurIPS, V ol. 37. 76058–76079

    2024

  160. [168]

    Jingyang Qiao, Zhizhong Zhang, Xin Tan, Yanyun Qu, Shouhong Ding, and Yuan Xie. 2024. LLaCA: Multimodal Large Language Continual Assistant. arXiv preprint arXiv:2410.10868 (2024)

    2024

  161. [169]

    Zi Qian, Xin Wang, Xuguang Duan, Pengda Qin, Yuhong Li, and Wenwu Zhu. 2023. Decouple before interact: Multi-modal prompt learning for continual visual question answering. In ICCV. 2953–2962

    2023

  162. [170]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. GPT 2; Language Models are Unsupervised Multitask Learners. In 2019 by OpenAI. Manuscript submitted to ACM 32 H. Guo et al

    2019

  163. [171]

    Haoran Que, Jiaheng Liu, Ge Zhang, Chenchen Zhang, Xingwei Qu, Yinghao Ma, Feiyu Duan, Zhiqi Bai, Jiakai Wang, Yuanxing Zhang, et al. 2024. D-cpt law: Domain-specific continual pre-training scaling law for large language models. In NeurIPS, V ol. 37. 90318–90354

    2024

  164. [172]

    Giuseppe Rauso, Riccardo Caccavale, and Alberto Finzi. 2024. Incremental Learning of Robotic Manipulation Tasks through Virtual Reality Demonstrations. In IROS. IEEE, 5176–5181

    2024

  165. [173]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research 21, 140 (2020), 1–67

    2020

  166. [174]

    Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2016. You only look once: Unified, real-time object detection. InCVPR. 779–788

    2016

  167. [175]

    Anastasia Razdaibiedina, Yuning Mao, Rui Hou, Madian Khabsa, Mike Lewis, and Amjad Almahairi. 2023. Progressive prompts: Continual learning for language models. arXiv preprint arXiv:2301.12314 (2023)

    2023 arXiv

  168. [176]

    Weijieying Ren, Xinlong Li, Lei Wang, Tianxiang Zhao, and Wei Qin. 2024. Analyzing and reducing catastrophic forgetting in parameter efficient tuning. arXiv preprint arXiv:2402.18865 (2024)

    2024 arXiv

  169. [177]

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. In NeurIPS, V ol. 28

    2015

  170. [178]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In CVPR. 10684–10695

    2022

  171. [179]

    David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. 2019. Experience replay for continual learning. In NeurIPS, V ol. 32

    2019

  172. [180]

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. 2023. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In CVPR. 22500–22510

    2023

  173. [181]

    Kaushik Roy, Akila Dissanayake, Brendan Tidd, and Peyman Moghadam. 2024. M2Distill: Multi-Modal Distillation for Lifelong Imitation Learning. arXiv preprint arXiv:2410.00064 (2024)

    2024 arXiv

  174. [182]

    Thomas Scialom, Tuhin Chakrabarty, and Smaranda Muresan. 2022. Fine-tuned language models are continual learners. arXiv preprint arXiv:2205.12393 (2022)

    2022 arXiv

  175. [183]

    Khaled Saab, Tao Tu, Wei-Hung Weng, Ryutaro Tanno, David Stutz, Ellery Wulczyn, Fan Zhang, Tim Strother, Chunjong Park, Elahe Vedadi, et al

  176. [184]

    arXiv preprint arXiv:2404.18416 (2024)

    Capabilities of gemini models in medicine. arXiv preprint arXiv:2404.18416 (2024)

    2024 arXiv

  177. [185]

    Haizhou Shi, Zihao Xu, Hengyi Wang, Weiyi Qin, Wenyuan Wang, Yibin Wang, Zifeng Wang, Sayna Ebrahimi, and Hao Wang. 2024. Continual learning of large language models: A comprehensive survey. arXiv preprint arXiv:2404.16789 (2024)

    2024 arXiv

  178. [186]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al . 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 (2024)

    2024 arXiv

  179. [187]

    Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. 2025. Vlm-r1: A stable and generalizable r1-style large vision-language model. arXiv preprint arXiv:2504.07615 (2025)

    2025 arXiv

  180. [188]

    Anton Sirota, Jozsef Csicsvari, Derek Buhl, and György Buzsáki. 2003. Communication between neocortex and hippocampus during sleep in rodents. PNAS 100, 4 (2003), 2065–2069

    2003

  181. [189]

    Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. 2017. Continual learning with deep generative replay. In NeurIPS, V ol. 30

    2017

  182. [190]

    Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

    2014 arXiv

  183. [191]

    Tejas Srinivasan, Ting-Yun Chang, Leticia Pinto Alva, Georgios Chochlakis, Mohammad Rostami, and Jesse Thomason. 2022. Climb: A continual learning benchmark for vision-and-language tasks. In NeurIPS, V ol. 35. 29440–29453

    2022

  184. [192]

    James Seale Smith, Yen-Chang Hsu, Lingyu Zhang, Ting Hua, Zsolt Kira, Yilin Shen, and Hongxia Jin. 2023. Continual diffusion: Continual customization of text-to-image diffusion with c-lora. arXiv preprint arXiv:2304.06027 (2023)

    2023 arXiv

  185. [193]

    Nan Song, Xiaofeng Yang, Ze Yang, and Guosheng Lin. 2024. Towards Lifelong Few-Shot Customization of Text-to-Image Diffusion.arXiv preprint arXiv:2411.05544 (2024)

    2024 arXiv

  186. [194]

    Huashan Sun and Yang Gao. 2024. Reviving Dormant Memories: Investigating Catastrophic Forgetting in Language Models through Rationale- Guidance Difficulty. arXiv preprint arXiv:2411.11932 (2024)

    2024 arXiv

  187. [195]

    Fan-Keng Sun, Cheng-Hao Ho, and Hung-Yi Lee. 2019. Lamol: Language modeling for lifelong language learning. arXiv preprint arXiv:1909.03329 (2019)

    2019 arXiv

  188. [196]

    Gan Sun, Wenqi Liang, Jiahua Dong, Jun Li, Zhengming Ding, and Yang Cong. 2024. Create your world: Lifelong text-to-image diffusion. IEEE TPAMI (2024)

    2024

  189. [197]

    Georgios Tziafas and Hamidreza Kasaei. 2024. Lifelong robot library learning: Bootstrapping composable and generalizable skills for embodied control with language models. In ICRA. IEEE, 515–522

    2024

  190. [198]

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530 (2024)

    2024 arXiv

  191. [199]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)

    2023 arXiv

  192. [200]

    Weikang Wan, Yifeng Zhu, Rutav Shah, and Yuke Zhu. 2024. Lotus: Continual imitation learning for robot manipulation through unsupervised skill discovery. In ICRA. IEEE, 537–544

    2024

  193. [201]

    Gido M Van de Ven, Tinne Tuytelaars, and Andreas S Tolias. 2022. Three types of incremental learning. Nature Machine Intelligence 4, 12 (2022), 1185–1197

    2022

  194. [202]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS, V ol. 30. Manuscript submitted to ACM Continual Learning for Generative AI: From LLMs to MLLMs and Beyond 33

    2017

  195. [203]

    Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, and Alexei A Efros. 2018. Dataset distillation. arXiv preprint arXiv:1811.10959 (2018)

    2018 arXiv

  196. [204]

    Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. 2024. A comprehensive survey of continual learning: Theory, method and application. IEEE TPAMI (2024)

    2024

  197. [205]

    Shaobo Wang, Yicun Yang, Zhiyuan Liu, Chenghao Sun, Xuming Hu, Conghui He, and Linfeng Zhang. 2025. Dataset distillation with neural characteristic function: A minmax perspective. In CVPR. 25570–25580

    2025

  198. [206]

    Yifan Wang, Yafei Liu, Chufan Shi, Haoling Li, Chen Chen, Haonan Lu, and Yujiu Yang. 2024. Inscl: A data-efficient continual learning paradigm for fine-tuning large language models with instructions. arXiv preprint arXiv:2403.11435 (2024)

    2024 arXiv

  199. [207]

    Xiao Wang, Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, and Xuan-Jing Huang. 2023. Orthogonal Subspace Learning for Language Model Continual Learning. In Findings of the Association for Computational Linguistics: EMNLP 2023 . 10658–10671

    2023

  200. [208]

    Xiao Wang, Yuansen Zhang, Tianze Chen, Songyang Gao, Senjie Jin, Xianjun Yang, Zhiheng Xi, Rui Zheng, Yicheng Zou, Tao Gui, et al. 2023. Trace: A comprehensive benchmark for continual learning in large language models. arXiv preprint arXiv:2310.06762 (2023)

    2023 arXiv

  201. [209]

    Zhilin Wang, Yafu Li, Xiaoye Qu, and Yu Cheng. 2025. SEE: Continual Fine-tuning with Sequential Ensemble of Experts. arXiv preprint arXiv:2504.06664 (2025)

    2025 arXiv

  202. [210]

    Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. 2022. Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks. arX...

    2022 arXiv

  203. [211]

    Ziqi Wang, Chang Che, Qi Wang, Yangyang Li, Zenglin Shi, and Meng Wang. 2024. Separable Mixture of Low-Rank Adaptation for Continual Visual Instruction Tuning. arXiv preprint arXiv:2411.13949 (2024)

    2024 arXiv

  204. [212]

    Chenshen Wu, Luis Herranz, Xialei Liu, Joost Van De Weijer, Bogdan Raducanu, et al . 2018. Memory replay gans: Learning to generate new categories without forgetting. In NeurIPS, V ol. 31

    2018

  205. [213]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS, V ol. 35. 24824–24837

    2022

  206. [214]

    Maciej Wołczyk, Michał Zaj ˛ ac, Razvan Pascanu, Łukasz Kuci´nski, and Piotr Miło´s. 2021. Continual world: A robotic benchmark for continual reinforcement learning. In NeurIPS, V ol. 34. 28496–28510

    2021

  207. [215]

    Tongtong Wu, Linhao Luo, Yuan-Fang Li, Shirui Pan, Thuy-Trang Vu, and Gholamreza Haffari. 2024. Continual learning for large language models: A survey. arXiv preprint arXiv:2402.01364 (2024)

    2024 arXiv

  208. [216]

    Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua. 2024. Next-gpt: Any-to-any multimodal llm. In ICML

    2024

  209. [217]

    Ting Wu, Jingyi Liu, Rui Zheng, Qi Zhang, Tao Gui, and Xuanjing Huang. 2023. Enhancing Contrastive Learning with Noise-Guided Attack: Towards Continual Relation Extraction in the Wild. arXiv preprint arXiv:2305.07085 (2023)

    2023 arXiv

  210. [218]

    Annie Xie and Chelsea Finn. 2022. Lifelong robotic reinforcement learning by retaining experiences. In CoLLAs. PMLR, 838–855

    2022

  211. [219]

    Xinjian Wu, Fanhu Zeng, Xiudong Wang, and Xinghao Chen. 2023. Ppt: Token pruning and pooling for efficient vision transformers. arXiv preprint arXiv:2310.01812 (2023)

    2023 arXiv

  212. [220]

    Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. 2025. The rise and potential of large language model based agents: A survey. Science China Information Sciences 68, 2 (2025), 121101

    2025

  213. [221]

    Yang Xie, Peiyao Hu, Junru Li, Jingwen Chen, Weibin Song, Xiao-Jing Wang, Tianming Yang, Stanislas Dehaene, Shiming Tang, Bin Min, et al

  214. [222]

    Junlin Xie, Zhihong Chen, Ruifei Zhang, Xiang Wan, and Guanbin Li. 2024. Large multimodal agents: A survey. arXiv preprint arXiv:2402.15116 (2024)

    2024 arXiv

  215. [223]

    Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. 2024. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408.12528 (2024)

    2024 arXiv

  216. [224]

    Gui Xue, Qi Dong, Chuansheng Chen, Zhonglin Lu, Jeanette A Mumford, and Russell A Poldrack. 2010. Greater neural pattern similarity across repetitions is associated with better memory. Science 330, 6000 (2010), 97–101

    2010

  217. [225]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115 (2024)

    2024 arXiv

  218. [226]

    Shuo Xing, Chengyuan Qian, Yuping Wang, Hongyuan Hua, Kexin Tian, Yang Zhou, and Zhengzhong Tu. 2025. Openemma: Open-source multimodal model for end-to-end autonomous driving. In WACV. 1001–1009

    2025

  219. [227]

    Xiaoyue Xu, Qinyuan Ye, and Xiang Ren. 2024. Stress-testing long-context language models with lifelong icl and task haystack. arXiv preprint arXiv:2407.16695 (2024)

    2024 arXiv

  220. [228]

    Shu Yang, Muhammad Asif Ali, Cheng-Long Wang, Lijie Hu, and Di Wang. 2024. MoRAL: MoE Augmented LoRA for LLMs’ Lifelong Learning. arXiv preprint arXiv:2402.11260 (2024)

    2024 arXiv

  221. [229]

    Yutao Yang, Jie Zhou, Xuanwen Ding, Tianyu Huai, Shunyu Liu, Qin Chen, Yuan Xie, and Liang He. 2025. Recent advances of foundation language models-based continual learning: A survey. Comput. Surveys 57, 5 (2025), 1–38

    2025

  222. [230]

    Guang Yang, Feng Pan, and Wen-Biao Gan. 2009. Stably maintained dendritic spines are associated with lifelong memories. Nature 462, 7275 (2009), 920–924

    2009

  223. [231]

    Pengzhi Yang, Xinyu Wang, Ruipeng Zhang, Cong Wang, Frans Oliehoek, and Jens Kober. 2024. Task-Unaware Lifelong Robot Learning with Retrieval-based Weighted Local Adaptation. arXiv preprint arXiv:2410.02995 (2024). Manuscript submitted to ACM 34 H. Guo et al

    2024

  224. [232]

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. 2025. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476 (2025)

    2025 arXiv

  225. [233]

    Yahan Yu, Duzhen Zhang, Yong Ren, Xuanle Zhao, Xiuyi Chen, and Chenhui Chu. 2025. Progressive LoRA for Multimodal Continual Instruction Tuning. In Findings of the Association for Computational Linguistics: ACL 2025 . 2779–2796

    2025

  226. [234]

    Minjong Yoo, Jinwoo Jang, Wei-Jin Park, and Honguk Woo. 2024. Exploratory retrieval-augmented planning for continual embodied instruction following. In NeurIPS, V ol. 37. 67034–67060

    2024

  227. [235]

    Dianzhi Yu, Xinni Zhang, Yankai Chen, Aiwei Liu, Yifei Zhang, Philip S Yu, and Irwin King. 2024. Recent Advances of Multimodal Continual Learning: A Comprehensive Survey. arXiv preprint arXiv:2410.05352 (2024)

    2024

  228. [236]

    Fanhu Zeng, Zhen Cheng, Fei Zhu, and Xu-Yao Zhang. 2024. CLIPMisD: Few-Shot Prompt Learning for Misclassification Detection with Vision Language Model. In ICONIP. Springer

    2024

  229. [237]

    Fanhu Zeng, Haiyang Guo, Fei Zhu, Li Shen, and Hao Tang. 2025. Parameter Efficient Merging for Multimodal Large Language Models with Complementary Parameter Adaptation. arXiv preprint arXiv:2502.17159 (2025)

    2025

  230. [238]

    Bo Yuan and Danpei Zhao. 2024. A survey on continual semantic segmentation: Theory, challenge, method and application. IEEE TPAMI (2024)

    2024

  231. [239]

    Michał Zawalski, William Chen, Karl Pertsch, Oier Mees, Chelsea Finn, and Sergey Levine. 2024. Robotic control via embodied chain-of-thought reasoning. arXiv preprint arXiv:2407.08693 (2024)

    2024 arXiv

  232. [240]

    Fanhu Zeng, Fei Zhu, Haiyang Guo, Xu-Yao Zhang, and Cheng-Lin Liu. 2024. Modalprompt: Dual-modality guided prompt for continual learning of large multimodal models. arXiv preprint arXiv:2410.05849 (2024)

    2024 arXiv

  233. [241]

    Min Zeng, Wei Xue, Qifeng Liu, and Yike Guo. 2023. Continual learning with dirichlet generative-based rehearsal. arXiv preprint arXiv:2309.06917 (2023)

    2023 arXiv

  234. [242]

    Fanhu Zeng and Deli Yu. 2024. M2M-TAG: Training-Free Many-to-Many Token Aggregation for Vision Transformer Acceleration. InWorkshop on Machine Learning and Compression, NeurIPS 2024

    2024

  235. [243]

    Fanhu Zeng, Deli Yu, Zhenglun Kong, and Hao Tang. 2025. Token Transforming: A Unified and Training-Free Token Compression Framework for Vision Transformer Acceleration. arXiv preprint arXiv:2506.05709 (2025)

    2025 arXiv

  236. [244]

    Duzhen Zhang, Yong Ren, Zhong-Zhi Li, Yahan Yu, Jiahua Dong, Chenxing Li, Zhilong Ji, and Jinfeng Bai. 2025. Enhancing Multimodal Continual Instruction Tuning with BranchLoRA. arXiv preprint arXiv:2506.02041 (2025)

    2025 arXiv

  237. [245]

    Gengwei Zhang, Liyuan Wang, Guoliang Kang, Ling Chen, and Yunchao Wei. 2023. Slca: Slow learner with classifier alignment for continual learning on a pre-trained model. In ICCV. 19148–19158

    2023

  238. [246]

    Biao Zhang, Zhongtao Liu, Colin Cherry, and Orhan Firat. 2024. When scaling meets llm finetuning: The effect of data, model and finetuning method. arXiv preprint arXiv:2402.17193 (2024)

    2024 arXiv

  239. [247]

    Dong Zhang, Shimin Li, Xin Zhang, Jun Zhan, Pengyu Wang, Yaqian Zhou, and Xipeng Qiu. 2023. Speechgpt: Empowering large language models with intrinsic cross-modal conversational abilities. arXiv preprint arXiv:2305.11000 (2023)

    2023 arXiv

  240. [248]

    Licheng Zhang, Zhendong Mao, Yixing Peng, Zheren Fu, and Yongdong Zhang. 2025. Multi-Prototype Grouping for Continual Learning in Visual Question Answering. In ICASSP. IEEE, 1–5

    2025

  241. [249]

    Tiantian Zhang, Zichuan Lin, Yuxing Wang, Deheng Ye, Qiang Fu, Wei Yang, Xueqian Wang, Bin Liang, Bo Yuan, and Xiu Li. 2023. Dynamics- adaptive continual reinforcement learning via progressive contextualization. IEEE TNNLS (2023)

    2023

  242. [250]

    Haotian Zhang, Junting Zhou, Haowei Lin, Hang Ye, Jianhua Zhu, Zihao Wang, Liangcai Gao, Yizhou Wang, and Yitao Liang. 2024. CLoG: Benchmarking Continual Learning of Image Generation Models. arXiv preprint arXiv:2406.04584 (2024)

    2024 arXiv

  243. [251]

    Liang Zhang, Qin Jin, Haoyang Huang, Dongdong Zhang, and Furu Wei. 2024. Respond in my language: Mitigating language inconsistency in response generation based on large language models. In ACL. 4177–4192

    2024

  244. [252]

    Xu-Yao Zhang, Cheng-Lin Liu, and Ching Y Suen. 2020. Towards robust pattern recognition: A review. Proc. IEEE 108, 6 (2020), 894–922

    2020

  245. [253]

    Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. 2023. Multimodal chain-of-thought reasoning in language models. arXiv preprint arXiv:2302.00923 (2023)

    2023 arXiv

  246. [254]

    Xi Zhang, Feifei Zhang, and Changsheng Xu. 2023. Vqacl: A novel visual question answering continual learning setting. In CVPR. 19102–19112

    2023

  247. [255]

    Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In NeurIPS, V ol. 28

    2015

  248. [256]

    Hongbo Zhao, Fei Zhu, Rundong Wang, Gaofeng Meng, and Zhaoxiang Zhang. 2025. MLLM-CL: Continual Learning for Multimodal Large Language Models. arXiv preprint arXiv:2506.05453 (2025)

    2025

  249. [257]

    Weixiang Zhao, Shilong Wang, Yulin Hu, Yanyan Zhao, Bing Qin, Xuanyu Zhang, Qing Yang, Dongliang Xu, and Wanxiang Che. 2024. Sapt: A shared attention framework for parameter-efficient continual learning of large language models. arXiv preprint arXiv:2401.08295 (2024). Manuscri...

    2024 arXiv

  250. [258]

    Zijian Zhang, Kaiyuan Zheng, Zhaorun Chen, Joel Jang, Yi Li, Siwei Han, Chaoqi Wang, Mingyu Ding, Dieter Fox, and Huaxiu Yao. 2024. Grape: Generalizing robot policy via preference alignment. arXiv preprint arXiv:2411.19309 (2024)

    2024 arXiv

  251. [259]

    Hengyuan Zhao, Ziqin Wang, Qixin Sun, Kaiyou Song, Yilin Li, Xiaolin Hu, Qingpei Guo, and Si Liu. 2025. LLaV A-CMoE: Towards Continual Mixture of Experts for Large Vision-Language Models. arXiv preprint arXiv:2503.21227 (2025)

    2025 arXiv

  252. [260]

    Junhao Zheng, Xidi Cai, Qiuke Li, Duzhen Zhang, ZhongZhi Li, Yingying Zhang, Le Song, and Qianli Ma. 2025. LifelongAgentBench: Evaluating LLM Agents as Lifelong Learners. arXiv preprint arXiv:2505.11942 (2025)

    2025 arXiv

  253. [261]

    Junhao Zheng, Xidi Cai, Shengjie Qiu, and Qianli Ma. 2025. Spurious Forgetting in Continual Learning of Language Models. arXiv preprint arXiv:2501.13453 (2025)

    2025 arXiv

  254. [262]

    Yilong Zhao, Chien-Yu Lin, Kan Zhu, Zihao Ye, Lequn Chen, Size Zheng, Luis Ceze, Arvind Krishnamurthy, Tianqi Chen, and Baris Kasikci. 2024. Atom: Low-bit quantization for efficient and accurate llm serving. MLSys 6 (2024), 196–209

    2024

  255. [263]

    Yingxiu Zhao, Yinhe Zheng, Zhiliang Tian, Chang Gao, Bowen Yu, Haiyang Yu, Yongbin Li, Jian Sun, and Nevin L Zhang. 2022. Prompt conditioned vae: Enhancing generative replay for lifelong learning in task-oriented dialogue. arXiv preprint arXiv:2210.07783 (2022)

    2022 arXiv

  256. [264]

    Zexin Zheng, Jia-Feng Cai, Xiao-Ming Wu, Yi-Lin Wei, Yu-Ming Tang, and Wei-Shi Zheng. 2025. iManip: Skill-Incremental Learning for Robotic Manipulation. arXiv preprint arXiv:2503.07087 (2025)

    2025 arXiv

  257. [265]

    Da-Wei Zhou, Hai-Long Sun, Han-Jia Ye, and De-Chuan Zhan. 2024. Expandable subspace ensemble for pre-trained model-based class-incremental learning. In CVPR. 23554–23564

    2024

  258. [266]

    Junhao Zheng, Qianli Ma, Zhen Liu, Binquan Wu, and Huawen Feng. 2024. Beyond anti-forgetting: Multimodal continual instruction tuning with positive forward transfer. arXiv preprint arXiv:2401.09181 (2024)

    2024 arXiv

  259. [267]

    Junhao Zheng, Shengjie Qiu, Chengming Shi, and Qianli Ma. 2025. Towards lifelong learning of large language models: A survey. Comput. Surveys 57, 8 (2025), 1–35

    2025

  260. [268]

    Zhi-Hua Zhou. 2022. Open-environment machine learning. National Science Review 9, 8 (2022), nwac123

    2022

  261. [269]

    Didi Zhu, Zhongyi Sun, Zexi Li, Tao Shen, Ke Yan, Shouhong Ding, Kun Kuang, and Chao Wu. 2024. Model tailor: Mitigating catastrophic forgetting in multi-modal large language models. arXiv preprint arXiv:2402.12048 (2024)

    2024 arXiv

  262. [270]

    Da-Wei Zhou, Qi-Wei Wang, Zhi-Hong Qi, Han-Jia Ye, De-Chuan Zhan, and Ziwei Liu. 2024. Class-incremental learning: A survey.IEEE TPAMI (2024)

    2024

  263. [271]

    Zhongyi Zhou, Yichen Zhu, Minjie Zhu, Junjie Wen, Ning Liu, Zhiyuan Xu, Weibin Meng, Ran Cheng, Yaxin Peng, Chaomin Shen, et al. 2025. Chatvla: Unified multimodal understanding and robot control with vision-language-action model. arXiv preprint arXiv:2502.14420 (2025)

    2025 arXiv

  264. [272]

    Fei Zhu, Xu-Yao Zhang, Zhen Cheng, and Cheng-Lin Liu. 2025. PASS++: A dual bias reduction framework for non-exemplar class-incremental learning. IEEE TPAMI (2025)

    2025

  265. [273]

    Fei Zhu, Xu-Yao Zhang, Chuang Wang, Fei Yin, and Cheng-Lin Liu. 2021. Prototype augmentation and self-supervision for incremental learning. In CVPR. 5871–5880

    2021

  266. [274]

    Fei Zhu, Zhen Cheng, Xu-yao Zhang, and Cheng-lin Liu. 2021. Class-incremental learning via dual augmentation. InNeurIPS, V ol. 34. 14306–14318

    2021

  267. [275]

    Fei Zhu, Shijie Ma, Zhen Cheng, Xu-Yao Zhang, Zhaoxiang Zhang, and Cheng-Lin Liu. 2024. Open-world machine learning: A review and new outlooks. arXiv preprint arXiv:2403.01759 (2024)

    2024 arXiv

  268. [278]

    Xunyu Zhu, Jian Li, Yong Liu, Can Ma, and Weiping Wang. 2024. A survey on model compression for large language models. Transactions of the Association for Computational Linguistics 12 (2024), 1556–1577. Manuscript submitted to ACM

    2024

  269. [2022]

    Science 375, 6581 (2022), 632–639

    Geometry of sequence working memory in macaque prefrontal cortex. Science 375, 6581 (2022), 632–639

    2022

  270. [2024]

    arXiv preprint arXiv:2406.19280 (2024)

    Huatuogpt-vision, towards injecting medical visual knowledge into multimodal llms at scale. arXiv preprint arXiv:2406.19280 (2024)

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.