REVIEW 18 cited by
A Comprehensive Survey of Continual Learning: Theory, Method and Application
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
A Comprehensive Survey of Continual Learning: Theory, Method and Application
read the original abstract
To cope with real-world dynamics, an intelligent system needs to incrementally acquire, update, accumulate, and exploit knowledge throughout its lifetime. This ability, known as continual learning, provides a foundation for AI systems to develop themselves adaptively. In a general sense, continual learning is explicitly limited by catastrophic forgetting, where learning a new task usually results in a dramatic performance degradation of the old tasks. Beyond this, increasingly numerous advances have emerged in recent years that largely extend the understanding and application of continual learning. The growing and widespread interest in this direction demonstrates its realistic significance as well as complexity. In this work, we present a comprehensive survey of continual learning, seeking to bridge the basic settings, theoretical foundations, representative methods, and practical applications. Based on existing theoretical and empirical results, we summarize the general objectives of continual learning as ensuring a proper stability-plasticity trade-off and an adequate intra/inter-task generalizability in the context of resource efficiency. Then we provide a state-of-the-art and elaborated taxonomy, extensively analyzing how representative methods address continual learning, and how they are adapted to particular challenges in realistic applications. Through an in-depth discussion of promising directions, we believe that such a holistic perspective can greatly facilitate subsequent exploration in this field and beyond.
Forward citations
Cited by 18 Pith papers
-
Voyager: An Open-Ended Embodied Agent with Large Language Models
Voyager achieves superior lifelong learning in Minecraft by combining an automatic exploration curriculum, a library of executable skills, and iterative LLM prompting with environment feedback, yielding 3.3x more uniq...
-
ReCoLoRA: Spectrum-Aware Recursive Consolidation for Continual LLM Fine-Tuning
Spectrum-initialized LoRA with elbow ranks and recursive SVD consolidation of the effective weight beats rank-swept PEFT baselines on three of four 7–8B models in continual GLUE fine-tuning.
-
Unsupervised Continual Clustering via Forward-Backward Knowledge Distillation
FBCC introduces unsupervised continual clustering via a teacher-student forward-backward distillation process that outperforms baselines on clustering accuracy while reducing catastrophic forgetting.
-
TFGN: Task-Free, Replay-Free Continual Pre-Training Without Catastrophic Forgetting at LLM Scale
TFGN is an architectural overlay for transformers enabling task-free, replay-free continual pre-training across heterogeneous domains at LLM scale with near-zero backward transfer and high gradient orthogonality.
-
Rotation-Preserving Supervised Fine-Tuning
RPSFT improves the in-domain versus out-of-domain performance trade-off during LLM supervised fine-tuning by penalizing rotations in pretrained singular subspaces as a proxy for loss-sensitive directions.
-
Comprehensive AI governance requires addressing non-model gains
Non-model gains via inference, systems, and assets can drive AI capabilities independently of base models, requiring governance beyond model-level evaluation and mitigation.
-
You Don't Need Public Tests to Generate Correct Code
DryRUN lets LLMs create their own test inputs and run internal simulations for self-correcting code generation, matching the performance of test-dependent methods like CodeSIM on LiveCodeBench without public tests or ...
-
EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecycle
EvolveR enables LLM agents to self-evolve via a closed loop of distilling interaction trajectories into strategic principles offline and retrieving them to guide online decisions with policy reinforcement, yielding be...
-
Mask the Target: A Plug-and-Play Regularizer Against LoRA Forgetting
A plug-and-play KL regularizer that masks the target token and renormalizes probabilities to improve the learning-forgetting trade-off in LoRA adaptation of LLMs.
-
Anytime Training with Schedule-Free Spectral Optimization
SF-NorMuon is a new schedule-free spectral optimizer that closes the gap with tuned AdamW on 125M-772M parameter models across 1-8x Chinchilla horizons while providing stationarity guarantees.
-
Hybrid Edge-HPC Systems for Low-Latency Data-Driven Inference
RBF decouples edge inference from HPC simulation and training by using pluggable surrogate models at the edge with asynchronous updates to maintain low-latency predictions despite irregular model improvements.
-
HEDP: A Hybrid Energy-Distance Prompt-based Framework for Domain Incremental Learning
HEDP uses energy regularization inspired by Helmholtz free energy plus hybrid energy-distance weighting in prompts to improve domain selection and achieve a 2.57% accuracy gain on benchmarks like CORe50 while mitigati...
-
Incremental learning for audio classification with Hebbian Deep Neural Networks
A kernel plasticity approach in Hebbian DNNs for incremental sound classification achieves 76.3% accuracy over five steps on ESC-50, outperforming the 68.7% baseline without plasticity.
-
EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecycle
EvolveR proposes a closed-loop self-evolution system for LLM agents that distills experiences into principles offline and applies reinforcement during online task interactions to achieve better performance on multi-ho...
-
Hybrid Edge-HPC Systems for Low-Latency Data-Driven Inference
RBF is a hybrid edge-HPC architecture that decouples low-latency edge inference using surrogate models from asynchronous HPC-driven model updates for simulation-bounded cyber-physical systems.
-
LIFE -- an energy efficient advanced continual learning agentic AI framework for frontier systems
LIFE is a proposed agentic framework that combines four components to enable incremental, flexible, and energy-efficient continual learning for HPC operations such as latency spike mitigation.
-
The Rise and Potential of Large Language Model Based Agents: A Survey
The paper surveys the origins, frameworks, applications, and open challenges of AI agents built on large language models.
-
Domain-incremental audio classification using domain-specific experts and prototype classifier
Domain-incremental audio classification via frozen domain-specific experts plus prototype classifier on concatenated features yields 78.15% micro / 77.03% macro accuracy on the DCASE 2026 Task 7 development set.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.