Pith. sign in

REVIEW 10 cited by

Continual Learning with Pre-Trained Models: A Survey

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2401.16386 v2 pith:FDXJ5VJ4 submitted 2024-01-29 cs.LG cs.CV

Continual Learning with Pre-Trained Models: A Survey

Da-Wei Zhou , Hai-Long Sun , Jingyi Ning , Han-Jia Ye , De-Chuan Zhan This is my paper
classification cs.LG cs.CV
keywords learningdatacontinualknowledgemethodsmodelpre-trainedsurvey
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

Nowadays, real-world applications often face streaming data, which requires the learning system to absorb new knowledge as data evolves. Continual Learning (CL) aims to achieve this goal and meanwhile overcome the catastrophic forgetting of former knowledge when learning new ones. Typical CL methods build the model from scratch to grow with incoming data. However, the advent of the pre-trained model (PTM) era has sparked immense research interest, particularly in leveraging PTMs' robust representational capabilities. This paper presents a comprehensive survey of the latest advancements in PTM-based CL. We categorize existing methodologies into three distinct groups, providing a comparative analysis of their similarities, differences, and respective advantages and disadvantages. Additionally, we offer an empirical study contrasting various state-of-the-art methods to highlight concerns regarding fairness in comparisons. The source code to reproduce these evaluations is available at: https://github.com/sun-hailong/LAMDA-PILOT

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. CMAP: Cross-Modal Adaptive Prompting for Multi-Domain Task-Incremental Learning

    cs.CV 2026-05 unverdicted novelty 6.0

    CMAP uses text-space task routing with frozen CLIP prototypes, multi-prototype cross-modal confidence, and symmetric gating to achieve new state-of-the-art results on the MTIL benchmark with 2.5M parameters.

  2. Beyond Point-wise Neural Collapse: A Topology-Aware Hierarchical Classifier for Class-Incremental Learning

    cs.CV 2026-05 unverdicted novelty 6.0

    HC-SOINN with STAR captures topological manifold structure in class features and aligns it to non-linear drift, improving over point-wise NCM when integrated into existing CIL methods.

  3. Collaborative Parameter Learning: Mitigating Forgetting via Parameter-Level Gradient Analysis

    cs.LG 2026-01 conditional novelty 6.0

    Collaborative Parameter Learning freezes 50-75% of parameters whose updates cause forgetting and updates only the 25-50% that mitigate it, allowing LLMs to learn 20-48% more new questions with negligible forgetting an...

  4. iGSP:Implicit Gradient Subspace Projection for Efficient Continual Learning of Vision-Language Models

    cs.CV 2026-05 unverdicted novelty 5.0

    iGSP uses implicit gradient subspace projection in two phases to enable efficient continual adaptation of vision-language models, claiming SOTA accuracy with 42.7% fewer trainable parameters and 86.9% less total param...

  5. HEDP: A Hybrid Energy-Distance Prompt-based Framework for Domain Incremental Learning

    cs.AI 2026-05 unverdicted novelty 5.0

    HEDP uses energy regularization inspired by Helmholtz free energy plus hybrid energy-distance weighting in prompts to improve domain selection and achieve a 2.57% accuracy gain on benchmarks like CORe50 while mitigati...

  6. A Faster Path to Continual Learning

    cs.LG 2026-04 unverdicted novelty 5.0

    C-Flat Turbo accelerates continual learning by skipping redundant flatness gradients via direction-invariance observations and linear adaptive scheduling, delivering 1-1.25x speedup with comparable accuracy.

  7. OrthoPhys: Physically Plausible Video Generation with Orthogonal-View Geometry Guidance

    cs.CV 2026-03 unverdicted novelty 5.0

    Generating synchronized four-view orthogonal foreground videos with geometry-enhanced attention, then using them as rigid guidance, improves physical realism in video generation over direct 2D methods.

  8. Token Factory: Efficiently Integrating Diverse Signals into Large Recommendation Models

    cs.IR 2026-06 unverdicted novelty 4.0

    Token Factory transforms traditional signals into soft tokens for efficient integration and compression into Large Recommendation Models, avoiding prompt length explosion while enhancing performance.

  9. A Functional Data Framework For Analyzing Shapes and Textures in Images

    stat.ME 2026-06 unverdicted novelty 4.0

    Proposes a frugal functional representation for star-shaped image objects to analyze contours and textures, illustrated on supervised classification.

  10. Sparse Orthogonal Parameters Tuning for Continual Learning

    cs.LG 2024-11 unverdicted novelty 4.0

    SoTU merges sparse orthogonal delta parameters learned across streaming tasks to fuse knowledge and mitigate forgetting in pre-trained model continual learning.