Pith. sign in

REVIEW 2 major objections 5 minor 64 references

Forward-Only Continual Learning

T0 review · 2 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FoRo shows that continual learning with frozen pre-trained models can run entirely forward-only, using evolutionary prompt tuning and recursive least-squares classifier updates.

desk verdict A forward-only CL method with a correct RLS core, but the CIL claim is unreproducible because test-time prompt selection is never defined; worth a serious referee but needs major revision. read the letter →

arxiv 2509.01533 v1 pith:HNU24E6H submitted 2025-09-01 cs.LG cs.CV

Jiao Chen , Jiayi He , Fangfang Chen , Zuohong Lv , Jianhua Tang This is my paper
classification cs.LGcs.CV
keywords ContinualLearningCatastrophicForgettingPre-trainedModelsForward-onlyCMA-ESPromptTuningKnowledgeEncodingClass-Incremental
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FoRo is a continual-learning method that keeps a pre-trained vision transformer frozen and replaces backpropagation with two forward-only mechanisms: prompts tuned by the CMA-ES evolutionary strategy, and a classifier updated by recursive least squares through a knowledge encoding matrix. The paper aims to show that catastrophic forgetting can be controlled without any gradient computation, which matters for edge devices, real-time systems, and other resource-constrained deployments. Across CIFAR-100, ImageNet-R, and CUB-200, FoRo reports the best average accuracy (84.5%) and the lowest average forgetting (3.4%) among the compared methods, while also using less memory and run time than backpropagation-based approaches. The authors frame this as evidence that continual learning can be reframed: instead of managing interference during gradient updates, design a learning process that avoids such updates altogether.

What carries the argument

The load-bearing object is the knowledge encoding matrix R = (X^T X + γI)^−1, updated incrementally via the Woodbury identity as each task arrives. R compresses all accumulated feature statistics into a d×d matrix and turns classifier updates into a recursive least-squares step, so new classes can be added by extending classifier columns while R itself stays fixed. The second mechanism is CMA-ES prompt search, which treats prompt selection as black-box optimization over a fitness function containing cross-entropy plus an activation-statistics discrepancy term. Together these two mechanisms make every update forward-only: prompts are found by sampling and evaluation, and the classifier is upd

What would settle it

Run the ImageNet-R 25-task protocol with test samples stripped of task labels and a task-agnostic prompt-retrieval rule; if average accuracy drops far below the reported 76.7% (or the evaluation silently requires task identity), the central claim that FoRo works in class-incremental learning is not supported.

Watch

Extended reading notes

Core claim

The central claim is that class-incremental learning on top of a frozen pre-trained ViT can be solved forward-only by separating task adaptation from knowledge retention. FoRo optimizes input prompts with CMA-ES against a fitness function that combines cross-entropy with an activation-discrepancy regularizer, giving each task a good prompt without backpropagation. It then projects the CLS features through a fixed nonlinear random projection and maintains a knowledge encoding matrix R, the regularized inverse covariance of all features seen so far, updated online via the Woodbury identity. Classifier weights are updated by a recursive least-squares rule that the authors show is equivalent to

Load-bearing premise

Everything hinges on test samples being fed through the same task-specific prompt that was used when training that task's classifier, yet the paper does not specify how that prompt is selected when the incoming sample's task is unknown.

Editorial extensions

If this is right

  • Continual learning with frozen pre-trained models does not require backpropagation; a purely forward pipeline can match or exceed gradient-based prompt, prototype, and replay methods on accuracy and forgetting.
  • Because R is d×d and independent of the number of classes, new classes can be appended incrementally without resizing the accumulated knowledge matrix, which supports long task sequences without replaying old data.
  • Removing backpropagation changes the compute trade-off: more forward passes per sample are needed, but memory usage and wall-clock time drop, making the method more plausible for resource-constrained deployment.
  • The activation-discrepancy regularizer acts as a rehearsal-free knowledge-preservation mechanism by aligning layer statistics across tasks, complementing the explicit classifier-side retention provided by the knowledge encoding matrix.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The recursive least-squares update is a generic identity, so the classifier-side mechanism should transfer to any frozen fixed-dimensional feature extractor, including audio, text, or video backbones, not just ViT image features.
  • Test-time task identity is a live open question: the reported numbers assume each test sample is processed with the prompt tuned for its task, but the paper does not define a prompt-selection mechanism when the task label is unknown; this is the natural next experiment.
  • CMA-ES and the knowledge encoding matrix are independent components, so the prompt optimizer could be replaced by a cheaper zeroth-order or closed-form estimator without changing the classifier update, and vice versa.
  • The NRP dimension results show a broad performance plateau around 8,000, suggesting the large projection size may be reducible on tighter compute budgets without much accuracy loss.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes FoRo, a forward-only, gradient-free continual learning method for pre-trained vision transformers. FoRo combines two components: (i) prompt tuning via CMA-ES, where input-level prompts are optimized using a fitness function of cross-entropy plus an activation-statistics discrepancy regularizer, and (ii) a Knowledge Encoding Mechanism (KEM), which uses nonlinear random projection and recursive least squares (Woodbury identity) to incrementally update a linear classifier without replaying old data. Experiments on CIFAR-100, ImageNet-R, and CUB-200 under class-incremental protocols (T=5 and T=25) claim that FoRo achieves the highest average accuracy (84.5%) and lowest average forgetting (3.4%) among compared methods, while reducing memory and run time. Ablations and sensitivity studies support the contribution of each component.

Significance. If the reported results hold, FoRo would be a valuable addition to continual learning, particularly for resource-constrained multimedia applications, because it avoids backpropagation entirely. The KEM recursive least squares update (Eqs. 7-16) is a correct and clean application of the Woodbury identity, and the paper provides a useful empirical comparison including efficiency metrics. The CMA-ES prompt tuning is a reasonable black-box alternative to gradient-based prompt methods. However, the central empirical claim is currently not reproducible as stated because the class-incremental inference protocol is underspecified and the prompt-optimization objective is internally ambiguous. These issues are load-bearing, so the experimental evidence needs substantial clarification before the claims can be accepted.

major comments (2)
  1. [Algorithm 1 and §5.2 (Table 1)] The class-incremental evaluation lacks a test-time prompt selection rule. Algorithm 1 maintains one best prompt p_t^best per task and updates the shared classifier W in that task's feature space. At inference in CIL, task identity is unavailable; the paper never specifies how a test sample is assigned to one of the stored prompts. If oracle task IDs were used during evaluation, the setup is task-incremental, not class-incremental, and the accuracy/forgetting numbers in Table 1 are not comparable to the other CIL methods. If a selection mechanism exists (e.g., confidence-based or feature-similarity-based), it must be stated and validated. As written, the headline 84.5% average accuracy is not reproducible under the CIL protocol defined in §3.1.
  2. [§4.1, Eq. (5) and Algorithm 1] The fitness function's cross-entropy term (a) assumes that logits for the current task's labels are available. However, in Algorithm 1, the classifier W is updated only after the prompt has been selected (line 9), and W at that point contains only classes from previous tasks. No temporary classifier or head extension is described for computing \hat{y}_i^t during CMA-ES evaluation. Without specifying what classifier is used in Eq. (5), the prompt-optimization objective is undefined. This affects every experimental result, since prompt quality depends directly on this fitness.
minor comments (5)
  1. [§5.4, Table 3 vs. §5.3, Fig. 3(a)] The caption of Table 3 states that population size K works well in [2,6], while Fig. 3(a) says performance stabilizes when K>10. Please reconcile these statements and specify the default K used in the main experiments.
  2. [Eq. (5)] The inner summation over y_i^t ∈ Y_t is unconventional; cross-entropy is usually defined over one-hot vectors or a single label. Please clarify the notation and the exact form of the loss.
  3. [References] ADAM is cited as [53], but reference [53] is titled "Continual learning with pre-trained models: A survey." Please verify that the correct ADAM paper is cited.
  4. [§4.3] The claimed time complexity O(D_1^3 + D_2^2) for the KEM update is a simplification. Computing Eq. (11) involves the n_t × n_t inverse (I + X_t R X_t^T)^{-1}, so the per-task cost depends on n_t (e.g., O(d^2 n_t + d n_t^2)). This does not change the main message but should be stated more carefully.
  5. [§5.5.1] The sentence "their performance is expected to degrade even more" is speculative; either provide extrapolation or remove it.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity: KEM is a self-contained recursive least-squares derivation; self-citations are inspirational only.

full rationale

The paper's central derivation is the Knowledge Encoding Matrix (KEM) recursive update, Eqs. (9)-(16). This is a standard application of the Woodbury identity to the ridge-regression normal equations: R_t = (X_{1:t}^T X_{1:t} + gamma I)^{-1}, with W_t = W_{t-1} + R_t X_t^T(Y_t - X_t W_{t-1}); the algebra is shown in full and does not depend on any fitted quantity or on the paper's own outputs. The CMA-ES prompt tuning uses a fitness function (Eq. 5) defined on the current task's training data plus historical activation statistics; this is a training objective, not a prediction disguised as a result. The activation-discrepancy regularizer explicitly encourages low feature drift, so low forgetting is partly by design, but that is a method objective, not circularity. Self-citations (ACIL [50,55,56], etc.) motivate the analytic-update idea, but the derivation is self-contained; no uniqueness theorem or ansatz is imported from the authors' prior work. The one substantive concern is the unspecified test-time prompt selection in the class-incremental protocol: Algorithm 1 outputs per-task prompts p_t^best, but no inference rule is given for choosing a prompt for a test sample of unknown task. This is a reproducibility/correctness gap that could inflate accuracy if oracle task IDs are used, but it is not a circular reduction of a prediction to its inputs. Hence no circular steps; score 2 reflects only the minor non-load-bearing self-citation.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim relies on standard least-squares math, CMA-ES, and a pre-trained backbone. The knowledge encoding matrix is a name for a standard inverse scatter matrix, and historical statistics are summary statistics; no new physical or conceptual entities are postulated. The empirically chosen hyperparameters (gamma, alpha, lambda, NRP dimension, number of prompts, population size) are fitted on the evaluation benchmarks.

free parameters (6)
  • gamma (ridge regularization in KEM) = 0.1
    Set as the regularization factor in Eq. (7) and stated in implementation details; chosen empirically to balance performance and numerical stability.
  • alpha (moving average factor) = 0.1
    Used in Eq. (6) for updating historical statistics; set by hand and not rigorously tuned.
  • lambda (trade-off in fitness function) = 0.3
    Eq. (5); Table 4 shows the best performance on ImageNet-R with T=5 at lambda=0.3, and this value is implied as the default in main experiments.
  • NRP dimension = 8192
    Section 5.5.3: selected based on a sensitivity scan on CIFAR-100 with T=5, where 8000 gave the best accuracy (88.2%); 8192 is close to the peak.
  • Number of prompts = 3
    Section 5.3 (Fig. 3b): performance varies only slightly for 1-10 prompts; the paper fixes P=3 without fine-tuning.
  • CMA-ES population size = 6 (main results)
    Section 5.4 and Fig. 3(a): K=6 is the largest tested value used in the final results, chosen for accuracy; the paper notes K in [2,6] works well.
assumptions (5)
  • standard math The Woodbury identity and the closed-form regularized least squares solution (Eqs. 8-11) are correct and applicable.
    Section 4.2 relies on these standard matrix identities to justify the recursive update without approximation.
  • domain assumption Pre-trained ViT-B/16 embeddings provide a feature space in which linear classifiers and random projections are effective.
    This is the basis of the entire method, used throughout Section 5; it is a common assumption in this literature but not proven in the paper.
  • domain assumption The activation discrepancy regularizer (Eq. 5b) aligns representation statistics well enough to reduce forgetting.
    Section 4.1 proposes this regularizer; its efficacy is only demonstrated empirically and depends on the choice of lambda and alpha.
  • ad hoc to paper At test time the correct task-specific prompt can be identified or a single prompt works for all tasks.
    This is implicitly assumed by the evaluation but never stated or validated; it underpins the alignment between features and the recursively updated classifier.
  • domain assumption The nonlinear random projection with unspecified activation phi and random matrix W_rp preserves or improves class separability.
    Section 4.2 and Section 5.5.3 assume this, borrowing from RanPAC, but the specific choice of activation function and projection distribution is not provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Forward-Only Continual Learning." pith.science (2026). https://pith.science/paper/HNU24E6H

@misc{pith2026250901533,
  author       = {Pith},
  title        = {Pith review of: Forward-Only Continual Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HNU24E6H}},
  note         = {Machine review of arXiv:2509.01533}
}
read the original abstract

Catastrophic forgetting remains a central challenge in continual learning (CL) with pre-trained models. While existing approaches typically freeze the backbone and fine-tune a small number of parameters to mitigate forgetting, they still rely on iterative error backpropagation and gradient-based optimization, which can be computationally intensive and less suitable for resource-constrained environments. To address this, we propose FoRo, a forward-only, gradient-free continual learning method. FoRo consists of a lightweight prompt tuning strategy and a novel knowledge encoding mechanism, both designed without modifying the pre-trained model. Specifically, prompt embeddings are inserted at the input layer and optimized using the Covariance Matrix Adaptation Evolution Strategy (CMA-ES), which mitigates distribution shifts and extracts high-quality task representations. Subsequently, task-specific knowledge is encoded into a knowledge encoding matrix via nonlinear random projection and recursive least squares, enabling incremental updates to the classifier without revisiting prior data. Experiments show that FoRo significantly reduces average forgetting and improves accuracy. Thanks to forward-only learning, FoRo reduces memory usage and run time while maintaining high knowledge retention across long task sequences. These results suggest that FoRo could serve as a promising direction for exploring continual learning with pre-trained models, especially in real-world multimedia applications where both efficiency and effectiveness are critical.

Figures

Figures reproduced from arXiv: 2509.01533 by the authors.

Figure 1
Figure 1. FoRo employs a prompt optimization strategy (i.e., CMA-ES) and a knowledge encoding mechanism to achieve [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Knowledge Encoding. In CL, the current task knowledge is encoded into the KEM and then updates the weight of [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Parameter sensitivity analyses of our FoRo. Experiments are conducted on CIFAR-100 with [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Comparison of different task counts 𝑇 [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

64 extracted references · 54 canonical work pages

  1. [1]

    Kyra Ahrens, Hans Hergen Lehmann, Jae Hee Lee, and Stefan Wermter. 2023. Read Between the Layers: Leveraging Intra-Layer Representations for Rehearsal- Free Continual Learning with Pre-Trained Models. arXiv:2312.08888 (2023)

    arXiv 2023

  2. [2]

    Atılım Güneş Baydin, Barak A Pearlmutter, Don Syme, Frank Wood, and Philip Torr. 2022. Gradients without backpropagation. arXiv:2202.08587 (2022)

    arXiv 2022

  3. [3]

    Ondrej Bohdal, Da Li, Shell Xu Hu, and Timothy Hospedales. 2024. Feed-forward latent domain adaptation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 8490–8499

    work page 2024

  4. [4]

    Xusheng Cao, Haori Lu, Linlan Huang, Xialei Liu, and Ming-Ming Cheng. 2024. Generative Multi-modal Models are Good Class Incremental Learners. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 28706–28717

    work page 2024

  5. [5]

    Guohao Chen, Shuaicheng Niu, Deyu Chen, Shuhai Zhang, Changsheng Li, Yuan- qing Li, and Mingkui Tan. 2024. Cross-Device Collaborative Test-Time Adaptation. In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    work page 2024

  6. [6]

    Jiao Chen, Jiayi He, Jianhua Tang, Weihua Li, and Zihang Yin. 2025. Knowledge Efficient Federated Continual Learning for Industrial Edge Systems. IEEE Trans- actions on Network Science and Engineering 12, 3 (May-Jun. 2025), 2107–2120

    work page 2025

  7. [7]

    Jiao Chen, Jianhua Tang, and Weihua Li. 2023. Industrial Edge Intelligence: Federated-Meta Learning Framework for Few-Shot Fault Diagnosis. IEEE Trans. Network Sci. Eng. 10, 6 (Nov. 2023), 3561–3573. doi:10.1109/TNSE.2023.3266942

    work page arXiv 2023

  8. [8]

    Andrea Cossu, Antonio Carta, Lucia Passaro, Vincenzo Lomonaco, Tinne Tuyte- laars, and Davide Bacciu. 2024. Continual pre-training mitigates forgetting in language and vision. Neural Networks 179 (2024), 106492

    work page 2024

Show all 64 references
  1. [9]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In Proc. ICL...

    2020

  2. [10]

    Nikolaus Hansen. 2016. The CMA evolution strategy: A tutorial.arXiv:1604.00772 (2016)

    2016 arXiv

  3. [11]

    Jiayi He, Jiao Chen, Qianmiao Liu, Suyan Dai, Jianhua Tang, and Dongpo Liu

  4. [12]

    Fung, and Heng Ji

    Jiayi He, Hehai Lin, Qingyun Wang, Yi R. Fung, and Heng Ji. 2025. Self-Correction is More than Refinement: A Learning Framework for Visual and Language Rea- soning Tasks. In Findings of the Association for Computational Linguistics: ACL 2025, Wanxiang Che, Joyce Nabende, Ekate...

    2025

  5. [13]

    Geoffrey Hinton. 2022. The forward-forward algorithm: Some preliminary inves- tigations. arXiv preprint arXiv:2212.13345 2, 3 (2022), 5

    2022 arXiv

  6. [14]

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. 2022. Visual prompt tuning. In Euro- pean Conference on Computer Vision . Springer, 709–727

    2022

  7. [15]

    Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. 2020. How can we know what language models know? Transactions of the Association for Computa- tional Linguistics 8 (2020), 423–438

    2020

  8. [16]

    Zhiqi Kang, Liyuan Wang, Xingxing Zhang, and Karteek Alahari. 2025. Advancing Prompt-Based Methods for Replay-Independent General Continual Learning. In International Conference on Learning Representations (ICLR)

    2025

  9. [17]

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017. Overcoming catastrophic forgetting in neural networks. In Proceedings of the national academy...

    2017

  10. [18]

    Muhammad Rifki Kurniawan, Xiang Song, Zhiheng Ma, Yuhang He, Yihong Gong, Yang Qi, and Xing Wei. 2024. Evolving Parameterized Prompt Memory for Continual Learning. InProceedings of the AAAI Conference on Artificial Intelligence. 13301–13309

    2024

  11. [19]

    Jaewoo Lee, Jaehong Yoon, Wonjae Kim, Yunji Kim, and Sung Ju Hwang. 2024. STELLA: Continual Audio-Video Pre-training with SpatioTemporal Localized Alignment. In Forty-first International Conference on Machine Learning

    2024

  12. [20]

    Siyuan Li, Hao Li, Jin Zhang, Zhen Wang, Peng Liu, and Chongjie Zhang. 2023. IOB: integrating optimization transfer and behavior transfer for multi-policy reuse. Autonomous Agents and Multi-Agent Systems 38, 1 (Dec. 2023), 3

    2023

  13. [21]

    Zhizhong Li and Derek Hoiem. 2017. Learning without forgetting. IEEE transac- tions on pattern analysis and machine intelligence 40, 12 (2017), 2935–2947. MM ’25, October 27–31, 2025, Dublin, Ireland Jiao Chen, Jiayi He, Fangfang Chen, Zuohong Lv, and Jianhua Tang

    2017

  14. [22]

    Xialei Liu, Jiang-Tian Zhai, Andrew D Bagdanov, Ke Li, and Ming-Ming Cheng

  15. [23]

    Vincenzo Lomonaco, Lorenzo Pellegrini, Andrea Cossu, Antonio Carta, Gabriele Graffieti, Tyler L Hayes, Matthias De Lange, Marc Masana, Jary Pomponi, Gido M Van de Ven, et al. 2021. Avalanche: an end-to-end library for continual learning. In Proceedings of the IEEE/CVF Conferen...

    2021

  16. [24]

    Wan-Duo Kurt Ma, JP Lewis, and W Bastiaan Kleijn. 2020. The HSIC bottleneck: Deep learning without back-propagation. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 5085–5092

    2020

  17. [25]

    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Task-Adaptive Saliency Guidance for Exemplar-free Class Incremental Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 23954–23963

  18. [26]

    Arun Mallya and Svetlana Lazebnik. 2018. Packnet: Adding multiple tasks to a single network by iterative pruning. In Proc. CVPR. Salt Lake City, UT, USA, 7765–7773

    2018

  19. [27]

    Daniel Marczak, Bartłomiej Twardowski, Tomasz Trzciński, and Sebastian Cygert

  20. [28]

    Sadhika Malladi, Tianyu Gao, Eshaan Nichani, Alex Damian, Jason D Lee, Danqi Chen, and Sanjeev Arora. 2023. Fine-tuning language models with just forward passes. Advances in Neural Information Processing Systems36 (2023), 53038–53075

    2023

  21. [29]

    Seyed Iman Mirzadeh, Mehrdad Farajtabar, Razvan Pascanu, and Hassan Ghasemzadeh. 2020. Understanding the role of training regimes in continual learning. In Proc. NeurIPS, Vol. 33. Virtual, 7308–7320

    2020

  22. [30]

    Shuaicheng Niu, Chunyan Miao, Guohao Chen, Pengcheng Wu, and Peilin Zhao

  23. [31]

    Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. 2022. Efficient test-time model adaptation without forgetting. In International conference on machine learning . PMLR, 16888–16905

    2022

  24. [32]

    Mark D McDonnell, Dong Gong, Amin Parvaneh, Ehsan Abbasnejad, and Anton van den Hengel. 2024. Ranpac: Random projections and pre-trained models for continual learning. Advances in Neural Information Processing Systems 36 (2024)

    2024

  25. [33]

    Tingyu Qu, Tinne Tuytelaars, and Marie-Francine Moens. 2025. Introducing Routing Functions to Vision-Language Parameter-Efficient Fine-Tuning with Low-Rank Bottlenecks. In European Conference on Computer Vision . Springer, 291–308

    2025

  26. [34]

    Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. 2017. icarl: Incremental classifier and representation learning. In Proc. CVPR. Honolulu, HI, USA, 2001–2010

    2017

  27. [35]

    In International Conference on Machine Learning

    Test-time model adaptation with only forward passes. In International Conference on Machine Learning . 38298–38315

  28. [36]

    Hai-Long Sun, Da-Wei Zhou, Hanbin Zhao, Le Gan, De-Chuan Zhan, and Han-Jia Ye. 2024. MOS: Model Surgery for Pre-Trained Model-Based Class-Incremental Learning. arXiv preprint arXiv:2412.09441 (2024)

    2024 arXiv

  29. [37]

    Yongchun Qin, Pengfei Fang, and Hui Xue. 2024. PEARL: Input-Agnostic Prompt Enhancement with Negative Feedback Regulation for Class-Incremental Learning. arXiv preprint arXiv:2412.10900 (2024)

    2024 arXiv

  30. [38]

    Jianhua Tang, Fangfang Chen, Jiaping Li, and Zilong Liu. 2025. Learn to Schedule: Data Freshness-Oriented Intelligent Scheduling in Industrial IoT. IEEE Transac- tions on Cognitive Communications and Networking 11, 1 (Feb. 2025), 505–518

    2025

  31. [39]

    Yang, and Weihua Li

    Jianhua Tang, Jiao Chen, Jiayi He, Fangfang Chen, Zuohong Lv, Guangjie Han, Zuozhu Liu, Howard H. Yang, and Weihua Li. 2025. Towards General Industrial Intelligence: A Survey of Large Models as a Service in Industrial IoT. IEEE Communications Surveys & Tutorials (2025), 1–1. e...

    2025

  32. [40]

    Hai-Long Sun, Da-Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. 2023. Pilot: A pre-trained model-based continual learning toolbox. arXiv:2309.07117 (2023)

    2023 arXiv

  33. [41]

    Paul Vicol. 2023. Low-variance gradient estimation in unrolled computation graphs with es-single. In International Conference on Machine Learning . PMLR, 35084–35119

    2023

  34. [42]

    Tianxiang Sun, Yunfan Shao, Hong Qian, Xuanjing Huang, and Xipeng Qiu. 2022. Black-box tuning for language-model-as-a-service. In International Conference on Machine Learning. PMLR, Virtual Only, 20841–20855

    2022

  35. [43]

    Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. 2024. A comprehensive survey of continual learning: theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)

    2024

  36. [44]

    Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, et al. 2022. Dualprompt: Complementary prompting for rehearsal-free continual learning. In European conference on computer vision . Springer, 631–648

    2022

  37. [45]

    Jianhua Tang and Yao Zeng. 2025. UAV Data Acquisition and Processing As- sisted by UGV-Enabled Mobile Edge Computing. IEEE Transactions on Industrial Informatics 21, 5 (May 2025), 3695–3704

    2025

  38. [46]

    Shipeng Yan, Jiangwei Xie, and Xuming He. 2021. Der: Dynamically expandable representation for class incremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 3014–3023

    2021

  39. [47]

    Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie

  40. [48]

    Xianghu Yue, Xueyi Zhang, Yiming Chen, Chengwei Zhang, Mingrui Lao, Huip- ing Zhuang, Xinyuan Qian, and Haizhou Li. 2024. MMAL: Multi-Modal Analytic Learning for Exemplar-Free Audio-Visual Class Incremental Tasks. InProceed- ings of the 32nd ACM International Conference on Mul...

    2024

  41. [49]

    Gengwei Zhang, Liyuan Wang, Guoliang Kang, Ling Chen, and Yunchao Wei

  42. [50]

    Xiang Zhang, Run He, Jiao Chen, Di Fang, Ming Li, Ziqian Zeng, Cen Chen, and Huiping Zhuang. 2025. L3A: Label-Augmented Analytic Adaptation for Multi-Label Class Incremental Learning. In Forty-second International Conference on Machine Learning. https://openreview.net/forum?id...

    2025

  43. [51]

    Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. 2022. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . New Orle...

    2022

  44. [52]

    Linglan Zhao, Xuerui Zhang, Ke Yan, Shouhong Ding, and Weiran Huang. 2024. SAFE: Slow and Fast Parameter-Efficient Tuning for Continual Learning with Pre- Trained Models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    2024

  45. [53]

    Çağatay Yıldız, Nishaanth Kanna Ravichandran, Prishruit Punia, Matthias Bethge, and Beyza Ermis. 2024. Investigating Continual Pretraining in Large Language Models: Insights and Implications. arXiv:2402.17400 (2024)

    2024 arXiv

  46. [54]

    Huiping Zhuang, Run He, Kai Tong, Ziqian Zeng, Cen Chen, and Zhiping Lin

  47. [55]

    Huiping Zhuang, Zhenyu Weng, Run He, Zhiping Lin, and Ziqian Zeng. 2023. GKEAL: Gaussian Kernel Embedded Analytic Learning for Few-Shot Class Incre- mental Task. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 7746–7755. doi:10.1109...

    2023

  48. [56]

    Huiping Zhuang, Zhenyu Weng, Hongxin Wei, Renchunzi Xie, Kar-Ann Toh, and Zhiping Lin. 2022. ACIL: Analytic class-incremental learning with absolute memorization and privacy protection. Advances in Neural Information Processing Systems 35 (Dec. 2022), 11602–11614

    2022

  49. [58]

    Hongbo Zhao, Bolin Ni, Junsong Fan, Yuxi Wang, Yuntao Chen, Gaofeng Meng, and Zhaoxiang Zhang. 2024. Continual forgetting for pre-trained vision mod- els. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 28631–28642

    2024

  50. [60]

    Da-Wei Zhou, Hai-Long Sun, Jingyi Ning, Han-Jia Ye, and De-Chuan Zhan. 2024. Continual learning with pre-trained models: A survey. arXiv:2401.16386 (2024)

    2024 arXiv

  51. [62]

    In Proceedings of the AAAI Conference on Artificial Intelligence

    DS-AL: A dual-stream analytic learning for exemplar-free class-incremental learning. In Proceedings of the AAAI Conference on Artificial Intelligence . 17237– 17244

  52. [2011]

    Technical report (2011)

    The caltech-ucsd birds-200-2011 dataset. Technical report (2011)

    2011

  53. [2023]

    In Proceedings of the IEEE/CVF International Conference on Computer Vision

    Slca: Slow learner with classifier alignment for continual learning on a pre-trained model. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 19148–19158

  54. [2024]

    In 2024 IEEE/CIC International Conference on Communications in China (ICCC)

    Continual Learning with Diffusion-based Generative Replay for Industrial Streaming Data. In 2024 IEEE/CIC International Conference on Communications in China (ICCC). Hangzhou, China, 1015–1020

    2024

  55. [2025]

    In European Conference on Computer Vision

    Magmax: Leveraging model merging for seamless continual learning. In European Conference on Computer Vision . Springer, 379–395

  56. [6421]

    https://aclanthology.org/2025.findings-acl.331/

    2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.