RECORD
GACL: Exemplar-Free Generalized Analytic Continual Learning
Machine Learning · Submitted 23 Mar 2024
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Empirically, we conduct extensive experiments where, compared with existing GCIL methods, our GACL exhibits a consistently leading performance across various datasets and GCIL settings.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Class incremental learning (CIL) trains a network on sequential tasks with separated categories in each task but suffers from catastrophic forgetting, where models quickly lose previously learned knowledge when acquiring new tasks. The generalized CIL (GCIL) aims to address the CIL problem in a more real-world scenario, where incoming data have mixed data categories and unknown sample size distribution. Existing attempts for the GCIL either have poor performance or invade data privacy by saving exemplars. In this paper, we propose a new exemplar-free GCIL technique named generalized analytic continual learning (GACL). The GACL adopts analytic learning (a gradient-free training technique) and delivers an analytical (i.e., closed-form) solution to the GCIL scenario. This solution is derived via decomposing the incoming data into exposed and unexposed classes, thereby attaining a weight-invariant property, a rare yet valuable property supporting an equivalence between incremental learning and its joint training. Such an equivalence is crucial in GCIL settings as data distributions among different tasks no longer pose challenges to adopting our GACL. Theoretically, this equivalence property is validated through matrix analysis tools. Empirically, we conduct extensive experiments where, compared with existing GCIL methods, our GACL exhibits a consistently leading performance across various datasets and GCIL settings. Source code is available at https://github.com/CHEN-YIZHU/GACL.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Class incremental learning (CIL) trains a network on sequential tasks with separated categories in each task but suffers from catastrophic forgetting, where models quickly lose previously learned knowledge when acquiring new tasks. | background 1.00 | 0.06 | 0.02 | asserted 1.00 |
| s1 | The generalized CIL (GCIL) aims to address the CIL problem in a more real-world scenario, where incoming data have mixed data categories and unknown sample size distribution. | background 0.99 | 0.06 | 0.03 | asserted 1.00 |
| s2 | Existing attempts for the GCIL either have poor performance or invade data privacy by saving exemplars. | gap 0.89 | 0.04 | 0.03 | asserted 1.00 |
| s3 | In this paper, we propose a new exemplar-free GCIL technique named generalized analytic continual learning (GACL). | contribution 0.99 | 0.89 | 0.11 | asserted 1.00 |
| s4 | The GACL adopts analytic learning (a gradient-free training technique) and delivers an analytical (i.e., closed-form) solution to the GCIL scenario. | method 0.91 | 0.89 | 0.11 | asserted 1.00 |
| s5 | This solution is derived via decomposing the incoming data into exposed and unexposed classes, thereby attaining a weight-invariant property, a rare yet valuable property supporting an equivalence between incremental learning and its joint training. | method 0.90 | 0.95 | 0.19 | asserted 1.00 |
| s6 | Such an equivalence is crucial in GCIL settings as data distributions among different tasks no longer pose challenges to adopting our GACL. | interpretation 0.97 | 0.58 | 0.15 | asserted 1.00 |
| s7 | Theoretically, this equivalence property is validated through matrix analysis tools. | method 0.43 | 0.88 | 0.06 | asserted 1.00 |
| s8 | Empirically, we conduct extensive experiments where, compared with existing GCIL methods, our GACL exhibits a consistently leading performance across various datasets and GCIL settings. | result 1.00 | 0.97 | 0.71 | asserted 1.00 |
| s9 | Source code is available at https://github.com/CHEN-YIZHU/GACL. | contribution 0.63 | 0.34 | 0.02 | asserted 0.94 |
What was examined, and what was not
- arXiv listing abstract v3
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Class incremental learning (CIL) trains a network on sequential tasks with separated categories in each task but suffers from catastrophic forgetting, where models quickly lose previously learned knowledge when acquiring new tasks. The generalized CIL (GCIL) aims to address the CIL problem in a more real-world scenario, where incoming data have mixed data categories and unknown sample size distribution. Existing attempts for the GCIL either have poor performance or invade data privacy by saving exemplars. In this paper, we propose a new exemplar-free GCIL technique named generalized analytic continual learning (GACL). The GACL adopts analytic learning (a gradient-free training technique) and delivers an analytical (i.e., closed-form) solution to the GCIL scenario. This solution is derived via decomposing the incoming data into exposed and unexposed classes, thereby attaining a weight-invariant property, a rare yet valuable property supporting an equivalence between incremental learning and its joint training. Such an equivalence is crucial in GCIL settings as data distributions among different tasks no longer pose challenges to adopting our GACL. Theoretically, this equivalence property is validated through matrix analysis tools. Empirically, we conduct extensive experiments where, compared with existing GCIL methods, our GACL exhibits a consistently leading performance across various datasets and GCIL settings. Source code is available at https://github.com/CHEN-YIZHU/GACL.
Human review
References
How the literature uses it
3 arXiv papers cite this in 6 sentences
Papers citing it · 5 sentences without a confident label
Uses its method or tool 1 paper
-
For the existing ACL methods, we uses its generalized implementation introduced by [ref] .
Online Analytic Exemplar-Free Continual Learning with Large Models for Imbalanced Autonomous Driving Task · Result Comparison · 0.97
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.