Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

Adapter Merging with Centroid Prototype Mapping for Scalable Class-Incremental Learning

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ACMap claims to break the accuracy–inference trade-off in exemplar-free class-incremental learning by merging per-task adapters into a single shared adapter and re-aligning old prototypes with one centroid shift measured on the current…

desk verdict Useful, honest adapter-merging CIL paper with real O(1) inference gains; the abstract overclaims SOTA parity because VTAB clearly breaks the centroid-mapping assumption. read the letter →

arxiv 2412.18219 v2 pith:6J6D2H4G submitted 2024-12-24 cs.CV

Takuma Fukuda , Hiroshi Kera , Kazuhiko Kawamoto This is my paper · ORCID
classification cs.CV
keywords class-incrementallearningexemplar-freecontinualadaptermergingcentroidprototypemappingcatastrophicforgettingparameter-efficientfine-tuningvisiontransformercosineclassifier
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Class-incremental learning asks a model to keep learning new classes without forgetting old ones, and privacy rules often forbid keeping old images. Existing exemplar-free methods built on pre-trained models either retrain a fresh adapter per task, which is accurate but slow at inference because every task's adapter must run, or reuse a single adapter, which is fast but less accurate. This paper claims a middle path: train one adapter per task but merge them by simple weight averaging into a single adapter, so inference costs stay constant no matter how many tasks have arrived. To keep old class prototypes comparable inside the merged space, the method shifts them by a single translation vector estimated from the current task's class centroids. On five benchmarks the method matches the accuracy of the strongest adapter-ensemble baseline while running at the speed of the fastest prototype baselines, for example a 39-fold inference speedup on the 40-task ImageNet-R setting.

What carries the argument

The central objects are the task-specific adapters $A_t$, the merged adapter $\bar{A}_t = (1 - 1/t)\bar{A}_{t-1} + (1/t)A_t$, and the centroid prototype mapping that approximates unavailable old prototypes by $P_i(\bar{A}_t) \approx P_i(\bar{A}_i) + \Delta P$, where $\Delta P$ is the mean difference between the current task's prototypes computed in $\bar{A}_t$ and in $\bar{A}_i$. The first task's trained weights replace the random initialization for all later adapters, creating a shared low-loss basin that makes weight averaging safe. Early stopping sets a threshold $L$ after which no new adapter is trained, so the merged adapter stops changing. These pieces together reduce inference from $O(T)$ to $O(1)$ while keeping the classifier a cosine classifier over re-aligned prototypes.

What would settle it

A decisive check is to measure the per-task drift on a benchmark with heterogeneous domains and balanced task sizes: compute $\Delta P_i = P_i(\bar{A}_t) - P_i(\bar{A}_i)$ for each old task $i$ and compare the spread of these shifts to the single centroid shift $\Delta P$. If the spread is comparable to inter-class prototype distances, the re-aligned prototypes will point at wrong classes and final accuracy should fall, as the paper's own VTAB B0 Inc10 result (Section 5.2, Table 1) hints by dropping after the fourth task.

Watch

Extended reading notes

Core claim

The paper's central claim is that catastrophic forgetting and scalability in exemplar-free class-incremental learning can be handled together by consolidating task-specific adapters into one merged adapter. The consolidation is average merging of adapter weights, made effective by initializing every task's adapter from the first task's trained weights so that the tasks occupy a shared low-loss basin. Because old data is unavailable, previous prototypes cannot be recomputed in the merged space; the paper claims that the drift of the current task's prototypes between an old adapter and the merged adapter is a good proxy for the drift of every earlier task's prototypes, so a single affine shift $\Delta P = E[P_t(\bar{A}_t) - P_t(\bar{A}_i)]$ re-aligns all old prototypes. With this mapping, ACMap reports final-accuracy parity with the EASE ensemble (70.49 versus 70.58 on ImageNet-R task 40) while reducing inference time from 916.5 seconds to 23.5 seconds. The paper also claims that stopping the merging after about ten tasks saves training computation with essentially no accuracy loss.

Load-bearing premise

The whole method depends on the assumption that the shift between the current task's class prototypes computed in an old adapter and in the merged adapter is about the same for every earlier task, so one average shift can re-align all stored prototypes; if older tasks drift in different directions, those prototypes point to the wrong places.

Editorial extensions

If this is right

  • Inference time stays constant as tasks grow: on ImageNet-R B0 Inc5 task 40, ACMap runs in 23.5 seconds versus 916.5 seconds for the strongest ensemble baseline, a 39-fold speedup, with final accuracy 70.49 versus 70.58.
  • On five benchmarks, ACMap matches or slightly exceeds the state-of-the-art adapter-ensemble accuracy on all settings except VTAB B0 Inc10, where the ensemble baseline keeps separate adapters per domain.
  • Early stopping at $L=10$ achieves accuracy nearly identical to $L=\infty$ on CIFAR-100 and ImageNet-R, so training cost can be cut without hurting accuracy.
  • Both initial-weight replacement and centroid prototype mapping contribute: the ablation shows each raises accuracy, and their combination gives the best results.
  • Because only one merged adapter is used at inference, the method stays exemplar-free and privacy-compatible.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: if the drift of prototypes is roughly task-invariant, the same centroid-shift trick could be applied to other parameter-efficient modules, such as prompt pools, to obtain constant inference time there too.
  • Editorial extension: the dependence of the shift on task order suggests a testable refinement — instead of a single global shift, use a shift estimated from the most recent tasks and interpolate for older tasks, which might recover the VTAB gap while keeping $O(1)$ inference.
  • Editorial extension: the early-stopping result implies that after roughly ten tasks the merged adapter has stabilized, so the method could be extended to never-ending streams by freezing the merged adapter and training new adapters only when a drift signal, such as prototype cosine similarity, exceeds a threshold.
  • Editorial extension: the paper does not report total stored-parameter counts against ensemble methods; a direct comparison would clarify the memory advantage of storing one merged adapter rather than $T$ adapters.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes ACMap, an exemplar-free class-incremental learning method that trains a task-specific adapter for each task, incrementally averages their weights into a single merged adapter, and uses a centroid prototype mapping to align previously computed prototypes to the current merged-adapter subspace. The claimed benefits are constant (O(1)) inference time independent of the number of tasks, competitive accuracy with state-of-the-art methods such as EASE, and scalability via an early-stopping threshold. The method is evaluated on five benchmarks (CIFAR-100, CUB, ImageNet-R, ImageNet-A, VTAB) under the exemplar-free protocol, with comparisons to prompt- and adapter-based baselines. The central empirical claim is that ACMap matches EASE's accuracy on most benchmarks while being roughly 39x faster at inference on 40-task ImageNet-R.

Significance. If the results hold, ACMap is a valuable contribution to the continual learning literature: it directly addresses the accuracy-inference-time trade-off in pre-trained-model-based CIL, achieves strong performance with a simple and reproducible mechanism, and provides code. The paper's strengths include clean ablation studies (Table 3 and Table 4), a clear complexity argument for O(1) inference, and a thorough set of experiments across diverse datasets. The centroid prototype mapping is an original idea, and the analysis of the loss landscape (Figure 3 and Appendix G) provides intuitive support for why weight averaging works in this setting. The method is also genuinely exemplar-free, which is important for privacy-sensitive applications. However, the significance is moderated by the paper's own results showing a substantial accuracy drop on VTAB, and by the absence of error bars, which makes it difficult to assess whether the small observed differences from EASE are meaningful.

major comments (3)
  1. [Abstract and Section 5.2, Table 1] The abstract states that ACMap 'matches state-of-the-art accuracy,' but the results in Table 1 do not support an unqualified claim. On VTAB B0 Inc10, ACMap achieves an average accuracy of 91.21 and final accuracy of 87.56, versus EASE's 93.61 and 93.55, a gap of 2.4 and 6.0 percentage points respectively. On CUB B0 Inc10, IN-R B0 Inc5, and IN-A B0 Inc20, ACMap is also slightly below EASE in final accuracy. The only benchmark where ACMap clearly exceeds EASE on both metrics is CIFAR-100 B0 Inc5. The abstract and introduction should qualify the claim, for example by stating that ACMap matches state-of-the-art accuracy on most benchmarks while providing a large inference-speed advantage, and the VTAB exception should be explicitly acknowledged in the abstract.
  2. [Section 4.2, Eqs. (6)-(7), and Table 1] The centroid prototype mapping assumes that a single shift vector ∆p, estimated from current-task prototypes computed in the current and an older adapter, applies unchanged to all previous tasks' prototypes. This assumption is load-bearing for the method's accuracy, but the paper's validation in Figure 6 and Appendix H relies on computing true prototypes with previous-task validation data, which is not available in the exemplar-free CIL setting. The VTAB result in Table 1 is a concrete failure case: the mapping does not adequately correct the prototypes, leading to a large accuracy drop relative to EASE. While the authors attribute this to data imbalance and show parity on a balanced VTAB variant (Appendix E.2), the standard imbalanced protocol is the one used in the main comparison, and the robustness of the mapping to realistic imbalances is therefore not established. I recommend that the authors either (a) restrict the central claim to settings where the task distribution is reasonably balanced, or (b) provide additional evidence—beyond cosine similarity on held-out validation data—that the shift vector is stable across tasks, such as measuring the variance of ∆p across different current tasks.
  3. [Section 5.1 and Table 1] All ACMap results are reported as averages over five runs without standard deviations, confidence intervals, or significance tests. Since the method's accuracy is within 0.1–1.5 percentage points of EASE on several benchmarks (e.g., IN-R final accuracy 70.49 vs. 70.58, IN-A final accuracy 56.19 vs. 55.04), the reader cannot judge whether these differences are statistically meaningful. The claim of 'comparable' or 'slightly better' performance should be supported by error bars or at least by a statement of variance across seeds. This is a standard expectation for empirical papers and is necessary to assess the robustness of the headline result.
minor comments (6)
  1. [Section 5.1] Typo: 'We follow the the protocol in [59]' should be 'We follow the protocol in [59]'.
  2. [Section 3.1] Typo: 'In examplar-based CIL' should be 'In exemplar-based CIL'.
  3. [Section 2] In the paragraph beginning 'CIL with model merging:', there is a missing space before the colon: 'CIL with model merging:Several' should read 'CIL with model merging: Several'.
  4. [Figure 4 caption] The caption 'The curve showing the differences in cosine similarity that arise when earlier task prototypes are substituted for prototypes in subsequent subspaces' is grammatically awkward; consider rewording to 'Cosine similarity between the true prototypes in the current subspace and the substitutes used when earlier task prototypes are reused.'
  5. [Section 4.2, Algorithm 1] The notation ∆P in Eq. (7) and in Algorithm 1 is defined as a matrix formed by repeating the vector ∆p, but Eq. (6) writes Pi(¯At) ≈ Pi(¯Ai) + ∆P, which is ambiguous because Pi(¯Ai) is a matrix while ∆p is a vector. Clarify the broadcasting convention, for instance by writing Pi(¯At) ≈ Pi(¯Ai) + 1⊤⊗∆p.
  6. [Appendix E.2] The balanced-VTAB experiment is described in the text but the figure caption (Figure F) does not state the number of runs or whether the same seeds are used; please specify the experimental setup for this ablation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: centroid prototype mapping is a forward heuristic validated offline, not a fit to the targets it predicts.

full rationale

The paper's derivation chain is self-contained. The only candidate circular step is the centroid prototype mapping in Eqs. (6)-(7), where the shift Delta P is estimated from available current-task prototypes and applied to older prototypes. This is a heuristic assumption, not a fit to the result being predicted: the target prototypes P_i(Abar_t) are never used to estimate Delta P, and the mapping is not tuned on previous-task validation data. The offline validation in Figures 6 and H uses previous-task validation data only to measure how well the fixed mapping aligns prototypes; those true prototypes do not enter the algorithm or any fitted parameter. The early-stopping threshold L is chosen from validation cosine similarity (Appendix D), but the ablation in Table 4 shows L=10 is nearly indistinguishable from L=infinity, so no accuracy claim is forced by that choice. Baselines are taken from external prior work [59] by different authors, so there is no load-bearing self-citation. The abstract's unqualified 'matches SOTA' claim is weakened by the VTAB result (91.21 vs EASE 93.61 average accuracy), but that is an empirical limitation of the shift-invariance assumption, not a circular argument.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The method introduces no new entities; it relies on two main domain assumptions: the transferability of centroid drift across tasks and the existence of a low-loss basin under adapter averaging. The single free parameter, early stopping L, is shown by ablation to be non-critical.

free parameters (1)
  • Early stopping threshold L = 10
    Selected from the cosine similarity curves of Sim(P_t(A_{t-1}), P_t(A_t)) in Appendix D; Table 4 shows L=10 is near L=infinity, so the choice is not decisive for accuracy.
assumptions (3)
  • domain assumption The current task's centroid drift between old and new adapter subspaces transfers to all previous tasks (Eq. 6-7).
    Centroid prototype mapping in Section 4.2 relies on this; the paper validates it only via cosine similarity measured on validation data from previous tasks, unavailable in actual CIL use.
  • domain assumption Weight averaging of adapters initialized from shared weights yields a low-loss basin that supports merging (Section 4.1, Figures 3 and I).
    The loss landscape analysis is empirical and shows the basin for most datasets but not for VTAB.
  • standard math Prototypes computed on current-task validation data are sufficient to form a cosine classifier for that task (Eq. 2-4).
    This is the standard prototype-classifier assumption inherited from SimpleCIL and EASE.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adapter Merging with Centroid Prototype Mapping for Scalable Class-Incremental Learning." pith.science (2026). https://pith.science/paper/6J6D2H4G

@misc{pith2026241218219,
  author       = {Pith},
  title        = {Pith review of: Adapter Merging with Centroid Prototype Mapping for Scalable Class-Incremental Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6J6D2H4G}},
  note         = {Machine review of arXiv:2412.18219}
}
read the original abstract

We propose Adapter Merging with Centroid Prototype Mapping (ACMap), an exemplar-free framework for class-incremental learning (CIL) that addresses both catastrophic forgetting and scalability. While existing methods involve a trade-off between inference time and accuracy, ACMap consolidates task-specific adapters into a single adapter, thus achieving constant inference time across tasks without sacrificing accuracy. The framework employs adapter merging to build a shared subspace that aligns task representations and mitigates forgetting, while centroid prototype mapping maintains high accuracy by consistently adapting representations within the shared subspace. To further improve scalability, an early stopping strategy limits adapter merging as tasks increase. Extensive experiments on five benchmark datasets demonstrate that ACMap matches state-of-the-art accuracy while maintaining inference time comparable to the fastest existing methods. The code is available at https://github.com/tf63/ACMap.

Figures

Figures reproduced from arXiv: 2412.18219 by the authors.

Figure 1
Figure 1. Comparison of the final top-1 accuracy and inference [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An Illustration of ACMap. ACMap sequentially trains an adapter for each task, starting from shared initial weights and incremen [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Visualization of the test error using linearly interpolated [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The curve showing the differences in cosine similarity that arise when earlier task prototypes are substituted for prototypes in [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Top-1 accuracy curve during CIL, comparing prototype-based methods: SimpleCIL (denoted as Simple), APER, and EASE. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Cosine similarity curves of Sim(Pˆ1(A¯1), P1(A¯t)), with solid lines showing the similarity between mapped and true proto￾types, and semi-transparent lines between unmapped and true pro￾totypes. The prototypes aligned through centroid prototype map￾ping move closer to …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SAFE-Merge: Data-Free Continual Model Merging with General Knowledge Preservation

    cs.LG 2026-08 conditional novelty 6.0 of 10

    SAFE-Merge masks risk-prone parameter updates and recovers lost task information with a constrained low-rank correction, achieving the best H-score in data-free continual model merging benchmarks.

  2. Continual Knowledge Consolidation LORA for Domain Incremental Learning

    cs.LG 2025-10 conditional novelty 5.0 of 10

    CONEC-LoRA reports state-of-the-art accuracy on four domain-incremental benchmarks by combining task-shared and task-specific LoRAs with a stochastic classifier and a learned domain-ID selector.

Reference graph

Works this paper leans on

62 extracted references · 60 canonical work pages · cited by 2 Pith papers

  1. [1]

    Git Re-Basin: Merging Models modulo Permutation Symmetries

    Samuel Ainsworth, Jonathan Hayase, and Siddhartha Srini- vasa. Git Re-Basin: Merging Models modulo Permutation Symmetries. In ICLR, 2023. 5

    work page 2023

  2. [2]

    Expert Gate: Lifelong Learning with a Network of Experts

    Rahaf Aljundi, Punarjay Chakravarty, and Tinne Tuytelaars. Expert Gate: Lifelong Learning with a Network of Experts. In CVPR, pages 7120–7129, 2017. 2

    work page 2017

  3. [3]

    Gradient based sample selection for online continual learning

    Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Ben- gio. Gradient based sample selection for online continual learning. In NeurIPS, pages 11816–11825, 2019. 1, 2

    work page 2019

  4. [4]

    Efficient Lifelong Learning with A-GEM

    Arslan Chaudhry, Marc’Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient Lifelong Learning with A-GEM. In ICLR, 2019. 2

    work page 2019

  5. [5]

    AdaptFormer: Adapting Vision Transformers for Scalable Visual Recogni- tion

    Shoufa Chen, Chongjian GE, Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang, and Ping Luo. AdaptFormer: Adapting Vision Transformers for Scalable Visual Recogni- tion. In NeurIPS, pages 16664–16678, 2022. 7

    work page 2022

  6. [6]

    Learning Without Mem- orizing

    Prithviraj Dhar, Rajat Vikram Singh, Kuan-Chuan Peng, Ziyan Wu, and Rama Chellappa. Learning Without Mem- orizing. In CVPR, pages 5133–5141, 2019. 2

    work page 2019

  7. [7]

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

    Alexey Dosovitskiy et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. InICLR, 2021. 3

    work page 2021

  8. [8]

    PODNet: Pooled Outputs Dis- tillation for Small-Tasks Incremental Learning

    Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle. PODNet: Pooled Outputs Dis- tillation for Small-Tasks Incremental Learning. In ECCV, pages 86–102, 2020. 2

    work page 2020

Show all 62 references
  1. [9]

    DyTox: Transformers for Continual Learning with DYnamic TOken eXpansion

    Arthur Douillard, Alexandre Ram ´e, Guillaume Couairon, and Matthieu Cord. DyTox: Transformers for Continual Learning with DYnamic TOken eXpansion. In CVPR, pages 9275–9285, 2022. 2

    2022

  2. [10]

    Robert M. French. Catastrophic forgetting in connection- ist networks. Trends in Cognitive Sciences, pages 128–135,

  3. [11]

    Dynamic Few-Shot Visual Learning Without Forgetting

    Spyros Gidaris and Nikos Komodakis. Dynamic Few-Shot Visual Learning Without Forgetting. In CVPR, pages 4367– 4375, 2018. 2, 3

    2018

  4. [12]

    Pre-trained models: Past, present and future

    Xu Han et al. Pre-trained models: Past, present and future. AI Open, pages 225–250, 2021. 2

    2021

  5. [13]

    Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey

    Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey. Trans. on Machine Learning Re- search, 2024. 2

    2024

  6. [14]

    Natural Adversarial Examples

    Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Stein- hardt, and Dawn Song. Natural Adversarial Examples. In CVPR, pages 15262–15271, 2021. 6, 1

    2021

  7. [15]

    The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization

    Dan Hendrycks et al. The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization. In ICCV, pages 8320–8329, 2021. 6, 1

    2021

  8. [16]

    Distilling the Knowledge in a Neural Network

    Geoffrey Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the Knowledge in a Neural Network. In NIPS Workshops,

  9. [17]

    LoRA: Low-Rank Adaptation of Large Language Models

    Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-Rank Adaptation of Large Language Models. In ICLR, 2022. 2

    2022

  10. [18]

    Distilling Causal Effect of Data in Class-Incremental Learning

    Xinting Hu, Kaihua Tang, Chunyan Miao, Xian-Sheng Hua, and Hanwang Zhang. Distilling Causal Effect of Data in Class-Incremental Learning. In CVPR, pages 3957–3966,

  11. [19]

    Class-Incremental Learning with CLIP: Adaptive Representation Adjustment and Parameter Fusion

    Linlan Huang et al. Class-Incremental Learning with CLIP: Adaptive Representation Adjustment and Parameter Fusion. In ECCV, pages 214–231, 2024. 3

    2024

  12. [20]

    Editing models with task arithmetic

    Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. In ICLR, 2023. 4

    2023

  13. [21]

    Vi- sual Prompt Tuning

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Vi- sual Prompt Tuning. In ECCV, pages 709–727, 2022. 1, 2

    2022

  14. [22]

    Overcoming catastrophic forgetting in neural networks

    James Kirkpatrick et al. Overcoming catastrophic forgetting in neural networks. National Academy of Sciences , pages 3521–3526, 2017. 2

    2017

  15. [23]

    Learning Multiple Layers of Features from Tiny Images

    Alex Krizhevsky. Learning Multiple Layers of Features from Tiny Images. University of Toronto, 2009. 6, 1

    2009

  16. [24]

    Deep Model Fusion: A Survey

    Weishi Li, Yong Peng, Miao Zhang, Liang Ding, Han Hu, and Li Shen. Deep Model Fusion: A Survey. arXiv:2309.15698, 2023. 4, 5

    2023 arXiv

  17. [25]

    Convergent Learning: Do different neural net- works learn the same representations? In NIPS Workshop, pages 196–212, 2015

    Yixuan Li, Jason Yosinski, Jeff Clune, Hod Lipson, and John Hopcroft. Convergent Learning: Do different neural net- works learn the same representations? In NIPS Workshop, pages 196–212, 2015. 5

    2015

  18. [26]

    Learning without Forgetting

    Zhizhong Li and Derek Hoiem. Learning without Forgetting. IEEE TPAMI, pages 2935–2947, 2018. 2

    2018

  19. [27]

    InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learning

    Yan-Shuo Liang and Wu-Jun Li. InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learning. In CVPR, pages 23638–23647, 2024. 3, 4

    2024

  20. [28]

    RMM: Rein- forced Memory Management for Class-Incremental Learn- ing

    Yaoyao Liu, Bernt Schiele, and Qianru Sun. RMM: Rein- forced Memory Management for Class-Incremental Learn- ing. In NeurIPS, pages 3478–3490, 2021. 2

    2021

  21. [29]

    Gradient episodic memory for continual learning

    David Lopez-Paz and Marc’Aurelio Ranzato. Gradient episodic memory for continual learning. In NeurIPS, pages 6470–6479, 2017. 1, 2

    2017

  22. [30]

    Merging Models with Fisher-Weighted Averaging

    Michael S Matena and Colin A Raffel. Merging Models with Fisher-Weighted Averaging. InNeurIPS, pages 17703– 17716, 2022. 4

    2022

  23. [31]

    McDonnell, Dong Gong, Amin Parveneh, Ehsan Abbasnejad, and Anton van den Hengel

    Mark D. McDonnell, Dong Gong, Amin Parveneh, Ehsan Abbasnejad, and Anton van den Hengel. RanPAC: random projections and pre-trained models for continual learning. In NeurIPS, pages 12022–12053, 2023. 3, 4

    2023

  24. [32]

    Learning to Remember: A Synap- tic Plasticity Driven Framework for Continual Learning

    Oleksiy Ostapenko, Mihai Puscas, Tassilo Klein, Patrick Jah- nichen, and Moin Nabi. Learning to Remember: A Synap- tic Plasticity Driven Framework for Continual Learning. In CVPR, pages 11313–11321, 2019. 2

    2019

  25. [33]

    AdapterFusion: Non- Destructive Task Composition for Transfer Learning

    Jonas Pfeiffer, Aishwarya Kamath, Andreas R ¨uckl´e, Kyunghyun Cho, and Iryna Gurevych. AdapterFusion: Non- Destructive Task Composition for Transfer Learning. In EACL, pages 487–503, 2021. 1

    2021

  26. [34]

    DualNet: Con- tinual Learning, Fast and Slow

    Quang Pham, Chenghao Liu, and Steven Hoi. DualNet: Con- tinual Learning, Fast and Slow. In NeurIPS, pages 16131– 16144, 2021. 2

    2021

  27. [35]

    iTAML: An Incre- mental Task-Agnostic Meta-learning Approach

    Jathushan Rajasegaran, Salman Khan, Munawar Hayat, Fa- had Shahbaz Khan, and Mubarak Shah. iTAML: An Incre- mental Task-Agnostic Meta-learning Approach. In CVPR, pages 13588–13597, 2020. 3

    2020

  28. [36]

    Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. iCaRL: Incremental Clas- sifier and Representation Learning. In CVPR, pages 5533– 5542, 2017. 1, 2, 6, 3, 4

    2017

  29. [37]

    ImageNet-21K Pretraining for the Masses

    Tal Ridnik, Emanuel Ben-Baruch, Asaf Noy, and Lihi Zel- nik. ImageNet-21K Pretraining for the Masses. In NeurIPS Track on Datasets and Benchmarks, 2021. 6, 1

    2021

  30. [38]

    Continual learning with deep generative replay

    Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. Continual learning with deep generative replay. In NeurIPS, pages 2994–3003, 2017. 1, 2

    2017

  31. [39]

    Privacy-Preserving Deep Learning

    Reza Shokri and Vitaly Shmatikov. Privacy-Preserving Deep Learning. In ACM Conf. on CCS, pages 1310–1321, 2015. 1

    2015

  32. [40]

    On Learning the Geodesic Path for Incremental Learning

    Christian Simon, Piotr Koniusz, and Mehrtash Harandi. On Learning the Geodesic Path for Incremental Learning. In CVPR, pages 1591–1600, 2021. 2

    2021

  33. [41]

    Smith, L

    J. Smith, L. Karlinsky, V . Gutta, P. Cascante-Bonilla, D. Kim, A. Arbelle, R. Panda, R. Feris, and Z. Kira. CODA- Prompt: COntinual Decomposed Attention-Based Prompt- ing for Rehearsal-Free Continual Learning. In CVPR, pages 11909–11919, 2023. 1, 2, 6, 7

    2023

  34. [42]

    Prototypical networks for few-shot learning

    Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In NeurIPS, pages 4080– 4090, 2017. 2

    2017

  35. [43]

    Optimizing Mode Con- nectivity via Neuron Alignment

    Norman Tatro, Pin-Yu Chen, Payel Das, Igor Melnyk, Prasanna Sattigeri, and Rongjie Lai. Optimizing Mode Con- nectivity via Neuron Alignment. In NeurIPS, pages 15300– 15311, 2020. 5

    2020

  36. [44]

    Is Learning The n-th Thing Any Easier Than Learning The First? In NeurIPS, pages 640–646, 1995

    Sebastian Thrun. Is Learning The n-th Thing Any Easier Than Learning The First? In NeurIPS, pages 640–646, 1995. 1

    1995

  37. [45]

    GCR: Gradient Coreset Based Replay Buffer Selection for Continual Learning

    Rishabh Tiwari, Krishnateja Killamsetty, Rishabh Iyer, and Pradeep Shenoy. GCR: Gradient Coreset Based Replay Buffer Selection for Continual Learning. In CVPR, pages 99–108, 2022. 2

    2022

  38. [46]

    Gomez, Łukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, pages 6000–6010, 2017. 3

    2017

  39. [47]

    The Caltech-UCSD Birds-200- 2011 Dataset

    Catherine Wah, Steve Branson, Peter Welinder, Pietro Per- ona, and Serge Belongie. The Caltech-UCSD Birds-200- 2011 Dataset. Technical report, California Institute of Tech- nology, 2011. 6, 1

    2011

  40. [48]

    FOSTER: Feature Boosting and Compression for Class-Incremental Learning

    Fu-Yun Wang, Da-Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. FOSTER: Feature Boosting and Compression for Class-Incremental Learning. In ECCV, pages 398–414,

  41. [49]

    DualPrompt: Complementary Prompt- ing for Rehearsal-Free Continual Learning

    Zifeng Wang et al. DualPrompt: Complementary Prompt- ing for Rehearsal-Free Continual Learning. In ECCV, pages 631–648, 2022. 1, 2, 6, 7

    2022

  42. [50]

    Learning to Prompt for Continual Learn- ing

    Zifeng Wang et al. Learning to Prompt for Continual Learn- ing. In CVPR, pages 139–149, 2022. 1, 2, 6, 7

    2022

  43. [51]

    Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time

    Mitchell Wortsman et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In ICML, pages 23965–23998,

  44. [52]

    TKIL: Tangent Kernel Optimization for Class Balanced Incremental Learning

    Jinlin Xiang and Eli Shlizerman. TKIL: Tangent Kernel Optimization for Class Balanced Incremental Learning. In ICCV Workshops, pages 3529–3539, 2023. 3

    2023

  45. [53]

    TIES-Merging: Resolving Interfer- ence When Merging Models

    Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raf- fel, and Mohit Bansal. TIES-Merging: Resolving Interfer- ence When Merging Models. In NeurIPS, pages 7093–7115,

  46. [54]

    S. Yan, J. Xie, and X. He. DER: Dynamically Expandable Representation for Class Incremental Learning. In CVPR, pages 3013–3022, 2021. 1, 2, 3, 4

    2021

  47. [55]

    Semantic Drift Compensation for Class-Incremental Learning

    Lu Yu, Bartlomiej Twardowski, Xialei Liu, Luis Herranz, Kai Wang, Yongmei Cheng, Shangling Jui, and Joost van de Weijer. Semantic Drift Compensation for Class-Incremental Learning. In CVPR, pages 6982–6991, 2020. 4

    2020

  48. [56]

    A Large-scale Study of Representa- tion Learning with the Visual Task Adaptation Benchmark

    Xiaohua Zhai et al. A Large-scale Study of Representa- tion Learning with the Visual Task Adaptation Benchmark. arXiv:1910.04867, 2020. 6, 1

    1910 arXiv

  49. [57]

    A Model or 603 Exemplars: Towards Memory- Efficient Class-Incremental Learning

    Da-Wei Zhou, Qi-Wei Wang, Han-Jia Ye, and De-Chuan Zhan. A Model or 603 Exemplars: Towards Memory- Efficient Class-Incremental Learning. In ICLR, 2023. 1, 2, 3, 4

    2023

  50. [58]

    Continual learning with pre-trained mod- els: A survey

    Da-Wei Zhou, Hai-Long Sun, Jingyi Ning, Han-Jia Ye, and De-Chuan Zhan. Continual learning with pre-trained mod- els: A survey. In Int. Joint.Conf. on AI , pages 8363–8371,

  51. [59]

    Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning

    Da-Wei Zhou, Hai-Long Sun, Han-Jia Ye, and De-Chuan Zhan. Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning. In CVPR, pages 23554–23564, 2024. 1, 2, 3, 6, 7, 4

    2024

  52. [60]

    Class-Incremental Learning: A Survey

    Da-Wei Zhou, Qi-Wei Wang, Zhi-Hong Qi, Han-Jia Ye, De- Chuan Zhan, and Ziwei Liu. Class-Incremental Learning: A Survey. IEEE TPAMI, pages 1–20, 2024. 2

    2024

  53. [61]

    Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Need

    Da-Wei Zhou, Han-Jia Ye, De-Chuan Zhan, and Ziwei Liu. Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Need. IJCV, 2024. 1, 2, 3, 6, 7

    2024

  54. [62]

    https://github.com/sun-hailong/CVPR24-Ease, 2024

    Da-Wei Zhou, Hai-Long Sun, Han-Jia Ye, and De-Chuan Zhan. https://github.com/sun-hailong/CVPR24-Ease, 2024. 6, 2 Adapter Merging with Centroid Prototype Mapping for Scalable Class-Incremental Learning Supplementary Material A. Dataset Details This section outlines the benchmar...

    2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.