Pith. sign in

REVIEW 4 major objections 6 minor 70 references

This paper claims that a frozen pre-trained backbone can keep learning new classes sequentially when three small trainable additions — a subspace-projection adapter, low-rank sub-networks, and distance regularization — are attached to it, a

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 17:32 UTC pith:B3XV2I5M

load-bearing objection Strong benchmark claims, but the core forward path as written has a dimension mismatch (d1 vs 2d1), so the reported numbers are not reproducible from the manuscript alone. the 4 major comments →

arxiv 2607.17593 v1 pith:B3XV2I5M submitted 2026-07-20 cs.CV

Miles: Metric Learning with Expandable Subspace for Pre-Trained Model-Based Class-Incremental Learning

Kai Jiang , Zisong Lin , Hongyuan Zhang , Xueru Bai , Xuelong Li This is my paper
classification cs.CV
keywords class incremental learningpre-trained modelsmetric learningprototypeparameter-efficient fine-tuningcatastrophic forgettingsubspace projectiondistance regularization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to solve class incremental learning with a pre-trained model by making the frozen backbone's feature space reusable for many tasks. Its central claim is that old categories naturally form clusters in a pre-trained model's feature space, so each new task can be learned as a small metric-learning problem: pull new-class features toward their prototypes while keeping distances across all task subspaces comparable. To do this, MILES adds a per-task adapter that projects features into a task subspace, a per-task low-rank sub-network that extends the backbone's representation, and a distance-regularization term that stops the model from overfitting a recent task by compressing its feature distances. If correct, the method would give exemplar-free continual learning both strong final accuracy and low forgetting, with only a small per-task parameter cost and a single forward pass at inference.

Core claim

MILES claims that the plasticity-stability trade-off in pre-trained-model-based class incremental learning can be handled by prototype-guided metric learning rather than by prompt pooling, adapter ensembles, or exemplar rehearsal. The method keeps the pre-trained backbone entirely frozen. For each task it trains a subspace prototype projection adapter that shortens distances between current task features and their class prototypes, and a subspace representation extension sub-network of low-rank MLPs that mimics each transformer block using the backbone's intermediate attention features. A distance regularization term then enforces that average feature-to-prototype distances remain balanced a

What carries the argument

The central object is a set of per-task lightweight modules attached to a frozen Vision Transformer backbone. The SPP adapter is a residual MLP plus a progressive down/up projection along the channel dimension, initialized to zero so it starts as the identity; it projects backbone features into a task-specific subspace close to that task's class prototypes. The SRE sub-network stacks two low-rank MLPs per transformer block, taking the backbone's intermediate attention output as a skip input, so it extends the frozen representation by re-learning intermediate features. The DR loss averages distances from current-task features to prototypes and matches this average to the stored average distan

Load-bearing premise

The stability loss in Eq. (10) assumes old-class features can be represented by pseudo-Gaussian distributions with per-class variances, and the paper does not explain how those variances are computed, stored, or updated in the exemplar-free setting, so the forgetting suppression may not transfer to an implementation that has to guess them.

What would settle it

Reimplement MILES and check what happens when the per-class variance σ_c in Eq. (10) is replaced by a single global variance or by a variance estimated from the current task's own feature statistics. If the forgetting rate on CIFAR-100 at T=10 rises substantially from the reported 3.12% and approaches the ablation without the stability loss, then the pseudo-Gaussian variance assumption is load-bearing and underspecified. Any public code release that omits the exact σ_c computation would make the reported result non-reproducible.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If correct, exemplar-free continual learning can outperform methods that store old samples, so memory-constrained applications could drop rehearsal buffers without losing accuracy.
  • Since the backbone stays frozen and each task adds only small adapters and low-rank sub-networks, inference needs one forward pass through the backbone, and the per-task compute overhead grows slowly with the number of tasks.
  • Because classification is prototype-based, a new task can be inserted by computing its prototypes and training its modules, without touching previously trained modules.
  • The reported low forgetting rates on long 50-task sequences suggest the method's stability mechanism is what prevents degradation over many incremental steps, not just on short runs.
  • The gap over the runner-up in several datasets means even a small gain in prototype-distance calibration can translate into a meaningful accuracy advantage over prompt- and adapter-based competitors.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's reliance on per-class Gaussian pseudo-features suggests a natural, testable extension: replace the scalar variance with a full per-class covariance or a mixture model, which should help classes with multimodal or elongated feature distributions.
  • The distance-regularization term implicitly treats feature distances as if they are comparable across tasks; this may make the method sensitive to task order or to tasks whose natural feature scale differs greatly, even though the paper's order-robustness experiments show small fluctuations.
  • The method's foundation on pre-trained feature clustering means it would likely degrade under strong distribution shift; the paper itself flags this in its limitation section, so an uncertainty-aware prototype mechanism for out-of-distribution streams is the most direct next step.
  • Because the final classifier is purely prototype-based, the same machinery could be lifted to incremental few-shot or open-world recognition, where new classes arrive with very few samples and no rehearsal.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes MILES, an exemplar-free pre-trained-model-based class-incremental learning method combining (i) Subspace Prototype Projection (SPP), a per-task adapter that maps backbone features into a task subspace and is trained with a prototype-center loss; (ii) Subspace Representation Extension (SRE), per-task low-rank sub-networks that consume intermediate backbone features and produce a subspace feature that is concatenated with the backbone feature; and (iii) Distance Regularization (DR), two auxiliary losses (Eqs. (21) and (23)) that balance average distances across task subspaces. Experiments on six benchmarks (CIFAR-100, CUB-200, Omnibenchmark, ImageNet-A, ImageNet-R, FOOD-101) at T=10 and T=50 report state-of-the-art average accuracy and low forgetting relative to prompt-, adapter-, and expansion-based baselines, along with ablations, hyperparameter sensitivity studies, parameter/MAC comparisons, and t-SNE visualizations. The central claim is that MILES achieves SOTA performance, e.g., 93.93% average accuracy on CIFAR-100 at T=10 and 62.33% on ImageNet-A at T=10 (Tabs. II and III).

Significance. Conditional on the implementation matching the described method, the claimed results are interesting: exemplar-free SOTA across six standard PTM-CIL benchmarks with a modest parameter footprint (Fig. 8c) and a single forward pass at inference. The paper has several genuine strengths: the ablation study reports means and standard deviations (Tab. IV), sensitivity to the hyperparameters β/γ and to r/d3 is documented (Fig. 8b, Tab. V), computational overhead is measured (Fig. 8d), the clustering motivation is checked empirically on multiple backbones (Tab. VI), and an explicit limitation paragraph acknowledges that OOD robustness is not systematically studied. These elements make the empirical claim more credible than a bare accuracy table. However, the architecture is incompletely specified (the SPP/SRE concatenation is dimensionally inconsistent, and the pseudo-Gaussian variance σ_c is never defined), the main accuracy tables carry no error bars despite three repeats, and no code is released. These issues are decisive for the review.

major comments (4)
  1. [Sec. III-B/III-C/III-D; Eqs. (4)-(8), (18); Algorithm 2] The method as written has no valid forward pass. The SPP adapter A_t is defined with W_mlp ∈ R^{d1×d1}, b_mlp ∈ R^{d1}, and identity/down-up branches operating on a d1-dimensional input (Eqs. (4)-(8)), and Eq. (12) evaluates A_i on F(x). Yet Eq. (18), Eq. (22), and Algorithm 2 line 3 feed A_m(F(x) ⊕ S_m(x))—a 2d1-dimensional concatenation—with no projection or reshaping layer specified anywhere in Sec. III. Additionally, Eq. (12) and Algorithm 2 define two different test-time features for the same adapter. The results in Tabs. II-III therefore cannot be attributed to the described architecture without undocumented implementation choices; the exact adapter input dimension, the mechanism for handling the concatenated vector, and the variant used to produce the tables must be stated.
  2. [Sec. III-C; Eqs. (10)-(11)] Eq. (10) defines L_stable as the mean distance between pseudo-old features A_t(ˆf_c) and the new-task prototypes. Since Eq. (11) minimizes L_center + β·L_stable, this term as written pulls old-class pseudo-features toward the new prototypes—the opposite of the stated intent ('increase the distance between the old categories and the new task prototypes'; 'maximizes the distance between out-of-task features and intra-task categories'). If the intended effect is repulsion, the term must appear with a negative sign or be defined as a negative distance. The manuscript must specify the sign used in the experiments; otherwise the reported forgetting suppression cannot be attributed to this mechanism.
  3. [Sec. III-C/E; Eqs. (10), (20)] The pseudo-Gaussian assumption G_c(p_c, σ_c) requires per-class variance σ_c for old classes at every future task, but the paper never specifies how σ_c is computed, stored, or updated in the exemplar-free setting: Algorithm 1 only mentions prototypes, and Sec. III-A explicitly forbids retaining old data. If σ_c is estimated on task-t data and persisted, the storage and update rule must be given; if it is unavailable for old classes, Eqs. (10) and (20) cannot be implemented as written. This is essential for reproducing the stability and distance-regularization losses.
  4. [Tabs. II-III; Sec. IV-A] The main results are reported without standard deviations although the protocol states three repeats (Sec. IV-A) and Tab. IV provides std for ablations. Several reported margins are small (e.g., 0.88% over DGR on CIFAR-100 T=10; 0.99% on CUB T=10 in Tab. II), so without error bars or per-seed results these advantages cannot be statistically assessed. Moreover, no code is released. Given the specification gaps raised in the previous comments, a reimplementation could plausibly diverge from the reported numbers; code release or a detailed architecture appendix is needed to substantiate the SOTA claim.
minor comments (6)
  1. [Sec. III-A/B; Eqs. (9), (12)] The Manhattan distance |·| is used without a definition; specify that the L1 norm is intended and state whether prototypes/features are normalized before distance computation.
  2. [Algorithm 1; line 6] The instruction 'Recalculate the prototypes' is ambiguous: are prototypes recomputed with the frozen backbone F or through the updated adapter A_t? If through A_t, the optimization target in Eq. (9) drifts during training; the exact choice should be stated.
  3. [Sec. III-E; Eq. (18)] The task-average constraint draws one random previous task per sample; this is a high-variance estimator. Please state how many tasks/samples are drawn per batch and whether this design choice was ablated.
  4. [Sec. IV-D; Tab. V] In Tab. V the recommended setting r=4, d3=16 yields 84.75%, while r=2, d3=96 yields 85.33% on ImageNet-R T=10. The 'performance remains stable' claim is fair, but the chosen default is not the best configuration found; clarify the selection criterion.
  5. [Figs. 6-7; Title/Abstract] The trend-figure annotations contain garbled text (e.g., '1.73/s8593'), and the method name is inconsistently capitalized (Miles vs. MILES). Please correct.
  6. [Sec. III-C; Eqs. (7)-(8)] The down/up projection notation mixes f_in, f_down, and f; the per-stage dimension bookkeeping should be made explicit in the formulas, complementing the helpful 768→192→48→12 example.

Circularity Check

0 steps flagged

No significant circularity: MILES's claims are empirical and externally benchmarked.

full rationale

The central claim of MILES is an empirical accuracy result on six standard PTM-based CIL benchmarks, not a derived quantity that reduces to fitted constants. The prototypes in Eq. (2) are computed directly from training data, and the losses in Eqs. (9)-(24) are training objectives that pull features toward those prototypes; this is standard metric-learning behavior, not a prediction that is equivalent to its own inputs. The ablation study in Tab. IV independently varies the three proposed components (SPP, SRE, DR) and shows incremental accuracy gains, which supports the causal claim without relying on a circular argument. The paper's self-citations ([7], [56], [58]) appear in the related-work survey and are not used to justify the core mechanism, so they are not load-bearing. The under-specified pseudo-Gaussian variance in Eq. (10) and the apparent concatenation-dimension mismatch in Secs. III-B/III-D are implementation or correctness concerns, not circularity: the manuscript's equations do not identify any predicted output with an input by construction. The appended limitation about OOD robustness also concedes scope rather than indicating circular reasoning. No specific reduction of a claimed result to its inputs can be exhibited, so the appropriate finding is no significant circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 0 invented entities

MILES introduces no new physical entities. Its main free parameters are loss weights and architectural ranks chosen on ImageNet-R. The most consequential unstated assumptions are the transferability of frozen ViT features and the availability of old-class Gaussian variance for pseudo-feature sampling.

free parameters (4)
  • β (stability loss weight) = 0.1
    Trade-off between center loss and stability loss in Eq. (24); default selected after ImageNet-R sensitivity study (Fig. 8b).
  • γ (distance regularization weight) = 0.001
    Weight for task/class average distance constraints in Eq. (24); chosen via sensitivity study on ImageNet-R.
  • r (SPP downsampling ratio) = 4
    Controls SPP bottleneck dimension; selected from {2,4,8} on ImageNet-R (Tab. V).
  • d3 (SRE low-rank width) = 16
    Width of the low-rank MLPs in each SRE block; selected from {4,16,48,96} on ImageNet-R (Tab. V).
axioms (3)
  • domain assumption Pre-trained ViT features of unseen classes cluster around class prototypes, making Manhattan-distance-to-prototype a usable classifier for downstream classes.
    Stated as motivation in §I and supported empirically by Tab. VI, but it is a transferability assumption; if false, the SPP/SRE machinery has no reliable prototype structure to exploit.
  • domain assumption Old-class features can be represented by pseudo-Gaussian distributions G_c(p_c, σ_c) with known variance, for sampling pseudo-features.
    Eq. (10) in §III-C requires per-class variance σ_c for old classes, but the paper never defines how σ_c is computed or stored in the exemplar-free setting.
  • domain assumption Frozen backbone features remain a valid semantic representation across all incremental tasks; task-specific subspaces do not require realignment.
    The whole parameter-expansion design assumes the frozen ViT does not drift and that each subspace can be learned independently; the paper does not analyze distribution shift or OOD robustness, and its own limitation section concedes this.

pith-pipeline@v1.3.0-alltime-deepseek · 22200 in / 11193 out tokens · 105419 ms · 2026-08-01T17:32:13.799602+00:00 · methodology

0 comments
read the original abstract

Class Incremental Learning (CIL) aims to learn new concepts consistently from a data stream without forgetting. Unlike typical CIL methods which need to learn a model from scratch, pre-trained model (PTM) can easily adapt to a new task with fine-tuning. However, existing PTM-based CIL methods fail to achieve a trade-off between performance and computational expenditure, i.e., they either adopt the same parameter space so that leading catastrophic forgetting, or expand a new branch for each task but adding more computational cost. To this end, we propose MetrIc Learning with Expandable Subspace (Miles) to harness the prior information within pre-trained knowledge, thereby orchestrating an efficient expansion of the parameter space through guided optimization. Specifically, it decouples the learnable modules with the pre-trained model, exploiting prior information from intermediate features of the backbone network to enable more flexible parameter expansion. Then, a central loss is adopted to guide the new category to cluster towards the corresponding prototype in the new task subspace while incorporating an auxiliary distance regularization term to maintain metric equilibrium across tasks. Extensive experiments on six benchmark datasets demonstrate that Miles achieves state-of-the-art performance in various CIL settings.

Figures

Figures reproduced from arXiv: 2607.17593 by Hongyuan Zhang, Kai Jiang, Xuelong Li, Xueru Bai, Zisong Lin.

Figure 1
Figure 1. Figure 1: Average accuracies of incremental trends for different approaches. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Motivation of the MILES. adopt these prototypes as prior knowledge from pre-training to guide the optimization of adapting new task instead of only applying the current task data, thus avoiding model preference for new tasks. Specifically, we propose Subspace Prototype Projection (SPP) to cascade a tiny learnable module after the backbone for each task to reduce the distance between features and the corres… view at source ↗
Figure 3
Figure 3. Figure 3: Overall structure of the proposed method. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Structure of a SPP adapter. In the training phase, only a set of SPP adapter and SRE sub-network is involved to maintain a low training cost. The SRE sub-network learns features from samples of the current task and intermediate features from the backbone network, aiming to augment the representation capacity of the frozen backbone. Then, the output subspace feature is concatenated with the original feature… view at source ↗
Figure 5
Figure 5. Figure 5: Structure of a SRE sub-network. Then, the distance between the intra-task categories and their corresponding prototypes is optimized according to Eq. (9). Lcenter (xi , yi) = |At (F (xi)) − pyi | (9) Considering that merely optimizing the SPP adapter At by Eq. (9) may induce overfitting to new tasks (as all input vectors will be pulled closer to the prototypes of new cate￾gories), we propose to introduce t… view at source ↗
Figure 6
Figure 6. Figure 6: Incremental trends on six different datasets with the 10-step setting. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Incremental trends on six different datasets with the 50-step setting. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Additional experiments. (a) MILES with different learning order: Different learning orders are applied for MILES in the ImageNet-R dataset. The learning orders are determined by random seeds of {1994, 1995, 1996, 1997, 1998}. (b) Hyperparameter sensitivity experiment: We validate MILES with different combinations of hyper-parameters in the ImageNet-R datasets and report the average accuracy. (c) Analysis o… view at source ↗
Figure 9
Figure 9. Figure 9: T-SNE visualization in original feature space and feature subspaces. The model is incrementally trained on three tasks within the CIFAR-100 dataset, [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: T-SNE visualization of original features and corresponding task subspace projection features. In each group, the left is the visualization of the original [PITH_FULL_IMAGE:figures/full_fig_p013_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

70 extracted references · 3 linked inside Pith

  1. [1]

    Multi-label auroral image classification based on cnn and transformer,

    H. Su, Q. Yang, Y . Ning, Z. Hu, and L. Liu, “Multi-label auroral image classification based on cnn and transformer,” IEEE Transactions on Image Processing, vol. 34, pp. 1835– 1848, 2025

    2025

  2. [3]

    Rectified noise: A generative model using positive-incentive noise,

    Z. Gu, Y . Xu, S. Huang, Y . Guo, and H. Zhang, “Rectified noise: A generative model using positive-incentive noise,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 40, 2026, pp. 4357–4365

    2026

  3. [4]

    Detector with classifier2: An end-to-end multi-stream feature aggregation network for fine-grained object detection in remote sensing images,

    S. Zheng, Z. Wu, Y . Xu, C. He, and Z. Wei, “Detector with classifier2: An end-to-end multi-stream feature aggregation network for fine-grained object detection in remote sensing images,”IEEE Transactions on Image Processing, vol. 34, pp. 2707–2720, 2025

    2025

  4. [5]

    Catastrophic forgetting in connectionist net- works,

    R. M. French, “Catastrophic forgetting in connectionist net- works,”Trends in cognitive sciences, vol. 3, no. 4, pp. 128– 135, 1999

    1999

  5. [6]

    Overcoming catastrophic forgetting in neural networks,

    J. Kirkpatrick et al., “Overcoming catastrophic forgetting in neural networks,”Proceedings of the national academy of sciences, vol. 114, no. 13, pp. 3521–3526, 2017. 14

    2017

  6. [7]

    Multimodal continual learning with mllms from multi- scenario perspectives,

    K. Jiang, S. Huang, X. Chen, J. Shao, H. Zhang, and X. Li, “Multimodal continual learning with mllms from multi- scenario perspectives,”arXiv preprint arXiv:2511.18507, 2025

    arXiv 2025

  7. [8]

    Multi-target pan- class intrinsic relevance driven model for improving semantic segmentation in autonomous driving,

    Y . Cai, L. Dai, H. Wang, and Z. Li, “Multi-target pan- class intrinsic relevance driven model for improving semantic segmentation in autonomous driving,”IEEE Transactions on Image Processing, vol. 30, pp. 9069–9084, 2021

    2021

  8. [9]

    Adaptive dual-axis style-based recalibration network with class-wise statistics loss for imbalanced medical image classification,

    X. Zhang et al., “Adaptive dual-axis style-based recalibration network with class-wise statistics loss for imbalanced medical image classification,”IEEE Transactions on Image Processing, vol. 34, pp. 2081–2096, 2025

    2081

  9. [10]

    A continual learning survey: Defy- ing forgetting in classification tasks,

    M. De Lange et al., “A continual learning survey: Defy- ing forgetting in classification tasks,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 7, pp. 3366–3385, 2022

    2022

  10. [11]

    Online continual learning in image classification: An empir- ical survey,

    Z. Mai, R. Li, J. Jeong, D. Quispe, H. Kim, and S. Sanner, “Online continual learning in image classification: An empir- ical survey,”Neurocomputing, vol. 469, pp. 28–51, 2022

    2022

  11. [12]

    Class-incremental learning: Sur- vey and performance evaluation on image classification,

    M. Masana, X. Liu, B. Twardowski, M. Menta, A. D. Bag- danov, and J. Van De Weijer, “Class-incremental learning: Sur- vey and performance evaluation on image classification,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 5, pp. 5513–5533, 2022

    2022

  12. [13]

    Class incremental learning via contrastive complementary augmen- tation,

    X. Wang, X. Yang, K. Wei, Y . Gu, and C. Deng, “Class incremental learning via contrastive complementary augmen- tation,”IEEE Transactions on Image Processing, vol. 34, pp. 3663–3673, 2025

    2025

  13. [14]

    Ntk-guided few-shot class incremental learning,

    J. Liu, Z. Ji, Y . Pang, and Y . Yu, “Ntk-guided few-shot class incremental learning,”IEEE Transactions on Image Process- ing, vol. 33, pp. 6029–6044, 2024

    2024

  14. [15]

    Viewmask-1-to-3: Multi-view consistent image generation via multimodal discrete diffusion models,

    R. Zhu, Z. Huang, J. Sun, P. Luo, H. Zhang, and X. Li, “Viewmask-1-to-3: Multi-view consistent image generation via multimodal discrete diffusion models,” inProceedings of the 43rd International Conference on Machine Learning (ICML), 2026

    2026

  15. [16]

    Learning without forgetting,

    Z. Li and D. Hoiem, “Learning without forgetting,”IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 12, pp. 2935–2947, 2017

    2017

  16. [17]

    Expert gate: Lifelong learning with a network of experts,

    R. Aljundi, P. Chakravarty, and T. Tuytelaars, “Expert gate: Lifelong learning with a network of experts,” inProceedings of the IEEE conference on computer vision and pattern recog- nition, 2017, pp. 3366–3375

    2017

  17. [18]

    Semantic drift compensation for class- incremental learning,

    L. Yu et al., “Semantic drift compensation for class- incremental learning,” inProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, 2020, pp. 6982–6991

    2020

  18. [19]

    Icarl: Incremental classifier and representation learning,

    S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert, “Icarl: Incremental classifier and representation learning,” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2017, pp. 2001–2010

    2017

  19. [20]

    Rmm: Reinforced memory management for class-incremental learning,

    Y . Liu, B. Schiele, and Q. Sun, “Rmm: Reinforced memory management for class-incremental learning,”Advances in Neu- ral Information Processing Systems, vol. 34, pp. 3478–3490, 2021

    2021

  20. [21]

    Class-incremental exemplar compression for class-incremental learning,

    Z. Luo, Y . Liu, B. Schiele, and Q. Sun, “Class-incremental exemplar compression for class-incremental learning,” inPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 11 371–11 380

    2023

  21. [22]

    Der: Dynamically expandable representation for class incremental learning,

    S. Yan, J. Xie, and X. He, “Der: Dynamically expandable representation for class incremental learning,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 3014–3023

    2021

  22. [23]

    Foster: Feature boosting and compression for class-incremental learn- ing,

    F.-Y . Wang, D.-W. Zhou, H.-J. Ye, and D.-C. Zhan, “Foster: Feature boosting and compression for class-incremental learn- ing,” inEuropean conference on computer vision, Springer, 2022, pp. 398–414

    2022

  23. [24]

    Beef: Bi-compatible class-incremental learning via energy-based expansion and fusion,

    F.-Y . Wang et al., “Beef: Bi-compatible class-incremental learning via energy-based expansion and fusion,” inThe Eleventh International Conference on Learning Representa- tions, 2023

    2023

  24. [25]

    A model or 603 exemplars: Towards memory-efficient class-incremental learning,

    D.-W. Zhou, Q.-W. Wang, H.-J. Ye, and D.-C. Zhan, “A model or 603 exemplars: Towards memory-efficient class-incremental learning,” inThe Eleventh International Conference on Learn- ing Representations, 2023

    2023

  25. [26]

    Large-margin contrastive learning with distance polarization regularizer,

    S. Chen, G. Niu, C. Gong, J. Li, J. Yang, and M. Sugiyama, “Large-margin contrastive learning with distance polarization regularizer,” inInternational conference on machine learning, PMLR, 2021, pp. 1673–1683

    2021

  26. [27]

    Enhance vision-language alignment with noise,

    S. Huang, H. Zhang, and X. Li, “Enhance vision-language alignment with noise,” inProceedings of the AAAI Conference on Artificial Intelligence, 2025

    2025

  27. [28]

    Learning contrastive embedding in low-dimensional space,

    S. Chen, C. Gong, J. Li, J. Yang, G. Niu, and M. Sugiyama, “Learning contrastive embedding in low-dimensional space,” Advances in Neural Information Processing Systems, vol. 35, pp. 6345–6357, 2022

    2022

  28. [29]

    Re- visiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need,

    D.-W. Zhou, Z.-W. Cai, H.-J. Ye, D.-C. Zhan, and Z. Liu, “Re- visiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need,”International Journal of Computer Vision, pp. 1–21, 2024

    2024

  29. [30]

    Continual learning with pre-trained models: A survey,

    D.-W. Zhou, H.-L. Sun, J. Ning, H.-J. Ye, and D.-C. Zhan, “Continual learning with pre-trained models: A survey,”arXiv preprint arXiv:2401.16386, 2024

    Pith/arXiv arXiv 2024

  30. [31]

    Deep metric learning for few-shot image classification: A review of recent develop- ments,

    X. Li, X. Yang, Z. Ma, and J.-H. Xue, “Deep metric learning for few-shot image classification: A review of recent develop- ments,”Pattern Recognition, vol. 138, p. 109 381, 2023

    2023

  31. [32]

    Prototypical networks for few-shot learning,

    J. Snell, K. Swersky, and R. Zemel, “Prototypical networks for few-shot learning,”Advances in neural information processing systems, vol. 30, 2017

    2017

  32. [33]

    Lora: Low-rank adaptation of large language models.,

    E. J. Hu et al., “Lora: Low-rank adaptation of large language models.,”ICLR, vol. 1, no. 2, p. 3, 2022

    2022

  33. [34]

    Data augmentation of contrastive learning is estimating positive-incentive noise,

    H. Zhang, Y . Xu, S. Huang, and X. Li, “Data augmentation of contrastive learning is estimating positive-incentive noise,” arXiv preprint arXiv:2408.09929, 2024

    Pith/arXiv arXiv 2024

  34. [35]

    Visual prompt tuning,

    M. Jia et al., “Visual prompt tuning,” inEuropean conference on computer vision, Springer, 2022, pp. 709–727

    2022

  35. [36]

    Adaptformer: Adapting vision transformers for scalable visual recognition,

    S. Chen et al., “Adaptformer: Adapting vision transformers for scalable visual recognition,”Advances in Neural Information Processing Systems, vol. 35, pp. 16 664–16 678, 2022

    2022

  36. [37]

    Expand- able subspace ensemble for pre-trained model-based class- incremental learning,

    D.-W. Zhou, H.-L. Sun, H.-J. Ye, and D.-C. Zhan, “Expand- able subspace ensemble for pre-trained model-based class- incremental learning,” inProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, 2024, pp. 23 554–23 564

    2024

  37. [38]

    Podnet: Pooled outputs distillation for small-tasks incremen- tal learning,

    A. Douillard, M. Cord, C. Ollion, T. Robert, and E. Valle, “Podnet: Pooled outputs distillation for small-tasks incremen- tal learning,” inComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, Springer, 2020, pp. 86–102

    2020

  38. [39]

    Class- incremental learning with cross-space clustering and con- trolled transfer,

    A. Ashok, K. Joseph, and V . N. Balasubramanian, “Class- incremental learning with cross-space clustering and con- trolled transfer,” inEuropean Conference on Computer Vision, Springer, 2022, pp. 105–122

    2022

  39. [40]

    Improving global generalization and local personalization for federated learning,

    L. Meng et al., “Improving global generalization and local personalization for federated learning,”IEEE Transactions on Neural Networks and Learning Systems, vol. 36, no. 1, pp. 76– 87, 2025

    2025

  40. [41]

    Model atten- tion expansion for few-shot class-incremental learning,

    X. Wang, Z. Ji, Y . Yu, Y . Pang, and J. Han, “Model atten- tion expansion for few-shot class-incremental learning,”IEEE Transactions on Image Processing, vol. 33, pp. 4419–4431, 2024

    2024

  41. [42]

    Class incremental learning with multi-teacher distillation,

    H. Wen et al., “Class incremental learning with multi-teacher distillation,” in2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 28 443– 28 452

    2024

  42. [43]

    Protoconnet: Prototypical augmentation and alignment for open-set few-shot image classification,

    K. Shi et al., “Protoconnet: Prototypical augmentation and alignment for open-set few-shot image classification,”Dis- plays, p. 103 364, 2026. 15

    2026

  43. [44]

    Class- wise balancing data replay for federated class-incremental learning,

    Z. Qi, Y .-P. Tang, L. Meng, H. Yu, X. Li, and X. Meng, “Class- wise balancing data replay for federated class-incremental learning,” inThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025

    2025

  44. [45]

    Rainbow memory: Continual learning with a memory of diverse sam- ples,

    J. Bang, H. Kim, Y . Yoo, J.-W. Ha, and J. Choi, “Rainbow memory: Continual learning with a memory of diverse sam- ples,” inProceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, 2021, pp. 8218–8227

    2021

  45. [46]

    Memorizing com- plementation network for few-shot class-incremental learning,

    Z. Ji, Z. Hou, X. Liu, Y . Pang, and X. Li, “Memorizing com- plementation network for few-shot class-incremental learning,” IEEE Transactions on Image Processing, vol. 32, pp. 937–948, 2023

    2023

  46. [47]

    Dytox: Transformers for continual learning with dynamic token ex- pansion,

    A. Douillard, A. Ram ´e, G. Couairon, and M. Cord, “Dytox: Transformers for continual learning with dynamic token ex- pansion,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 9285– 9295

    2022

  47. [48]

    Overcoming recency bias of normalization statistics in continual learning: Balance and adaptation,

    Y . Lyu et al., “Overcoming recency bias of normalization statistics in continual learning: Balance and adaptation,”Ad- vances in Neural Information Processing Systems, vol. 36, pp. 25 475–25 494, 2023

    2023

  48. [49]

    Explore how to inject beneficial noise in mllms,

    R. Zhu, S. Huang, Z. Jiao, and H. Zhang, “Explore how to inject beneficial noise in mllms,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 40, 2026, pp. 29 150–29 158

    2026

  49. [50]

    Learning to prompt for continual learning,

    Z. Wang et al., “Learning to prompt for continual learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 139–149

    2022

  50. [51]

    Dualprompt: Complementary prompting for rehearsal-free continual learning,

    Z. Wang et al., “Dualprompt: Complementary prompting for rehearsal-free continual learning,” inEuropean conference on computer vision, Springer, 2022, pp. 631–648

    2022

  51. [52]

    Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learn- ing,

    J. S. Smith et al., “Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learn- ing,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 11 909–11 919

    2023

  52. [53]

    Fecam: Exploiting the heterogeneity of class distributions in exemplar-free continual learning,

    D. Goswami, Y . Liu, B. Twardowski, and J. Van De Weijer, “Fecam: Exploiting the heterogeneity of class distributions in exemplar-free continual learning,”Advances in Neural Infor- mation Processing Systems, vol. 36, pp. 6582–6595, 2023

    2023

  53. [54]

    Weighted ensemble models are strong continual learners,

    I. E. Marouf, S. Roy, E. Tartaglione, and S. Lathuili `ere, “Weighted ensemble models are strong continual learners,” inEuropean Conference on Computer Vision, Springer, 2024, pp. 306–324

    2024

  54. [55]

    Gradient reweighting: Towards imbalanced class- incremental learning,

    J. He, “Gradient reweighting: Towards imbalanced class- incremental learning,” inProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, 2024, pp. 16 668–16 677

    2024

  55. [56]

    Recurrent network expansion for class incremental learning,

    K. Jiang, X. Bai, and F. Zhou, “Recurrent network expansion for class incremental learning,”IEEE Transactions on Neural Networks and Learning Systems, vol. 37, no. 1, pp. 122–135, 2026

    2026

  56. [57]

    Boosting multiple views for pretrained-based continual learning,

    Q. Tran et al., “Boosting multiple views for pretrained-based continual learning,” inThe Thirteenth International Confer- ence on Learning Representations, 2025

    2025

  57. [58]

    Mixture of noise for pre-trained model-based class-incremental learning,

    K. Jiang, Z. Shi, D. Zhang, H. Zhang, and X. Li, “Mixture of noise for pre-trained model-based class-incremental learning,” inThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025

    2025

  58. [59]

    Integrating task-specific and universal adapters for pre-trained model-based class- incremental learning,

    Y . Wang, D.-W. Zhou, and H.-J. Ye, “Integrating task-specific and universal adapters for pre-trained model-based class- incremental learning,” inProceedings of the IEEE/CVF Inter- national Conference on Computer Vision, 2025, pp. 806–816

    2025

  59. [60]

    Knowledge memorization and rumination for pre-trained model-based class-incremental learning,

    Z. Gao et al., “Knowledge memorization and rumination for pre-trained model-based class-incremental learning,” in Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 20 523–20 533

    2025

  60. [61]

    Safe: Slow and fast parameter-efficient tuning for continual learning with pre-trained models,

    L. Zhao, X. Zhang, K. Yan, S. Ding, and W. Huang, “Safe: Slow and fast parameter-efficient tuning for continual learning with pre-trained models,”Advances in Neural Information Processing Systems, vol. 37, pp. 113 772–113 796, 2024

    2024

  61. [62]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy et al., “An image is worth 16x16 words: Transformers for image recognition at scale,”arXiv preprint arXiv:2010.11929, 2020

    Pith/arXiv arXiv 2010

  62. [63]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,”Handbook of Systemic Autoimmune Diseases, vol. 1, no. 4, 2009

    2009

  63. [64]

    The caltech-ucsd birds-200-2011 dataset,

    C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie, “The caltech-ucsd birds-200-2011 dataset,”Technical Report CNS-TR-2011-001, 2011

    2011

  64. [65]

    Benchmarking omni- vision representation through the lens of visual realms,

    Y . Zhang, Z. Yin, J. Shao, and Z. Liu, “Benchmarking omni- vision representation through the lens of visual realms,” in European Conference on Computer Vision, Springer, 2022, pp. 594–611

    2022

  65. [66]

    Natural adversarial examples,

    D. Hendrycks, K. Zhao, S. Basart, J. Steinhardt, and D. Song, “Natural adversarial examples,”CVPR, 2021

    2021

  66. [67]

    The many faces of robustness: A critical analysis of out-of-distribution generalization,

    D. Hendrycks et al., “The many faces of robustness: A critical analysis of out-of-distribution generalization,”ICCV, 2021

    2021

  67. [68]

    Food-101– mining discriminative components with random forests,

    L. Bossard, M. Guillaumin, and L. Van Gool, “Food-101– mining discriminative components with random forests,” in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VI 13, Springer, 2014, pp. 446–461

    2014

  68. [69]

    Un- certainty modeling for out-of-distribution generalization,

    X. Li, Y . Dai, Y . Ge, J. Liu, Y . Shan, and L. DUAN, “Un- certainty modeling for out-of-distribution generalization,” in International Conference on Learning Representations, 2022

    2022

  69. [70]

    Harnessing out-of-distribution examples via augmenting content and style,

    Z. Huang et al., “Harnessing out-of-distribution examples via augmenting content and style,” inThe Eleventh International Conference on Learning Representations, 2023

    2023

  70. [71]

    Distribution shift inversion for out-of-distribution prediction,

    R. Yu, S. Liu, X. Yang, and X. Wang, “Distribution shift inversion for out-of-distribution prediction,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 3592–3602. Kai Jiangreceived the B.E. and M.E. degrees from Xidian University, Xi’an, China, in 2022 and 2025, respectively. He is currently pursuing the Ph.D...

    2023