Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

CKAA: Cross-subspace Knowledge Alignment and Aggregation for Robust Continual Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that CKAA keeps class-incremental learning accurate even when the task-id predictor is wrong, by aligning feature subspaces during training and softly mixing adapters during inference.

desk verdict Strong empirical paper with a load-bearing but under-validated simulation component; the tau contradiction is real but fixable. read the letter →

arxiv 2507.09471 v1 pith:ALS6CKEC submitted 2025-07-13 cs.CV

Lingfeng He , De Cheng , Zhiheng Ma , Huaijie Wang , Dingwen Zhang , Nannan Wang , Xinbo Gao This is my paper
classification cs.CV
keywords class-incrementallearningcontinualparameter-efficientfine-tuningadaptertask-idmisidentificationfeaturesubspacealignmentmixtureofadaptersvisualprompttuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper targets a known failure mode of parameter-efficient continual learning: methods assign one sub-module per task and a task recognizer picks the sub-module at test time, but when the recognizer misidentifies the task, features land in the wrong subspace and accuracy drops sharply. CKAA addresses this during training with Dual-level Knowledge Alignment (DKA), which aligns same-class features across subspaces with a contrastive loss and trains a task-adaptive classifier on simulated features of previous classes projected into the current subspace. During inference, Task-Confidence-guided Mixture of Adapters (TC-MoA) replaces hard sub-module selection with confidence-weighted mixing. The paper claims CKAA outperforms existing PEFT-based continual learning methods, with the largest reported gain being 3.03% Last-acc over the previous best method on ImageNetR, and with growing advantages at 50- and 100-task splits.

What carries the argument

The load-bearing mechanism is the subspace-shift transfer in Eq. 6: previous-class features $\tilde f_i$ are sampled from Gaussian approximations of their shared-space distributions, an affinity matrix $G_{ij}$ connects each sampled feature to its nearest current shared features, and the current subspace shift $z_j - f_j$ is transferred as a weighted average to produce simulated features $\tilde z_i$. These pseudo-features let the task-adaptive classifier be trained on old-class features placed in the current subspace, turning the wrong-subspace regime into a training-time input. The other half is TC-MoA's routing: the shared classifier's logits are masked to a top-$K_c$ set and softmaxed into task-confidence scores that weight adapter outputs block by block, so a mistaken top-1 task-id does not cause hard selection of the wrong adapter.

What would settle it

Run CKAA on a benchmark where one previous class has a deliberately bimodal feature distribution, and cache a small exemplar set to compute the true features of that class in the current subspace; compare those true features with the Eq. 6 pseudo-features, and check whether removing CA from the full model changes Last-acc. If the simulated features are far from the true ones and the full model does not beat the FA+TC-MoA baseline, the Gaussian-plus-shift-transfer assumption is falsified.

Watch

Extended reading notes

Core claim

The central claim is that feature-subspace misalignment caused by independently trained task-specific adapters is a main source of error under misleading task-ids, and that this can be repaired at two levels. Feature-level Alignment (FA) pulls same-class features from different task subspaces together contrastively, so a wrongly routed image still lands near its class. Classifier-level Alignment (CA) simulates what previous-task features would look like in the current subspace: it samples shared-space features from per-class Gaussians, builds an affinity matrix to current shared features, transfers the current subspace shift $z_j - f_j$ by a weighted average, and trains the current task-adaptive classifier on those pseudo-features and on real current features. The paper claims this produces a global classifier that separates correctly and incorrectly projected features. At inference, TC-MoA masks the shared classifier's logits to the top-$K_c$ classes, softmaxes them into task-confidence scores $\alpha^t_i$, and uses these as routing weights in a mixture-of-adapters forward pass. The paper reports state-of-the-art Last-acc and Avg-acc on ImageNetR, ImageNetA, CIFAR-100, DomainNet, CUB-200, and StanfordCars, with the largest margin over the previous best at 10-split ImageNetR (3.03 percentage points) and larger margins at 50- and 100-task splits.

Load-bearing premise

The classifier-level alignment assumes that previous-task features in the shared space are well described by a Gaussian per class and that the current task's subspace shift, measured on current data, transfers to previous data through the affinity matrix; if either approximation fails, the simulated pseudo-features can mislead the classifier rather than help it.

Editorial extensions

If this is right

  • If CKAA's reported results hold, sub-module continual learning no longer needs to be brittle to task-id errors: hard selection can be replaced by confidence-weighted mixing without storing exemplars.
  • The reported 50- and 100-task gains (for example 78.20 vs 69.48 Last-acc at 50-split ImageNetR) imply the alignment and aggregation scheme scales to very long task streams with modest added cost (reported +0.026% training cost at 100 tasks).
  • Because CA trains on simulated cross-subspace features, the method offers a route to classifier alignment without saving old images, which matters for memory-limited and privacy-constrained continual learning.
  • The global classifier built by averaging per-task classifier parameters (Eq. 8) is claimed to stay effective across all seen classes without extra training, simplifying deployment after each session.
  • The reported gains on fine-grained CUB-200 and StanfordCars suggest the alignment benefit extends beyond the four standard benchmarks used in the main comparison.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implicit, testable prediction is that CA's reported 2.67–5.40% Last-acc gain over FA+TC-MoA should shrink when previous classes have strongly multi-modal or non-Gaussian feature distributions, because the Gaussian sampling in Eq. 5 would produce poor pseudo-features.
  • The same subspace-shift transfer idea could be applied to other PEFT families such as LoRA or prefix tuning by defining the specific feature as the backbone output with that module active; the paper demonstrates only adapters and prompts.
  • TC-MoA's confidence scores suggest a continuous notion of task identity; one could test whether those scores correlate with image difficulty or domain membership, which the paper only visualizes as top-1 score distributions.
  • A direct validation of the simulation step would be to cache a small exemplar set for previous tasks and compare the cosine distance between Eq. 6 pseudo-features and true projected features, a check the paper does not report.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This manuscript proposes CKAA, a parameter-efficient fine-tuning (PEFT) based continual learning framework designed to be robust against misleading task-ids in class-incremental learning (CIL). The architecture consists of a task-shared visual prompt network serving as a task recognizer, task-specific adapters, and two proposed innovations: Dual-level Knowledge Alignment (DKA), which comprises Feature-level Alignment (FA) and Classifier-level Alignment (CA), and a Task-Confidence-guided Mixture of Adapters (TC-MoA) for adaptive inference. Experiments are conducted on four standard CIL benchmarks (ImageNetR, ImageNetA, CIFAR100, DomainNet), fine-grained datasets (CUB-200, StanfordCars), and long-term settings (20, 50, and 100 tasks). The central claim is that CKAA achieves state-of-the-art results, for example outperforming SSIAT by 3.03%, 1.10%, 0.91%, and 1.61% in Last-acc on the four standard benchmarks.

Significance. If the results hold, CKAA offers a meaningful advance for PEFT-based class-incremental learning, specifically in improving robustness to task-id misclassification. The paper's strengths include comprehensive experiments across multiple benchmarks and split settings, detailed ablations of each component (Tables VI–VIII), hyperparameter analysis (Fig. 6), and computational-cost analysis (Tables XII–XIV). The main concerns are that the Classifier-level Alignment rests on unvalidated Gaussian and subspace-transfer approximations, and that hyperparameters are selected on the same benchmarks used for the final comparisons. There is also a direct internal contradiction in the reported temperature hyperparameter. These issues are fixable but currently limit confidence in the generality of the quantitative claims.

major comments (3)
  1. [Section IV (Implementation Details) and Section IV.E] Implementation Details state 'τ in Eq.11 is set to 3.0', but Section IV.E states 'we set τ = 2.0 for all datasets based on experimental results.' These two statements are irreconcilable and must be corrected, since τ directly controls the TC-MoA routing weights in Eq. (11) and affects the reported results.
  2. [Section III.B (Classifier-level Alignment), Eqs. (5)-(6)] The CA component simulates previous-task features using a Gaussian approximation of shared-feature distributions and transfers the current-task subspace shift through the affinity matrix G. These two approximations are central to CA, which Table VI shows contributes 2.67%–5.40% in Last-acc. The paper does not validate the Gaussian assumption or the transferability of the subspace shift; Table VIII only ablates the presence of subspace transfer and affinities, not the fidelity of the simulation. I request a direct validation (e.g., comparing simulated features to real features of previous tasks on a held-out set) or a sensitivity analysis of the simulation parameters (e.g., number of Gaussian components, sample size Bs), and a discussion of how errors in the simulation would affect the robustness gains.
  3. [Section IV.E] The hyperparameters (τf, τg, Kg, Kc, τ) are selected based on experimental results on 10s-ImageNetR and 10s-ImageNetA, which are the same datasets used for the state-of-the-art comparisons in Table I. The authors should clarify whether a validation split was used for hyperparameter selection, and whether the same values are used for all datasets and split settings (except the stated dataset-specific Kc and the adapter width changes for the 50/100 splits). Without this clarification, the magnitude of the reported improvements over baselines cannot be fully assessed.
minor comments (5)
  1. [Section III.B] In the sentence describing Eq. (6), 'the residual between the specific feature {zi}B i=1 ad the shared feature' contains a typo: 'ad' should be 'and'.
  2. [Section IV] The dataset name 'ImegeNet-21K' appears in the description of ImageNetR; it should be 'ImageNet-21K'.
  3. [Section IV.E] The word 'propsoed' in the first paragraph of Section IV.E should be 'proposed'.
  4. [Algorithm 1, line 12] The comment 'Compute Lcu (Eq.7)' should read 'Compute Lca (Eq.7)' to match the notation used in Eq. (7) and the surrounding text.
  5. [Section III.B, Eq. (4)] The paper does not specify how the CSFA loss behaves when the positive set P(zk) is empty, which can occur if the randomly sampled set eCsp contains no feature with the same label as zk; please state the handling of this case.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CKAA's reported gains are empirical measurements against external baselines, and its losses and inference equations do not encode the benchmark numbers they claim to predict.

full rationale

CKAA is an empirical method paper rather than a formal derivation, and its claimed improvements do not reduce to its inputs by construction. The training objective in Eq. 9 combines cross-entropy (Eq. 3), cross-subspace feature alignment (Eq. 4), and classifier alignment (Eq. 7). The classifier-alignment loss uses pseudo-features produced by Eq. 6, which are Gaussian samples plus affinity-weighted residuals between current specific and shared features. These are approximations of previous-task distributions, but they do not restate the reported Last-acc or Avg-acc values, and no fitted parameter is renamed as a prediction. The global classifier in Eq. 8 is a parameter average, and the inference rule in Eq. 13 combines task-shared and aggregated classifiers without encoding the SOTA margins. The task-shared component follows VPT-NSP [3], a published and independently reproduced baseline, and the orthogonal update is also cross-cited to external works [42], [52], so the self-citation is not load-bearing. The Gaussian and subspace-transfer assumptions in Section III.B are unvalidated and may affect generalization, but an unvalidated approximation is not a circular reduction. Hyperparameter selection in Section IV.E is performed on the same four benchmarks used for the headline results, which is a model-selection and overfitting concern, but it is not a by-construction equivalence between the claim and its inputs under the stated circularity rubric. There is also an internal inconsistency between the Implementation Details (tau = 3.0) and the Parameter Analysis text (tau = 2.0), but that is a reproducibility issue rather than evidence of circularity. Overall, the derivation chain is self-contained with respect to the reported results.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The method relies on standard PEFT building blocks and introduces no new physical or mathematical entities. The main burden falls on the Gaussian approximation of previous-class features, the transferability of the current subspace shift to previous classes, and the reliability of shared-feature logits as task-confidence estimates. Hyperparameters are tuned on the evaluation benchmarks themselves, which is a common but noteworthy practice.

free parameters (6)
  • Loss trade-off weights (Lce, Lcsfa, Lca) = 1.0, 1.0, 1.0
    Set to 1.0 for simplicity in Eq. 9 without ablation; a hand-chosen scalar setting that affects the balance of alignment losses.
  • Temperature tau_f in CSFA loss (Eq. 4) = 0.05
    Selected by hyperparameter scan on 10s-ImageNetR and 10s-ImageNetA (Fig. 6c).
  • Temperature tau_g in affinity matrix (Eq. 5) = 0.2
    Selected by scan in Fig. 6e.
  • Number of nearest neighbors Kg (Eq. 5) = 20
    Selected by scan in Fig. 6d.
  • Top-Kc classes for task confidence (Eq. 10) = 20 for ImageNetA, 10 for others
    Dataset-specific selection based on experiments; not uniform across benchmarks.
  • Temperature tau in task-confidence softmax (Eq. 11) = 3.0 (Section IV Implementation) and 2.0 (Section IV.E)
    The paper contradicts itself; this temperature is tuned on the same test benchmarks.
assumptions (4)
  • domain assumption Previous task classes in the shared feature space follow Gaussian distributions G_{k,t}
    Used in CA to sample pseudo-features; introduced in Section III.B with no validation of normality.
  • domain assumption Subspace shift of current data (z_j - f_j) transfers to previous-class features via affinity matrix G
    Eq. 6 transfers the current residual to simulated previous features; assumed to hold across tasks.
  • domain assumption Orthogonal null-space update of prompts preserves the previous task feature space
    Borrowed from VPT-NSP [3], [42], [52]; underpins the stability of the shared feature space used for Gaussian statistics.
  • domain assumption Shared-feature classifier logits g(f_i) are a reliable proxy for task membership
    TC-MoA routing weights are computed from these logits (Eq. 10-11); no calibration analysis is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CKAA: Cross-subspace Knowledge Alignment and Aggregation for Robust Continual Learning." pith.science (2026). https://pith.science/paper/ALS6CKEC

@misc{pith2026250709471,
  author       = {Pith},
  title        = {Pith review of: CKAA: Cross-subspace Knowledge Alignment and Aggregation for Robust Continual Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ALS6CKEC}},
  note         = {Machine review of arXiv:2507.09471}
}
read the original abstract

Continual Learning (CL) empowers AI models to continuously learn from sequential task streams. Recently, parameter-efficient fine-tuning (PEFT)-based CL methods have garnered increasing attention due to their superior performance. They typically allocate a unique sub-module for learning each task, with a task recognizer to select the appropriate sub-modules for testing images. However, due to the feature subspace misalignment from independently trained sub-modules, these methods tend to produce ambiguous decisions under misleading task-ids. To address this, we propose Cross-subspace Knowledge Alignment and Aggregation (CKAA), a novel framework that enhances model robustness against misleading task-ids through two key innovations: (1) Dual-level Knowledge Alignment (DKA): By aligning intra-class feature distributions across different subspaces and learning a robust global classifier through a feature simulation process, DKA enables the model to distinguish features from both correct and incorrect subspaces during training. (2) Task-Confidence-guided Mixture of Adapters (TC-MoA): A robust inference scheme that adaptively aggregates task-specific knowledge from relevant sub-modules based on task-confidence scores, avoiding overconfidence in misleading task-id predictions. Extensive experiments demonstrate that CKAA outperforms existing PEFT-based CL methods.

Figures

Figures reproduced from arXiv: 2507.09471 by the authors.

Figure 1
Figure 1. Illustration of our motivation. Existing methods (a) encounter [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Classification accuracy of samples with correct and incorrect task-ids. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The framework of our proposed method. Our network architecture (a) is composed of a visual prompt-based task-shared network and adapter-based [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Visualization of the distributions of the top-1 task-confidence scores. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: The task-by-task accuracy changing curves of our baseline ( [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Parameter analysis of Kc and τ in TC-MoA, and τf , Kg and τg in DKA on 10S-ImageNetR and 10S-ImageNetA. G. Analysis of additional computational costs We conduct experiments to analyze the impact of adapter widths on 10s-ImageNetA and 10s-ImageNetR datasets, the results…

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Symbiosis-Inspired Knowledge Distillation for Incremental Object Detection

    cs.CV 2026-07 conditional novelty 6.0 of 10

    SIKD improves incremental object detection by distilling old-model responses in co-occurring, overlapping regions plus old-class rank structure, instead of discarding high-overlap predictions.

  2. Task-Driven Subspace Decomposition for Knowledge Sharing and Isolation in LoRA-based Continual Learning

    cs.LG 2026-02 unverdicted novelty 6.0 of 10

    LoDA introduces task-driven subspace decomposition in LoRA for continual learning to separate knowledge-sharing and isolation directions via energy objectives and closed-form recalibration, outperforming prior null-sp...

Reference graph

Works this paper leans on

78 extracted references · 48 canonical work pages · cited by 2 Pith papers

  1. [1]

    icarl: Incremental classifier and representation learning,

    S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert, “icarl: Incremental classifier and representation learning,” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2017, pp. 2001–2010

    2017

  2. [2]

    Semantically- shifted incremental adapter-tuning is a continual vitransformer,

    Y . Tan, Q. Zhou, X. Xiang, K. Wang, Y . Wu, and Y . Li, “Semantically- shifted incremental adapter-tuning is a continual vitransformer,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 23 252–23 262

    2024

  3. [3]

    Visual Prompt Tuning in Null Space for Continual Learning

    Y . Lu, S. Zhang, D. Cheng, Y . Xing, N. Wang, P. Wang, and Y . Zhang, “Visual prompt tuning in null space for continual learning,” arXiv preprint arXiv:2406.05658, 2024

    work page Pith review arXiv 2024

  4. [4]

    Overcoming catastrophic forgetting in neural networks,

    J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska et al. , “Overcoming catastrophic forgetting in neural networks,” Pro- ceedings of the national academy of sciences , vol. 114, no. 13, pp. 3521–3526, 2017

    2017

  5. [5]

    Learning without forgetting,

    Z. Li and D. Hoiem, “Learning without forgetting,” IEEE transactions on pattern analysis and machine intelligence , vol. 40, no. 12, pp. 2935– 2947, 2017

    2017

  6. [6]

    Overcoming catastrophic forgetting by incremental moment matching,

    S.-W. Lee, J.-H. Kim, J. Jun, J.-W. Ha, and B.-T. Zhang, “Overcoming catastrophic forgetting by incremental moment matching,” Advances in neural information processing systems , vol. 30, 2017

    work page 2017

  7. [7]

    Catastrophic interference in connec- tionist networks: The sequential learning problem,

    M. McCloskey and N. J. Cohen, “Catastrophic interference in connec- tionist networks: The sequential learning problem,” in Psychology of learning and motivation . Elsevier, 1989, vol. 24, pp. 109–165

    1989

  8. [8]

    On the importance of cross-task features for class-incremental learn- ing,

    A. Soutif-Cormerais, M. Masana, J. Van de Weijer, and B. Twardowski, “On the importance of cross-task features for class-incremental learn- ing,” arXiv preprint arXiv:2106.11930 , vol. 1, 2021

    arXiv 2021

Show all 78 references
  1. [9]

    A comprehensive survey of continual learning: theory, method and application,

    L. Wang, X. Zhang, H. Su, and J. Zhu, “A comprehensive survey of continual learning: theory, method and application,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024

    2024

  2. [10]

    Dark experience for general continual learning: a strong, simple baseline,

    P. Buzzega, M. Boschini, A. Porrello, D. Abati, and S. Calderara, “Dark experience for general continual learning: a strong, simple baseline,” Advances in neural information processing systems, vol. 33, pp. 15 920– 15 930, 2020

    2020

  3. [11]

    Class-incremental learning: survey and performance evaluation on image classification,

    M. Masana, X. Liu, B. Twardowski, M. Menta, A. D. Bagdanov, and J. Van De Weijer, “Class-incremental learning: survey and performance evaluation on image classification,” IEEE Transactions on Pattern Anal- ysis and Machine Intelligence , vol. 45, no. 5, pp. 5513–5533, 2022

    2022

  4. [12]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685 , 2021

    2021 arXiv

  5. [13]

    Adaptformer: Adapting vision transformers for scalable visual recogni- tion,

    S. Chen, C. Ge, Z. Tong, J. Wang, Y . Song, J. Wang, and P. Luo, “Adaptformer: Adapting vision transformers for scalable visual recogni- tion,” Advances in Neural Information Processing Systems , vol. 35, pp. 16 664–16 678, 2022

    2022

  6. [14]

    The power of scale for parameter-efficient prompt tuning,

    B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter-efficient prompt tuning,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , M.-F. Moens, X. Huang, L. Specia, and S. W.-t. Yih, Eds. Online and Punta Cana, Domin...

    2021

  7. [15]

    Scaling & shifting your features: A new baseline for efficient model tuning,

    D. Lian, D. Zhou, J. Feng, and X. Wang, “Scaling & shifting your features: A new baseline for efficient model tuning,” Advances in Neural Information Processing Systems , vol. 35, pp. 109–123, 2022

    2022

  8. [16]

    Learning to prompt for continual learning,

    Z. Wang, Z. Zhang, C.-Y . Lee, H. Zhang, R. Sun, X. Ren, G. Su, V . Perot, J. Dy, and T. Pfister, “Learning to prompt for continual learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 139–149

    2022

  9. [17]

    Inflora: Interference-free low-rank adaptation for continual learning,

    Y .-S. Liang and W.-J. Li, “Inflora: Interference-free low-rank adaptation for continual learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 23 638–23 647

    2024

  10. [18]

    Beyond prompt learn- ing: Continual adapter for efficient rehearsal-free continual learning,

    X. Gao, S. Dong, Y . He, Q. Wang, and Y . Gong, “Beyond prompt learn- ing: Continual adapter for efficient rehearsal-free continual learning,” arXiv preprint arXiv:2407.10281 , 2024

    2024 arXiv

  11. [19]

    Dualprompt: Complementary prompting for rehearsal-free continual learning,

    Z. Wang, Z. Zhang, S. Ebrahimi, R. Sun, H. Zhang, C.-Y . Lee, X. Ren, G. Su, V . Perot, J. Dyet al., “Dualprompt: Complementary prompting for rehearsal-free continual learning,” in European Conference on Computer Vision. Springer, 2022, pp. 631–648

    2022

  12. [20]

    Coda-prompt: Contin- ual decomposed attention-based prompting for rehearsal-free continual learning,

    J. S. Smith, L. Karlinsky, V . Gutta, P. Cascante-Bonilla, D. Kim, A. Arbelle, R. Panda, R. Feris, and Z. Kira, “Coda-prompt: Contin- ual decomposed attention-based prompting for rehearsal-free continual learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision a...

    2023

  13. [21]

    Expandable subspace ensemble for pre-trained model-based class-incremental learning,

    D.-W. Zhou, H.-L. Sun, H.-J. Ye, and D.-C. Zhan, “Expandable subspace ensemble for pre-trained model-based class-incremental learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 23 554–23 564

    2024

  14. [22]

    Class-incremental unsupervised domain adaptation via pseudo-label distillation,

    K. Wei, X. Yang, Z. Xu, and C. Deng, “Class-incremental unsupervised domain adaptation via pseudo-label distillation,” IEEE Transactions on Image Processing, vol. 33, pp. 1188–1198, 2024

    2024

  15. [23]

    On the effectiveness of lipschitz-driven rehearsal in continual learn- ing,

    L. Bonicelli, M. Boschini, A. Porrello, C. Spampinato, and S. Calderara, “On the effectiveness of lipschitz-driven rehearsal in continual learn- ing,” Advances in Neural Information Processing Systems , vol. 35, pp. 31 886–31 901, 2022

    2022

  16. [24]

    Online continual learning with maximal interfered retrieval,

    R. Aljundi, E. Belilovsky, T. Tuytelaars, L. Charlin, M. Caccia, M. Lin, and L. Page-Caccia, “Online continual learning with maximal interfered retrieval,” Advances in neural information processing systems , vol. 32, 2019

    2019

  17. [25]

    Gradient based sample IEEE TRANSACTIONS ON IMAGE PROCESSING 12 selection for online continual learning,

    R. Aljundi, M. Lin, B. Goujaud, and Y . Bengio, “Gradient based sample IEEE TRANSACTIONS ON IMAGE PROCESSING 12 selection for online continual learning,” Advances in neural information processing systems, vol. 32, 2019

    2019

  18. [26]

    Balanced destruction- reconstruction dynamics for memory-replay class incremental learning,

    Y . Zhou, J. Yao, F. Hong, Y . Zhang, and Y . Wang, “Balanced destruction- reconstruction dynamics for memory-replay class incremental learning,” IEEE Transactions on Image Processing , 2024

    2024

  19. [27]

    Continual learning with node- importance based adaptive group sparse regularization,

    S. Jung, H. Ahn, S. Cha, and T. Moon, “Continual learning with node- importance based adaptive group sparse regularization,” Advances in neural information processing systems , vol. 33, pp. 3647–3658, 2020

    2020

  20. [28]

    Continual learning through synaptic intelligence,

    F. Zenke, B. Poole, and S. Ganguli, “Continual learning through synaptic intelligence,” in International conference on machine learning. PMLR, 2017, pp. 3987–3995

    2017

  21. [29]

    Pamk: Prototype augmented multi-teacher knowledge transfer network for continual zero-shot learning,

    J. Lu and S. Sun, “Pamk: Prototype augmented multi-teacher knowledge transfer network for continual zero-shot learning,” IEEE Transactions on Image Processing, 2024

    2024

  22. [30]

    Geodesic-aligned gradient projection for continual task learning,

    B. Qiu, H. Qiu, H. Wen, L. Wang, Y . Dai, F. Meng, Q. Wu, and H. Li, “Geodesic-aligned gradient projection for continual task learning,” IEEE Transactions on Image Processing , 2025

    2025

  23. [31]

    Compacting, picking and growing for unforgetting continual learning,

    C.-Y . Hung, C.-H. Tu, C.-E. Wu, C.-H. Chen, Y .-M. Chan, and C.- S. Chen, “Compacting, picking and growing for unforgetting continual learning,” Advances in neural information processing systems , vol. 32, 2019

    2019

  24. [32]

    Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting,

    X. Li, Y . Zhou, T. Wu, R. Socher, and C. Xiong, “Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting,” in International conference on machine learning . PMLR, 2019, pp. 3925–3934

    2019

  25. [33]

    Der: Dynamically expandable representation for class incremental learning,

    S. Yan, J. Xie, and X. He, “Der: Dynamically expandable representation for class incremental learning,” in Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition , 2021, pp. 3014–3023

    2021

  26. [34]

    Model attention expansion for few-shot class-incremental learning,

    X. Wang, Z. Ji, Y . Yu, Y . Pang, and J. Han, “Model attention expansion for few-shot class-incremental learning,” IEEE Transactions on Image Processing, 2024

    2024

  27. [35]

    Memory efficient continual learning with transformers,

    B. Ermis, G. Zappella, M. Wistuba, A. Rawal, and C. Archambeau, “Memory efficient continual learning with transformers,” Advances in Neural Information Processing Systems , vol. 35, pp. 10 629–10 642, 2022

    2022

  28. [36]

    Ekpc: Elastic knowledge preservation and compensation for class-incremental learning,

    H. Wang, D. Cheng, L. He, Y . Li, J. Li, N. Wang, and X. Gao, “Ekpc: Elastic knowledge preservation and compensation for class-incremental learning,” arXiv preprint arXiv:2506.12351 , 2025

    2025 arXiv

  29. [37]

    Stpr: Spatiotemporal preservation and routing for exemplar-free video class-incremental learning,

    H. Wang, D. Cheng, G. Li, Z. Xu, L. He, J. Li, N. Wang, and X. Gao, “Stpr: Spatiotemporal preservation and routing for exemplar-free video class-incremental learning,” arXiv preprint arXiv:2505.13997 , 2025

    2025

  30. [38]

    Mamba-cl: Optimizing selective state space model in null space for continual learning,

    D. Cheng, Y . Lu, L. He, S. Zhang, X. Yang, N. Wang, and X. Gao, “Mamba-cl: Optimizing selective state space model in null space for continual learning,” arXiv preprint arXiv:2411.15469 , 2024

    2024 arXiv

  31. [39]

    Transfer without for- getting,

    M. Boschini, L. Bonicelli, A. Porrello, G. Bellitto, M. Pennisi, S. Palazzo, C. Spampinato, and S. Calderara, “Transfer without for- getting,” in European Conference on Computer Vision. Springer, 2022, pp. 692–709

    2022

  32. [40]

    Slca: Slow learner with classifier alignment for continual learning on a pre-trained model,

    G. Zhang, L. Wang, G. Kang, L. Chen, and Y . Wei, “Slca: Slow learner with classifier alignment for continual learning on a pre-trained model,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 19 148–19 158

    2023

  33. [41]

    Consistent prompting for rehearsal- free continual learning,

    Z. Gao, J. Cen, and X. Chang, “Consistent prompting for rehearsal- free continual learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 28 463–28 473

    2024

  34. [42]

    Prompt gradient projection for continual learning,

    J. Qiao, X. Tan, C. Chen, Y . Qu, Y . Peng, Y . Xie et al. , “Prompt gradient projection for continual learning,” in The Twelfth International Conference on Learning Representations , 2023

    2023

  35. [43]

    Parameter-efficient transfer learning for nlp,

    N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for nlp,” in International conference on machine learning . PMLR, 2019, pp. 2790–2799

    2019

  36. [44]

    Prefix-tuning: Optimizing continuous prompts for generation,

    X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), ...

    2021

  37. [45]

    Visual prompt tuning,

    M. Jia, L. Tang, B.-C. Chen, C. Cardie, S. Belongie, B. Hariharan, and S.-N. Lim, “Visual prompt tuning,” in European Conference on Computer Vision. Springer, 2022, pp. 709–727

    2022

  38. [46]

    Disentangled prompt representation for domain generalization,

    D. Cheng, Z. Xu, X. Jiang, N. Wang, D. Li, and X. Gao, “Disentangled prompt representation for domain generalization,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 23 595–23 604

    2024

  39. [47]

    Adversarial domain prompt tuning and generation for single domain generalization,

    Z. Xu, D. Cheng, X. Jiang, N. Wang, D. Li, and X. Gao, “Adversarial domain prompt tuning and generation for single domain generalization,” in Proceedings of the Computer Vision and Pattern Recognition Confer- ence, 2025, pp. 18 584–18 595

    2025

  40. [48]

    Learning domain invariant prompt for vision-language models,

    C. Zhao, Y . Wang, X. Jiang, Y . Shen, K. Song, D. Li, and D. Miao, “Learning domain invariant prompt for vision-language models,” IEEE Transactions on Image Processing , vol. 33, pp. 1348–1360, 2024

    2024

  41. [49]

    Adapting vision-language models via learning to inject knowledge,

    S. Xuan, M. Yang, and S. Zhang, “Adapting vision-language models via learning to inject knowledge,” IEEE Transactions on Image Processing, 2024

    2024

  42. [50]

    Adaptive mixtures of local experts,

    R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton, “Adaptive mixtures of local experts,” Neural computation, vol. 3, no. 1, pp. 79–87, 1991

    1991

  43. [51]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929 , 2020

    2010 arXiv

  44. [52]

    Training networks in null space of feature covariance for continual learning,

    S. Wang, X. Li, J. Sun, and Z. Xu, “Training networks in null space of feature covariance for continual learning,” in Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition , 2021, pp. 184–193

    2021

  45. [53]

    A unified continual learning framework with general parameter-efficient tuning,

    Q. Gao, C. Zhao, Y . Sun, T. Xi, G. Zhang, B. Ghanem, and J. Zhang, “A unified continual learning framework with general parameter-efficient tuning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 11 483–11 493

    2023

  46. [54]

    Revisiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need,

    D.-W. Zhou, Z.-W. Cai, H.-J. Ye, D.-C. Zhan, and Z. Liu, “Revisiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need,” International Journal of Computer Vision , pp. 1–21, 2024

    2024

  47. [55]

    Class-incremental learning with clip: Adaptive representation adjustment and parameter fusion,

    L. Huang, X. Cao, H. Lu, and X. Liu, “Class-incremental learning with clip: Adaptive representation adjustment and parameter fusion,” 2024. [Online]. Available: https://arxiv.org/abs/2407.14143

    2024 arXiv

  48. [56]

    Vector quantization prompting for continual learning,

    L. Jiao, Q. Lai, Y . Li, and Q. Xu, “Vector quantization prompting for continual learning,” arXiv preprint arXiv:2410.20444 , 2024

    2024 arXiv

  49. [57]

    Dynamic integration of task-specific adapters for class incremental learning,

    J. Li, S. Wang, B. Qian, Y . He, X. Wei, Q. Wang, and Y . Gong, “Dynamic integration of task-specific adapters for class incremental learning,” in Proceedings of the Computer Vision and Pattern Recognition Confer- ence, 2025, pp. 30 545–30 555

    2025

  50. [58]

    Adapter merging with centroid prototype mapping for scalable class-incremental learning,

    T. Fukuda, H. Kera, and K. Kawamoto, “Adapter merging with centroid prototype mapping for scalable class-incremental learning,” in Proceed- ings of the Computer Vision and Pattern Recognition Conference , 2025, pp. 4884–4893

    2025

  51. [59]

    Self-expansion of pre-trained models with mixture of adapters for continual learning,

    H. Wang, H. Lu, L. Yao, and D. Gong, “Self-expansion of pre-trained models with mixture of adapters for continual learning,” in Proceedings of the Computer Vision and Pattern Recognition Conference , 2025, pp. 10 087–10 098

    2025

  52. [60]

    Lora subtraction for drift-resistant space in exemplar-free continual learning,

    X. Liu and X. Chang, “Lora subtraction for drift-resistant space in exemplar-free continual learning,” in Proceedings of the Computer Vision and Pattern Recognition Conference , 2025, pp. 15 308–15 318

    2025

  53. [61]

    Cl-lora: Continual low-rank adaptation for rehearsal-free class-incremental learning,

    J. He, Z. Duan, and F. Zhu, “Cl-lora: Continual low-rank adaptation for rehearsal-free class-incremental learning,” in Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 30 534– 30 544

    2025

  54. [62]

    OVOR: Oneprompt with virtual outlier regularization for rehearsal-free class-incremental learning,

    W.-C. Huang, C.-F. Chen, and H. Hsu, “OVOR: Oneprompt with virtual outlier regularization for rehearsal-free class-incremental learning,” in The Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https://openreview.net/ forum?id=FbuyDzZTPt

    2024

  55. [63]

    Convolutional prompting meets language models for continual learning,

    A. Roy, R. Moulick, V . K. Verma, S. Ghosh, and A. Das, “Convolutional prompting meets language models for continual learning,” in Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 23 616–23 626

    2024

  56. [64]

    The many faces of robustness: A critical analysis of out-of-distribution generalization,

    D. Hendrycks, S. Basart, N. Mu, S. Kadavath, F. Wang, E. Dorundo, R. Desai, T. Zhu, S. Parajuli, M. Guo et al. , “The many faces of robustness: A critical analysis of out-of-distribution generalization,” in Proceedings of the IEEE/CVF international conference on computer visio...

    2021

  57. [65]

    Natural adversarial examples,

    D. Hendrycks, K. Zhao, S. Basart, J. Steinhardt, and D. Song, “Natural adversarial examples,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2021, pp. 15 262–15 271

    2021

  58. [66]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky, G. Hinton et al. , “Learning multiple layers of features from tiny images,” 2009

    2009

  59. [67]

    Moment matching for multi-source domain adaptation,

    X. Peng, Q. Bai, X. Xia, Z. Huang, K. Saenko, and B. Wang, “Moment matching for multi-source domain adaptation,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 1406– 1415

    2019

  60. [68]

    Imagenet large IEEE TRANSACTIONS ON IMAGE PROCESSING 13 scale visual recognition challenge,

    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein et al., “Imagenet large IEEE TRANSACTIONS ON IMAGE PROCESSING 13 scale visual recognition challenge,” International journal of computer vision, vol. 115, pp. 211–252, 2015

    2015

  61. [69]

    Isolation and impartial aggregation: A paradigm of incremental learning without interference,

    Y . Wang, Z. Ma, Z. Huang, Y . Wang, Z. Su, and X. Hong, “Isolation and impartial aggregation: A paradigm of incremental learning without interference,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 8, 2023, pp. 10 209–10 217

    2023

  62. [70]

    Weighted ensemble models are strong continual learners,

    I. E. Marouf, S. Roy, E. Tartaglione, and S. Lathuili `ere, “Weighted ensemble models are strong continual learners,” arXiv preprint arXiv:2312.08977, 2023

    2023 arXiv

  63. [71]

    The caltech-ucsd birds-200-2011 dataset,

    C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie, “The caltech-ucsd birds-200-2011 dataset,” 2011

    2011

  64. [72]

    3d object representations for fine-grained categorization,

    J. Krause, M. Stark, J. Deng, and L. Fei-Fei, “3d object representations for fine-grained categorization,” in Proceedings of the IEEE interna- tional conference on computer vision workshops , 2013, pp. 554–561

    2013

  65. [73]

    Generating instance-level prompts for rehearsal-free continual learning,

    D. Jung, D. Han, J. Bang, and H. Song, “Generating instance-level prompts for rehearsal-free continual learning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 11 847–11 857

    2023

  66. [74]

    Continual diffusion: Continual customization of text-to-image diffusion with c-lora,

    J. S. Smith, Y .-C. Hsu, L. Zhang, T. Hua, Z. Kira, Y . Shen, and H. Jin, “Continual diffusion: Continual customization of text-to-image diffusion with c-lora,” arXiv preprint arXiv:2304.06027 , 2023

    2023 arXiv

  67. [75]

    Hierarchical decomposition of prompt-based continual learning: Rethinking obscured sub-optimality,

    L. Wang, J. Xie, X. Zhang, M. Huang, H. Su, and J. Zhu, “Hierarchical decomposition of prompt-based continual learning: Rethinking obscured sub-optimality,” Advances in Neural Information Processing Systems , vol. 36, 2024

    2024

  68. [76]

    Emerging properties in self-supervised vision transformers,

    M. Caron, H. Touvron, I. Misra, H. J ´egou, J. Mairal, P. Bojanowski, and A. Joulin, “Emerging properties in self-supervised vision transformers,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 9650–9660

    2021

  69. [77]

    ibot: Image bert pre-training with online tokenizer,

    J. Zhou, C. Wei, H. Wang, W. Shen, C. Xie, A. Yuille, and T. Kong, “ibot: Image bert pre-training with online tokenizer,” arXiv preprint arXiv:2111.07832, 2021. Lingfeng He received the B.Sc. degree from Xidian University, Xi’an, China, in 2023. He is currently pursuing his M....

    2021 arXiv

  70. [2015]

    Student with the University of Technology Sydney, Australia

    From September 2011 to September 2013, he was a Visiting Ph.D. Student with the University of Technology Sydney, Australia. He is currently a Pro- fessor with the State Key Laboratory of Integrated Services Networks, Xidian University. Xinbo Gao (M’02-SM’07) received the B.Eng...

    2011

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.