Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Navigating Semantic Drift in Task-Agnostic Class-Incremental Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A method that estimates how far each old class mean has drifted in feature space, then calibrates class covariances against the previous network, reports the highest final and average accuracies among compared rehearsal-free…

desk verdict Empirically promising, mechanistically sloppy: the mean-shift proxy is unvalidated and the covariance loss does not calibrate old-class covariances, but the paper deserves a serious referee. read the letter →

arxiv 2502.07560 v2 pith:QKWT2S4Y submitted 2025-02-11 cs.CV

Fangwen Wu , Lechao Cheng , Shengeng Tang , Xiaofeng Zhu , Chaowei Fang , Dingwen Zhang , Meng Wang This is my paper · ORCID
classification cs.CV
keywords class-incrementallearningcatastrophicforgettingsemanticdriftmeanshiftcompensationcovariancecalibrationMahalanobisdistancelow-rankadaptationparameter-efficientfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Class-incremental learning asks a model to keep learning new object classes without forgetting old ones, and the hardest version denies the model the task ID at test time and gives it no stored examples from earlier classes. This paper claims that what actually degrades old-class features during incremental fine-tuning is a drift in their first and second moments: the class means move and the covariance structure changes, and correcting both moments is enough to largely undo the forgetting. The paper builds a training objective with a mean-shift compensation term that estimates every old class mean's new position from the embedding drift of current-task samples, plus a Mahalanobis-distance covariance calibration term that pushes current-task intra-class scatter to match the old network's scatter. On ImageNet-R, ImageNet-A, CUB-200 and CIFAR-100, with a frozen ViT backbone and learnable low-rank adapters, the reported final-task and average accuracies exceed the compared prompt-, adapter- and LoRA-based methods. If the claim holds, the lesson is that forgetting in this setting is mostly a moment-matching problem rather than a representational collapse.

What carries the argument

Semantic drift calibration: a pipeline that stores each class's mean $\mu_c$ and covariance $\Sigma_c$ after every task, then (1) compensates mean shift using $\hat{\Delta}\mu_c = \sum_i w_i \Delta\phi_\theta(x_i) / \sum_i w_i$ with Gaussian weights $w_i = \exp(-\|\phi_\theta^{t-1}(x_i) - \mu_c^{t-1}\|_2^2 / 2\sigma^2)$, and (2) calibrates covariance by minimizing the absolute difference of pairwise Mahalanobis distances, computed with the old covariance $\Sigma_c^{t-1}$, between embeddings from the old and current networks. The patch-token self-distillation term and the classifier re-fitting on Gaussian samples are auxiliary stabilizers. The mechanism works because it directly targets the two moments the paper identifies as drifting.

What would settle it

Compute the true old-class means on held-out old data after training a new task and compare them to the paper's estimated means: if the average error grows systematically with task distance, the mean-shift compensation is the weak link; separately, measure the Frobenius distance between old and new class covariance matrices while the covariance loss decreases, to check whether the loss actually aligns covariances or only matches pairwise distances.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the feature distribution of previously learned classes, when re-evaluated under a network updated on new tasks, shifts mainly in its mean and covariance, and this 'semantic drift' is the primary cause of the accuracy drop on old classes. Accordingly, the authors propose a semantic drift calibration method: after each task, the mean of every old class is corrected by a Gaussian-weighted average of per-sample embedding shifts, where samples closer to the old mean are trusted more, and during training a Mahalanobis-distance loss constrains the pairwise distances among current-task embeddings to match those produced by the frozen old network under the old covariance. Classifier heads are then re-fit on samples drawn from the calibrated Gaussian statistics for each class. The paper reports that this combination outperforms existing rehearsal-free, task-agnostic class-incremental methods on four benchmarks, with the largest gains on datasets whose train and test distributions are farthest from the pretraining data.

Load-bearing premise

The load-bearing premise is that the drift of old class means, which cannot be observed because old data is gone, can be estimated from the drift of embeddings of current-task images weighted by their nearness to the old mean; if current-task samples are not representative of where old-class features moved, the mean compensation is wrong.

Editorial extensions

If this is right

  • If the central claim is correct, an exemplar-free class-incremental model can reduce forgetting without storing old images: storing each class's mean and covariance is enough to calibrate the drift.
  • The results imply that the choice among task-specific, task-shared, and hybrid low-rank adapter structures has only a minor effect once the feature moments are calibrated, because Table 4 reports small differences across the three designs.
  • The method predicts that the same moment-calibration recipe should transfer to other parameter-efficient backbones such as prompts or adapters, since the drift is stated in terms of feature statistics rather than LoRA-specific mechanics.
  • The benchmark pattern suggests the gains should be largest when the current task's data are semantically distant from old classes, and smallest when the pretraining domain already covers the target data, which matches the reported ordering of ImageNet-R and ImageNet-A versus CIFAR-100 and CUB-200.
  • Storing calibrated class statistics also makes the final classifier cheap to re-fit after every task, so the approach is compatible with post-hoc head alignment rather than requiring task-id prediction at inference time.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Read literally, the covariance term in Eq. 10 matches pairwise Mahalanobis distances under the fixed old covariance, so the loss does not directly minimize a matrix distance between covariance matrices; the paper's description of it as aligning covariance matrices is looser than the objective it actually implements.
  • The mean-shift estimator assumes the per-sample embedding drift of current-task images is a faithful proxy for the drift of old-class means, an assumption that is most credible when new and old classes share low-level visual structure and least credible for a task with very different semantics.
  • An implicit extension is to apply mean compensation at inference time only, shifting stored prototypes by the estimated drift without retraining the classifier, which would isolate the contribution of mean shift from classifier re-fitting.
  • A natural test of the moment hypothesis would be to replace the Mahalanobis covariance term with a whitening transform or a direct eigenvalue alignment, to see whether the second moment is genuinely the active mechanism or whether the gain comes from the distance regularization alone.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies task-agnostic class-incremental learning with a pre-trained ViT backbone and LoRA adapters. It claims that the distribution gap between old and new classes is primarily driven by differences in the mean and covariance of feature embeddings, and proposes a semantic drift calibration method consisting of mean shift compensation (MSC), a Mahalanobis-distance covariance calibration loss (CC), and patch-token self-distillation. The method is evaluated on ImageNet-R, ImageNet-A, CUB-200, and CIFAR-100 under several task splits, reporting state-of-the-art final and average accuracies. Ablations show each component contributes to the final performance, and source code is released.

Significance. If the claims were fully substantiated, the paper would offer a simple, exemplar-free mechanism for reducing catastrophic forgetting in class-incremental learning with pre-trained models, a topic of active interest. The empirical evaluation is broad (four datasets, three task splits, three seeds) and the ablations demonstrate consistent gains from each proposed component; the public code release is a strength. However, the paper's central mechanism is not established as described: the mean shift estimator is inconsistent with the exemplar-free setting, and the covariance calibration loss does not align old-class covariances. The empirical results remain valuable, but the methodological narrative needs to be corrected and supported by additional validation before the claims can be accepted.

major comments (3)
  1. [Sec. 3.4.1, Eqs. (6)-(7), Algorithm 1] The mean shift compensation formula in Eqs. (6)-(7) is written as a weighted average over samples xi that 'belong to class c'. In the exemplar-free setting, no samples of previously learned classes are available at task t, yet Algorithm 1 applies this compensation to every old class c in the union of previous label sets. The sum can therefore only run over current-task samples, with weights determined by the proximity of those samples to the old class mean. This is not merely a notation issue: the proxy assumption that embedding shifts of current-task (new-class) images predict the drift of old-class means is unvalidated and is most brittle when the current task is semantically far from the old class, since the Gaussian kernel in Eq. (7) decays exponentially with squared embedding distance. The paper should explicitly state the sampling procedure and provide evidence for the proxy, for example by comparing estimated shifts with true shifts on a held-out subset of old data or by measuring old-class accuracy with and without MSC. Without such validation, the claim that 'mean shifts for all learned classes' are captured is not supported.
  2. [Sec. 3.4.2, Eq. (10)] The covariance calibration loss Lcov in Eq. (10) is summed over current-task classes c in C^t and uses pairs of current-task samples. It therefore imposes a consistency between old and current networks on the representations of new classes only; it does not update or align the stored covariance matrix Sigma^{t-1}_c of any previously learned class. Consequently, the abstract's statement that the method 'align[s] class-specific embedding covariances between old and current networks' for all classes is not supported by the presented loss. The authors should either show how old-class covariances are calibrated (for instance, by applying an estimated drift transformation to Sigma^{t-1}_c) or revise the claim to describe Lcov as a regularizer for current-task representations. This distinction matters because the paper's central narrative is that both first- and second-order moments of old-class distributions are calibrated.
  3. [Abstract, Sec. 1, Sec. 4.3] The claim that the gap in feature distribution between novel and existing tasks is 'primarily driven by differences in mean and covariance moments' is not quantified anywhere in the manuscript. The ablation study in Table 3 shows that adding MSC and CC improves accuracy, but it does not measure the distribution gap or decompose it into mean and covariance contributions. Without such an analysis, the 'primarily driven' assertion is an interpretation of empirical gains rather than a demonstrated finding. Adding a diagnostic study (for example, measuring moment-wise mismatches or the Wasserstein distance between old-class feature distributions before and after calibration) would substantiate the claim; alternatively, the claim should be softened to state that mean and covariance constraints improve performance.
minor comments (5)
  1. [Sec. 4.1.2] The Gaussian kernel width sigma in Eq. (7) and the number of sampled features sc in Eq. (11) are not reported; these hyperparameters are needed for reproducibility, since the behavior of the weighting in Eq. (7) depends critically on sigma.
  2. [Figure 2] In Figure 2, the loss label 'L_{siatill}' appears to be a typo for 'L_distill'; please correct it.
  3. [Sec. 3.3, Table 4] The LoRA structure names G-LoRA, E-LoRA, and Hydra-LoRA used in Table 4 are not explicitly defined in the text; the mapping to the task-shared, task-specific, and hybrid structures described in Sec. 3.3 should be stated.
  4. [Sec. 3.5] In the classifier alignment stage, sampling from N(mu_c, Sigma_c) for old classes uses the stored covariance Sigma_c, which is not updated by the covariance calibration loss. This should be acknowledged, as it means that second-order moment calibration for old classes is not actually reflected in the replayed samples.
  5. [Table 1] Several entries in Table 1 are marked '-'; the caption says missing implementations are denoted this way, but it would help readers to know whether the '-' entries are due to unavailable code or because the method was not applicable to that dataset.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor framing-level circularity: semantic drift is defined as mean/covariance shift and then 'revealed' to be driven by mean/covariance moments; the core benchmark evaluation is otherwise self-contained.

  1. self definitional [Abstract and Section 3.4 (Semantic Drift), Eqs. 3-4]
    "Both the mean and variance of feature distributions of the old classes change. This phenomenon is referred to as semantic drift. ... our study reveals that the gap in feature distribution between novel and existing tasks is primarily driven by differences in mean and covariance moments."

    The paper first defines 'semantic drift' as a change in the mean and variance of old-class feature distributions, then presents as an empirical finding that the distribution gap is 'primarily driven by differences in mean and covariance moments.' The claimed insight is entailed by the definition rather than derived from independent measurements, so the motivation for the mean-shift and covariance-calibration modules restates the setup. This is a framing-level circularity only: no evaluation number is produced by construction, no fitted constant is renamed as a prediction, and the benchmark results remain independent evidence for the method.

full rationale

No significant derivational or statistical circularity was found. The method is tested on held-out test sets of four public benchmarks, and the reported accuracies are not obtained by fitting parameters to the evaluation targets; the hand-tuned hyperparameters (distillation weight lambda = 0.4, LoRA rank r = 32, angular scale s = 20) are standard and stated to be chosen by sensitivity analysis. The mean-shift compensation of Eqs. 6-7 is explicitly inherited from prior work (Yu et al., 2020) rather than being re-derived from the paper's own outputs, and the covariance-calibration loss of Eq. 10 constrains current-task embeddings using the old network as a reference, which is a mechanism choice rather than a circular definition. The paper does cite works co-authored by its own authors (Wang et al., 2024c and 2024d), but those citations support auxiliary components such as patch-token distillation and prompt tuning and are not load-bearing for the central claim. The one identifiable circular element is the self-definitional framing that 'semantic drift' is defined as mean and covariance change and then 'revealed' to be driven by mean and covariance moments; this affects the presentation of the motivation, not the derivation of the results. There is no self-citation uniqueness theorem, no ansatz smuggled in solely via the authors' own prior work, and no known result merely renamed as organization. Overall, the core contribution is externally evaluated and the central performance claims are self-contained.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method rests on standard empirical assumptions in pre-trained model based CIL: Gaussianity of class embeddings, transferability of drift estimates across classes, and stability of pretrained feature statistics. Free parameters are mostly standard hyperparameters, but sigma is unreported and sc is unspecified.

free parameters (3)
  • sigma (Gaussian kernel width in Eq. 7) = not reported
    Controls the proximity weighting in mean shift estimation; no default or sensitivity analysis is given in the paper.
  • lambda (self-distillation loss weight) = 0.4
    Set through sensitivity analysis; it weights the Ldistill term in the total loss.
  • sc (number of sampled features per class for classifier alignment) = not specified
    Classifier retraining (Eq. 11) depends on this unstated sampling count, which affects the alignment procedure.
assumptions (3)
  • domain assumption Class embeddings follow a Gaussian distribution N(mu_c, Sigma_c)
    Section 3.5 relies on this Gaussian assumption to sample features for classifier retraining, citing Zhang et al. (2023).
  • domain assumption The drift measured on current-task samples can approximate the drift of old class means
    Inherited from Yu et al. (2020), used in Eqs. 6-7 to compensate old class means without old data, but not empirically validated for dissimilar tasks.
  • domain assumption Pre-trained ViT features are stable enough that per-class mean and covariance statistics are meaningful across incremental sessions
    Throughout Section 3, the method stores and updates mean and covariance statistics of frozen pretrained features and treats them as reliable summaries of class distributions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Navigating Semantic Drift in Task-Agnostic Class-Incremental Learning." pith.science (2026). https://pith.science/paper/QKWT2S4Y

@misc{pith2026250207560,
  author       = {Pith},
  title        = {Pith review of: Navigating Semantic Drift in Task-Agnostic Class-Incremental Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QKWT2S4Y}},
  note         = {Machine review of arXiv:2502.07560}
}
read the original abstract

Class-incremental learning (CIL) seeks to enable a model to sequentially learn new classes while retaining knowledge of previously learned ones. Balancing flexibility and stability remains a significant challenge, particularly when the task ID is unknown. To address this, our study reveals that the gap in feature distribution between novel and existing tasks is primarily driven by differences in mean and covariance moments. Building on this insight, we propose a novel semantic drift calibration method that incorporates mean shift compensation and covariance calibration. Specifically, we calculate each class's mean by averaging its sample embeddings and estimate task shifts using weighted embedding changes based on their proximity to the previous mean, effectively capturing mean shifts for all learned classes with each new task. We also apply Mahalanobis distance constraint for covariance calibration, aligning class-specific embedding covariances between old and current networks to mitigate the covariance shift. Additionally, we integrate a feature-level self-distillation approach to enhance generalization. Comprehensive experiments on commonly used datasets demonstrate the effectiveness of our approach. The source code is available at \href{https://github.com/fwu11/MACIL.git}{https://github.com/fwu11/MACIL.git}.

Figures

Figures reproduced from arXiv: 2502.07560 by the authors.

Figure 1
Figure 1. As new tasks are learned, the categories from previously tasks in the latest updated model continuously experience shifts in their means and variances, referred to as (a) Semantic Drift. In this paper, we calibrate such semantic drift by applying explicit mean shift compensation and implicit variance constraints (b). et al., 2019). This capability has broad applications in fields such as computer vision, robotics, a… view at source ↗
Figure 2
Figure 2. Illustration of our method at task t. The feature extractor at task t uses a frozen pre-trained ViT backbone with learnable LoRA modules. The output class tokens (yellow) are passed through a classifier to compute the classification loss Lcls, and the mean and covariance of each class are stored for each session. During training, class tokens (yellow and blue) are used to align class distributions via a covariance c… view at source ↗
Figure 3
Figure 3. The performance of each learning session under different settings of ImageNet-R and CIFAR100. All methods are initialized with ViT-B/16-IN21k. These curves are plotted by calculating the average performance across three different seeds. Additionally, we evaluate the performance of our method on both longer task sequences (20 tasks) and shorter task sequences (5 tasks) for CIFAR-100 and ImageNet-R, as reported in Tab… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Circumventing Safety Alignment in Large Language Models Through Embedding Space Toxicity Attenuation

    cs.CL 2025-07 conditional novelty 6.0 of 10

    ETTA bypasses LLM safety refusals by learning a linear toxicity direction in the embedding space and attenuating it in word embeddings at inference time.

Reference graph

Works this paper leans on

60 extracted references · 37 canonical work pages · cited by 1 Pith paper

  1. [1]

    Co2l: Contrastive continual learning

    Cha, H., Lee, J., and Shin, J. Co2l: Contrastive continual learning. In Proceedings of the IEEE/CVF International conference on computer vision, pp.\ 9516--9525, 2021

    2021

  2. [2]

    Efficient lifelong learning with a-gem

    Chaudhry, A., Ranzato, M., Rohrbach, M., and Elhoseiny, M. Efficient lifelong learning with a-gem. In ICLR, 2019

    work page 2019

  3. [3]

    Adaptformer: Adapting vision transformers for scalable visual recognition

    Chen, S., Ge, C., Tong, Z., Wang, J., Song, Y., Wang, J., and Luo, P. Adaptformer: Adapting vision transformers for scalable visual recognition. Advances in Neural Information Processing Systems, 35: 0 16664--16678, 2022

    2022

  4. [4]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=YicbFdNTTy

    2021

  5. [5]

    Orthogonal gradient descent for continual learning

    Farajtabar, M., Azizan, N., Mott, A., and Li, A. Orthogonal gradient descent for continual learning. In International Conference on Artificial Intelligence and Statistics, pp.\ 3762--3773. PMLR, 2020

    work page 2020

  6. [6]

    Beyond prompt learning: Continual adapter for efficient rehearsal-free continual learning

    Gao, X., Dong, S., He, Y., Wang, Q., and Gong, Y. Beyond prompt learning: Continual adapter for efficient rehearsal-free continual learning. In European Conference on Computer Vision, 2024 a

    work page 2024

  7. [7]

    Consistent prompting for rehearsal-free continual learning

    Gao, Z., Cen, J., and Chang, X. Consistent prompting for rehearsal-free continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 28463--28473, 2024 b

    work page 2024

  8. [8]

    D., Twardowski, B., and van de Weijer, J

    Gomez-Villa, A., Goswami, D., Wang, K., Bagdanov, A. D., Twardowski, B., and van de Weijer, J. Exemplar-free continual representation learning via learnable drift compensation. In European Conference on Computer Vision, pp.\ 473--490. Springer, 2024

    work page 2024

Show all 60 references
  1. [9]

    Resurrecting old classes with new data for exemplar-free continual learning

    Goswami, D., Soutif-Cormerais, A., Liu, Y., Kamath, S., Twardowski, B., van de Weijer, J., et al. Resurrecting old classes with new data for exemplar-free continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 28525--28534, 2024

    2024

  2. [10]

    Rethinking imagenet pre-training

    He, K., Girshick, R., and Doll \'a r, P. Rethinking imagenet pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 4918--4927, 2019

    2019

  3. [11]

    The many faces of robustness: A critical analysis of out-of-distribution generalization

    Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., Guo, M., et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF international conference on compute...

    2021

  4. [12]

    Natural adversarial examples

    Hendrycks, D., Zhao, K., Basart, S., Steinhardt, J., and Song, D. Natural adversarial examples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 15262--15271, 2021 b

    2021

  5. [13]

    Parameter-efficient transfer learning for nlp

    Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Gesmundo, A., Attariyan, M., and Gelly, S. Parameter-efficient transfer learning for nlp. In International conference on machine learning, pp.\ 2790--2799. PMLR, 2019

    2019

  6. [14]

    J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W

    Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lo RA : Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=nZeVKeeFYf9

    2022

  7. [15]

    OVOR : Oneprompt with virtual outlier regularization for rehearsal-free class-incremental learning

    Huang, W.-C., Chen, C.-F., and Hsu, H. OVOR : Oneprompt with virtual outlier regularization for rehearsal-free class-incremental learning. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=FbuyDzZTPt

    2024

  8. [16]

    Visual prompt tuning

    Jia, M., Tang, L., Chen, B.-C., Cardie, C., Belongie, S., Hariharan, B., and Lim, S.-N. Visual prompt tuning. In European Conference on Computer Vision, pp.\ 709--727. Springer, 2022

    2022

  9. [17]

    Vector quantization prompting for continual learning

    Jiao, L., Lai, Q., Li, Y., and Xu, Q. Vector quantization prompting for continual learning. NeurIPS, 2024

    2024

  10. [18]

    One-stage prompt-based continual learning

    Kim, Y., Li, Y., and Panda, P. One-stage prompt-based continual learning. In European Conference on Computer Vision, pp.\ 163--179. Springer, 2024

    2024

  11. [19]

    A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al

    Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114 0 (13): 0 3521--3526, 2017

    2017

  12. [20]

    Learning multiple layers of features from tiny images

    Krizhevsky, A. Learning multiple layers of features from tiny images. In Technical report, 2009. URL https://api.semanticscholar.org/CorpusID:18268744

    2009

  13. [21]

    R., Song, X., Ma, Z., He, Y., Gong, Y., Qi, Y., and Wei, X

    Kurniawan, M. R., Song, X., Ma, Z., He, Y., Gong, Y., Qi, Y., and Wei, X. Evolving parameterized prompt memory for continual learning. Proceedings of the AAAI Conference on Artificial Intelligence, 38 0 (12): 0 13301--13309, Mar. 2024. doi:10.1609/aaai.v38i12.29231. URL https:...

    2024 doi

  14. [22]

    The power of scale for parameter-efficient prompt tuning

    Lester, B., Al-Rfou, R., and Constant, N. The power of scale for parameter-efficient prompt tuning. In Moens, M.-F., Huang, X., Specia, L., and Yih, S. W.-t. (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp.\ 3045--3059, Online...

    2021 doi

  15. [23]

    Dynamic integration of task-specific adapters for class incremental learning

    Li, J., Wang, S., Qian, B., He, Y., Wei, X., and Gong, Y. Dynamic integration of task-specific adapters for class incremental learning. arXiv preprint arXiv:2409.14983, 2024

    2024 arXiv

  16. [24]

    and Hoiem, D

    Li, Z. and Hoiem, D. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40 0 (12): 0 2935--2947, 2017

    2017

  17. [25]

    and Li, W.-J

    Liang, Y.-S. and Li, W.-J. Inflora: Interference-free low-rank adaptation for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 23638--23647, 2024

    2024

  18. [26]

    Rmm: Reinforced memory management for class-incremental learning

    Liu, Y., Schiele, B., and Sun, Q. Rmm: Reinforced memory management for class-incremental learning. Advances in Neural Information Processing Systems, 34: 0 3478--3490, 2021

    2021

  19. [27]

    and Ranzato, M

    Lopez-Paz, D. and Ranzato, M. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017

    2017

  20. [28]

    Visual prompt tuning in null space for continual learning

    Lu, Y., Zhang, S., Cheng, D., Xing, Y., Wang, N., Wang, P., and Zhang, Y. Visual prompt tuning in null space for continual learning. NeurIPS, 2024

    2024

  21. [29]

    Mahalanobis, P. C. On the generalized distance in statistics. Proceedings of the National Institute of Sciences of India, 2: 0 49--55, 1936

    1936

  22. [30]

    Ran PAC : Random projections and pre-trained models for continual learning

    McDonnell, M., Gong, D., Parvaneh, A., Abbasnejad, E., and van den Hengel, A. Ran PAC : Random projections and pre-trained models for continual learning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=aec58UfBzA

    2023

  23. [31]

    I., Kemker, R., Part, J

    Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., and Wermter, S. Continual lifelong learning with neural networks: A review. Neural networks, 113: 0 54--71, 2019

    2019

  24. [32]

    Peng, C., Zhao, K., Wang, T., Li, M., and Lovell, B. C. Few-shot class-incremental learning from an open-set perspective. In European Conference on Computer Vision, pp.\ 382--397. Springer, 2022

    2022

  25. [33]

    Dualnet: Continual learning, fast and slow

    Pham, Q., Liu, C., and Hoi, S. Dualnet: Continual learning, fast and slow. Advances in Neural Information Processing Systems, 34, 2021

    2021

  26. [34]

    Rebuffi, S.-A., Kolesnikov, A., Sperl, G., and Lampert, C. H. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp.\ 2001--2010, 2017

    2001

  27. [35]

    Learning to learn without forgetting by maximizing transfer and minimizing interference

    Riemer, M., Cases, I., Ajemian, R., Liu, M., Rish, I., Tu, Y., and Tesauro, G. Learning to learn without forgetting by maximizing transfer and minimizing interference. In In International Conference on Learning Representations (ICLR), 2019

    2019

  28. [36]

    Imagenet large scale visual recognition challenge

    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115: 0 211--252, 2015

    2015

  29. [37]

    Gradient projection memory for continual learning

    Saha, G., Garg, I., and Roy, K. Gradient projection memory for continual learning. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=3AOj0RCNC2

    2021

  30. [38]

    K., Kim, J., and Kim, J

    Shin, H., Lee, J. K., Kim, J., and Kim, J. Continual learning with deep generative replay. Advances in neural information processing systems, 30, 2017

    2017

  31. [39]

    S., Karlinsky, L., Gutta, V., Cascante-Bonilla, P., Kim, D., Arbelle, A., Panda, R., Feris, R., and Kira, Z

    Smith, J. S., Karlinsky, L., Gutta, V., Cascante-Bonilla, P., Kim, D., Arbelle, A., Panda, R., Feris, R., and Kira, Z. Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning. In Proceedings of the IEEE/CVF Conference on Computer Visio...

    2023

  32. [40]

    Mos: Model surgery for pre-trained model-based class-incremental learning

    Sun, H.-L., Zhou, D.-W., Zhao, H., Gan, L., Zhan, D.-C., and Ye, H.-J. Mos: Model surgery for pre-trained model-based class-incremental learning. In AAAI, 2025

    2025

  33. [41]

    Semantically-shifted incremental adapter-tuning is a continual vitransformer

    Tan, Y., Zhou, Q., Xiang, X., Wang, K., Wu, Y., and Li, Y. Semantically-shifted incremental adapter-tuning is a continual vitransformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 23252--23262, 2024

    2024

  34. [42]

    Hydralora: An asymmetric lora architecture for efficient fine-tuning

    Tian, C., Shi, Z., Guo, Z., Li, L., and Xu, C. Hydralora: An asymmetric lora architecture for efficient fine-tuning. In Advances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  35. [43]

    The caltech-ucsd birds-200-2011 dataset

    Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. The caltech-ucsd birds-200-2011 dataset. Technical Report CNS-TR-2011-001, California Institute of Technology, 2011

    2011

  36. [44]

    Hierarchical decomposition of prompt-based continual learning: Rethinking obscured sub-optimality

    Wang, L., Xie, J., Zhang, X., Huang, M., Su, H., and Zhu, J. Hierarchical decomposition of prompt-based continual learning: Rethinking obscured sub-optimality. Advances in Neural Information Processing Systems, 36, 2024 a

    2024

  37. [45]

    A comprehensive survey of continual learning: theory, method and application

    Wang, L., Zhang, X., Su, H., and Zhu, J. A comprehensive survey of continual learning: theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024 b

    2024

  38. [46]

    Training networks in null space of feature covariance for continual learning

    Wang, S., Li, X., Sun, J., and Xu, Z. Training networks in null space of feature covariance for continual learning. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pp.\ 184--193, 2021

    2021

  39. [47]

    S-prompts learning with pre-trained transformers: An occam’s razor for domain incremental learning

    Wang, Y., Huang, Z., and Hong, X. S-prompts learning with pre-trained transformers: An occam’s razor for domain incremental learning. Advances in Neural Information Processing Systems, 35: 0 5682--5695, 2022 a

    2022

  40. [48]

    Improving knowledge distillation via regularizing feature direction and norm

    Wang, Y., Cheng, L., Duan, M., Wang, Y., Feng, Z., and Kong, S. Improving knowledge distillation via regularizing feature direction and norm. In European Conference on Computer Vision, pp.\ 20--37. Springer Nature Switzerland Cham, 2024 c

    2024

  41. [49]

    Revisiting the power of prompt for visual tuning

    Wang, Y., Cheng, L., Fang, C., Zhang, D., Duan, M., and Wang, M. Revisiting the power of prompt for visual tuning. arXiv preprint arXiv:2402.02382, 2024 d

    2024 arXiv

  42. [50]

    Dualprompt: Complementary prompting for rehearsal-free continual learning

    Wang, Z., Zhang, Z., Ebrahimi, S., Sun, R., Zhang, H., Lee, C.-Y., Ren, X., Su, G., Perot, V., Dy, J., et al. Dualprompt: Complementary prompting for rehearsal-free continual learning. In European Conference on Computer Vision, pp.\ 631--648. Springer, 2022 b

    2022

  43. [51]

    Learning to prompt for continual learning

    Wang, Z., Zhang, Z., Lee, C.-Y., Zhang, H., Sun, R., Ren, X., Su, G., Perot, V., Dy, J., and Pfister, T. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 139--149, 2022 c

    2022

  44. [52]

    Optimizing mode connectivity for class incremental learning

    Wen, H., Cheng, H., Qiu, H., Wang, L., Pan, L., and Li, H. Optimizing mode connectivity for class incremental learning. In Proceedings of the 40th International Conference on Machine Learning, volume 202, pp.\ 36940--36957. PMLR, 2023

    2023

  45. [53]

    Yu, L., Twardowski, B., Liu, X., Herranz, L., Wang, K., Cheng, Y., Jui, S., and Weijer, J. v. d. Semantic drift compensation for class-incremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 6982--6991, 2020

    2020

  46. [54]

    Continual learning through synaptic intelligence

    Zenke, F., Poole, B., and Ganguli, S. Continual learning through synaptic intelligence. In International conference on machine learning, pp.\ 3987--3995. PMLR, 2017

    2017

  47. [55]

    Fine-grained knowledge selection and restoration for non-exemplar class incremental learning

    Zhai, J.-T., Liu, X., Yu, L., and Cheng, M.-M. Fine-grained knowledge selection and restoration for non-exemplar class incremental learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\ 6971--6978, 2024

    2024

  48. [56]

    Slca: Slow learner with classifier alignment for continual learning on a pre-trained model

    Zhang, G., Wang, L., Kang, G., Chen, L., and Wei, Y. Slca: Slow learner with classifier alignment for continual learning on a pre-trained model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 19148--19158, 2023

    2023

  49. [57]

    Continual learning with pre-trained models: A survey

    Zhou, D.-W., Sun, H.-L., Ning, J., Ye, H.-J., and Zhan, D.-C. Continual learning with pre-trained models: A survey. In IJCAI, pp.\ 8363--8371, 2024 a

    2024

  50. [58]

    Expandable subspace ensemble for pre-trained model-based class-incremental learning

    Zhou, D.-W., Sun, H.-L., Ye, H.-J., and Zhan, D.-C. Expandable subspace ensemble for pre-trained model-based class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 23554--23564, 2024 b

    2024

  51. [59]

    Prototype augmentation and self-supervision for incremental learning

    Zhu, F., Zhang, X.-Y., Wang, C., Yin, F., and Liu, C.-L. Prototype augmentation and self-supervision for incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 5871--5880, June 2021

    2021

  52. [60]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.