Pith. sign in

REVIEW 3 major objections 5 minor 47 references

DPFormer: Dynamic Prompt Transformer for Continual Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DPFormer proposes that learned class and task prompts let one transformer learn new classes continually without forgetting previous ones, reporting top accuracy on CIFAR-100, ImageNet100 and ImageNet1K at a nearly constant parameter count.

desk verdict Coherent prompt-prototype architecture with good ablations, but the headline SOTA claim is undermined by an unablated 2,000-exemplar replay buffer and missing prompt-based baselines. read the letter →

arxiv 2506.07414 v1 pith:SGMRLLMC submitted 2025-06-09 cs.CV

Sheng-Kai Huang , Jiun-Feng Chang , Chun-Rong Huang This is my paper
classification cs.CV
keywords continuallearningclass-incrementaldeeptransformercatastrophicforgettingpromptvisioninter-taskconfusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DPFormer aims to solve class-incremental learning, the continual-learning setting in which a model must recognize every class it has ever seen while absorbing new classes from successive tasks. The paper's central claim is that two learned prompt modules, one holding class prototypes and one holding task prototypes, let a single vision transformer remember old knowledge and tell tasks apart, so one unified classifier can serve the full growing class set. The authors report the best average and final-task accuracy among the compared rehearsal, regularization, and dynamic-architecture methods on CIFAR-100, ImageNet100, and ImageNet1K, all with a nearly constant parameter count of about 10.6M. If the claim holds, continual learning models could keep learning indefinitely without expanding per task and without the forgetting that plagues simpler sequential training.

What carries the argument

The load-bearing mechanism is the two-level prototype prompting pipeline feeding a single unified classifier. A class prototype pool stores one learned vector per seen class; the class prompt module selects the most representative prototype by cosine similarity to the pooled encoder features, concatenates it with the average pool prototype and the patch tokens, and mixes them with multi-head self-attention. A task prototype pool then stores one vector per learned task; the task prompt module picks the likeliest task prototype with a small fully-connected softmax selector, appends the average task prototype, and mixes again, so each input carries explicit information about which task it belongs to. The final pooled feature goes to one label classifier trained with binary cross-entropy, a second auxiliary classifier that separates current-task classes from an old-task bucket, and a knowledge distillation term anchored to the previous model, with weights that shift toward distillation as old classes accumulate. This machinery is what the paper credits for freezing old knowledge, admitting new knowledge, and separating tasks while keeping the parameter count nearly independent of the number of tasks.

What would settle it

Run DPFormer on CIFAR-100 with 10 steps in a second configuration that removes the 2,000-exemplar rehearsal buffer entirely, and concurrently give the DyTox+ baseline the same 2,000-exemplar buffer. If buffer-free DPFormer falls to roughly the DyTox+ level, or if buffered DyTox+ matches or beats the reported 78.14 average accuracy, then the prompt modules are not the cause of the reported gains.

Watch

Extended reading notes

Core claim

The paper's central claim is that catastrophic forgetting and inter-task confusion in class-incremental learning can be handled by prompting a transformer with learned prototypes rather than growing or splitting the network. For each input image, the class prompt module selects the most similar class prototype from a pool covering all seen classes and concatenates it, together with the average pool prototype, onto the image's patch tokens; the task prompt module then selects the most likely task prototype and adds it the same way, and transformer encoders mix these tokens before a single label classifier scores all classes seen so far. Prototypes from earlier tasks are frozen while new class and task prototypes are trained for the incoming task, which the paper argues yields stability for old knowledge and plasticity for new knowledge at the same time. Three loss terms train the whole model jointly: binary cross-entropy over all seen classes, knowledge distillation toward the previous model's old-class outputs, and an auxiliary loss from a second classifier that separates current classes from a generic previous-task bucket. Training also rehearses a fixed set of 2,000 stored exemplars (20,000 on ImageNet1K), following the protocol of the compared baselines; the paper reports that this combination achieves the best average and last-task accuracy on CIFAR-100 under 10-, 20-, and 50-step settings and on ImageNet100 and ImageNet1K under 10-step settings, at a parameter count that stays near 10.6M as tasks grow.

Load-bearing premise

The paper attributes its accuracy gains to the class and task prompt modules, but every reported result is trained with a fixed rehearsal buffer of 2,000 exemplars on CIFAR-100 and ImageNet100 (20,000 on ImageNet1K), described in Section 4.1, and the prompt modules are never evaluated without that buffer, so the buffer alone could be driving the improvements.

Editorial extensions

If this is right

  • A single transformer with one unified classifier can serve a growing set of classes without per-task parameter growth: the parameter count stays near 10.64M across the 10-, 20-, and 50-step CIFAR-100 settings while average accuracy remains above 74%.
  • Prompting with frozen old prototypes plus newly learned prototypes is claimed to resolve the stability-plasticity dilemma: ablations show adding both prompt modules raises last-task accuracy from 54.02 to 69.57 on CIFAR-100 with 10 steps.
  • The design scales to large-scale settings: on ImageNet1K with 100 new classes per task, the paper reports 76.13 top-1 and 92.39 top-5 average accuracy, ahead of the dynamic-architecture baselines.
  • Forgetting scores computed against the best prior model stay below the DyTox+ baseline at 20 and 50 steps on CIFAR-100, so the authors claim the prototypes slow memory decay as the number of tasks grows.
  • The auxiliary classifier that separates current classes from old ones, together with the knowledge distillation loss, is credited with the gains in the loss-function ablation: combining all three losses raises last-task accuracy from 62.03 to 69.57 in the 10-step CIFAR-100 setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not isolate the contribution of its 2,000-exemplar rehearsal buffer from the prompt modules, since no ablation runs without replay; the natural next experiment is to measure DPFormer with and without the buffer and to give DyTox+ the same buffer.
  • Prototype selection relies on cosine similarity to a pooled feature, so encoder features that drift across many tasks could make old prototypes stale; a plausible extension is to re-register or refresh old prototypes after each task.
  • Because the prompt-and-single-classifier design is not tied to image classification, the same prototype-prompt scheme could transfer to incremental semantic segmentation or detection, a direction the paper itself names for future work.
  • The near-constant parameter cost makes the method attractive for long horizons; the natural stress test is a much larger number of tasks, where the frozen prototype pools grow but the network does not.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes DPFormer, a transformer-based class-incremental learning method built around two learned prompt mechanisms: a class prompt module that stores and selects class prototypes, and a task prompt module that stores and selects task prototypes. These prompts are integrated with a unified classification module trained with a binary cross entropy loss, a knowledge distillation loss, and an auxiliary loss. The authors report state-of-the-art accuracy on CIFAR-100, ImageNet100, and ImageNet1K under several class-incremental settings, and they provide ablations of the prompt modules, the loss components, and the attention mechanism.

Significance. If the reported results hold under a fair comparison, the method is a meaningful contribution: it combines prompt-based continual learning with a single classifier and a nearly fixed parameter count, and it outperforms the dynamic-architecture baseline DyTox+ on several benchmarks while requiring fewer parameters. The paper also includes useful ablations showing that each prompt module and each loss term contributes to the final accuracy, and the authors commit to releasing code. The main weakness is that the headline comparison uses a replay buffer for DPFormer while the principal dynamic-architecture baselines are rehearsal-free, so the attribution of the gains to the prompt modules is not yet established.

major comments (3)
  1. [Section 4.1, Tables 4 and 5] The central SOTA claim is confounded by the rehearsal buffer. Section 4.1 states that DPFormer uses 2,000 exemplars for CIFAR-100 and ImageNet100 and 20,000 for ImageNet1K, while the main baseline DyTox+ is a rehearsal-free dynamic-architecture method. Tables 4 and 5 then report gains over DyTox+, but Table 1's ablation keeps the buffer in every row, so it never measures the contribution of the prompt modules without rehearsal. This is not merely a protocol detail: Eq. (18) uses binary cross entropy over all seen classes, so for a current-task sample every old class is a negative, and without replayed old-class positives the loss actively suppresses old-class logits. The paper therefore does not demonstrate that the 78.14 vs. 75.54 gap on CIFAR-100 10 steps is due to the class/task prompt modules rather than to the 2,000-exemplar buffer. I request an ablation with no replay buffer and, ideally, a version of DyTox+ trained with the same buffer for an apples-to-apples comparison.
  2. [Tables 4 and 5; Related Work] The comparison omits the main prompt-based continual learning baselines. Since the proposed method is a prompt method, the evaluation should include L2P, DualPrompt, and CODA-Prompt, which are standard reference points for prompt-based class-IL. Their absence makes the statement in the Abstract that DPFormer 'achieves the best performance' stronger than what the evidence supports. Adding these baselines with the same evaluation protocol is necessary to substantiate the superiority claim.
  3. [Tables 4 and 5; Section 4.3] No multiple-seed results or error bars are reported. The tables list a single accuracy value per method and setting, so it is impossible to know whether the differences between DPFormer and DyTox+, for example 78.14 vs. 75.54, are statistically reliable. Given that the paper claims state-of-the-art performance, at least three seeds with standard deviations should be reported for the main comparisons.
minor comments (5)
  1. [Section 3.1] The sentence beginning 'Final, the auxiliary loss' has a typo: 'Final' should be 'Finally'.
  2. [Eq. (21) and surrounding text] The auxiliary loss uses a ground-truth label y_ext^i that indicates whether a sample belongs to a previous task. However, Section 3.2.4 describes c_aux as classifying current-task samples into current classes or 'an extra class' for previous-task samples. The paper should clarify explicitly how replay samples are labeled for this extra class and whether y_ext^i is always zero for current-task samples in a mini-batch.
  3. [Section 4.1] The sentence 'The aforementioned evaluation settings and metrics ... are the same as those in the state-of-the-art methods [5], [30], [32]' is misleading because [5] (iCaRL) uses an exemplar memory while [32] (DyTox) is rehearsal-free. The buffer sizes and replay protocol should be stated independently for each baseline, not lumped together.
  4. [Section 3.2.2, Eq. (5)] The class prototype selection uses arg max over cosine similarities, which is non-differentiable. The paper should state whether gradients flow only through the selected and averaged prototypes or through the similarity scores as well, as this affects how the prototypes are trained.
  5. [Figure 2] The forgetting-score plot is discussed in the text, but the axis labels and the exact protocol for computing the per-class best accuracy should be reproduced in the caption or the text to make the figure self-contained.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: DPFormer's reported results are measured benchmark accuracies; the replay-buffer confound is an experimental attribution issue, not a derivation-level circularity.

full rationale

The paper contains no analytical derivation whose output is equivalent to an input. The central claims are empirical: average and last accuracies measured on standard continual-learning benchmarks, and the ablations in Tables 1-3 vary the prompt modules, losses, and attention schemes while keeping all other settings fixed. None of these ablations fits a parameter to a target quantity and then reports that fitted quantity as a prediction. The prototype-selection rules and loss weights are design choices fixed before evaluation, and the reported numbers are measured outcomes, not quantities forced by the definitions. The main weakness is that a fixed rehearsal buffer (2,000 exemplars for CIFAR-100/ImageNet100, 20,000 for ImageNet1K) is present in every DPFormer row and is not ablated, while the leading dynamic-architecture baseline DyTox+ is rehearsal-free; this is a genuine potential confound for attributing the accuracy gap to the class/task prompt modules, and it is worth verifying in a rebuttal. However, a confounded comparison is not circularity: the accuracy values are external benchmark results rather than consequences of the paper's own equations or definitions. The paper also cites the authors' prior work [33] as a baseline and [44] for data augmentation, but neither self-citation is load-bearing for the central claim; [33] is used as a comparison method, not as justification for the prompt design. No self-citation chain forces the reported results, no fitted parameter is renamed as a prediction, and no known result is repackaged as a derivation. The derivation chain is therefore self-contained in the circularity sense, and the appropriate finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 3 assumptions · 2 invented entities

The central claim rests on the prompt modules and the unified losses; the free hyperparameters are a standard loss weight and architecture choices. The most consequential ledger entry is the rehearsal buffer, which is an external memory the method depends on but the abstract does not mention. The prompt pools are not physical entities but learned parameter sets, so they are listed with no independent evidence.

free parameters (4)
  • auxiliary loss weight lambda = 0.1
    Set without reported sensitivity analysis in Section 3.4; used in loss L = (1-alpha)L_bce + alpha L_kd + lambda L_aux.
  • rehearsal buffer size = 2000 (CIFAR-100, ImageNet100), 20000 (ImageNet1K)
    Chosen in Section 4.1 to match prior works; these stored exemplars directly affect accuracy and are not part of the prompt mechanism.
  • architecture hyperparameters = L=11, D=96, batch=200, lr=5e-4, epochs=500
    Given in Section 3.4; no per-task or per-dataset tuning is reported, but these choices affect all results.
  • task selection FC expansion = t neurons for task t
    The fully connected layer in the task selection block has t neurons; how the layer is initialized and expanded across tasks is not specified, so it functions as an unstated design choice.
assumptions (3)
  • domain assumption The comparison protocol, including the rehearsal buffer, number of epochs, and encoder pretraining, is identical to that of the cited baselines.
    Section 4.1 states the settings match [5], [30], [32], but the paper does not verify this for DyTox+, which is rehearsal-free; if the protocol differs, the claimed superiority is not established.
  • domain assumption The feature encoder is trained from scratch or with the same pretraining as the baselines.
    Section 3.2.1 describes the encoder but never states whether weights are initialized from a pretrained model; training budget and initialization are known to be decisive in continual learning.
  • domain assumption The final accuracy is not inflated by test-set-based hyperparameter selection.
    No validation protocol is described in Section 4; if hyperparameters were selected using the test split, the reported numbers would be optimistic.
invented entities (2)
  • class prototype pool
    purpose: Stores one learned prototype per class to provide class memory and plasticity
    An internal parameter buffer; its utility is only evidenced by benchmark accuracy, with no external falsifiable prediction.
  • task prototype pool
    purpose: Stores one learned prototype per task to encode inter-task discrepancy
    An internal parameter buffer; no handle outside the reported benchmarks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DPFormer: Dynamic Prompt Transformer for Continual Learning." pith.science (2026). https://pith.science/paper/SGMRLLMC

@misc{pith2026250607414,
  author       = {Pith},
  title        = {Pith review of: DPFormer: Dynamic Prompt Transformer for Continual Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SGMRLLMC}},
  note         = {Machine review of arXiv:2506.07414}
}
read the original abstract

In continual learning, solving the catastrophic forgetting problem may make the models fall into the stability-plasticity dilemma. Moreover, inter-task confusion will also occur due to the lack of knowledge exchanges between different tasks. In order to solve the aforementioned problems, we propose a novel dynamic prompt transformer (DPFormer) with prompt schemes. The prompt schemes help the DPFormer memorize learned knowledge of previous classes and tasks, and keep on learning new knowledge from new classes and tasks under a single network structure with a nearly fixed number of model parameters. Moreover, they also provide discrepant information to represent different tasks to solve the inter-task confusion problem. Based on prompt schemes, a unified classification module with the binary cross entropy loss, the knowledge distillation loss and the auxiliary loss is proposed to train the whole model in an end-to-end trainable manner. Compared with state-of-the-art methods, our method achieves the best performance in the CIFAR-100, ImageNet100 and ImageNet1K datasets under different class-incremental settings in continual learning. The source code will be available at our GitHub after acceptance.

Figures

Figures reproduced from arXiv: 2506.07414 by the authors.

Figure 1
Figure 1. The overview of the DPFormer. The DPFormer consists of four modules including the feature encoder module, the class prompt module, [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Comparison of Forgetting Scores in the CIFAR-100 dataset with [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

47 extracted references · 43 canonical work pages

  1. [5]

    icarl: Incremental classifier and representation learning,

    S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert, “icarl: Incremental classifier and representation learning,” inProc. IEEE Conf. Computer Vision and Pattern Recognition, 2017, pp. 2001–2010

    work page 2017

  2. [30]

    Der: Dynamically expandable represen- tation for class incremental learning,

    S. Yan, J. Xie, and X. He, “Der: Dynamically expandable represen- tation for class incremental learning,” inProc. IEEE Conf. Computer Vision and Pattern Recognition, 2021, pp. 3013–3022

    work page 2021

  3. [32]

    Dytox: Trans- formers for continual learning with dynamic token expansion,

    A. Douillard, A. Ram ´e, G. Couairon, and M. Cord, “Dytox: Trans- formers for continual learning with dynamic token expansion,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2022, pp. 9285–9295

    work page 2022

  4. [1]

    How does a brain build a cogni- tive code?

    S. Grossberg and S. Grossberg, “How does a brain build a cogni- tive code?”Studies of Mind and Brain, vol. 70, pp. 1–52, 1982

    work page 1982

  5. [2]

    The stability-plasticity dilemma: Investigating the continuum from catastrophic forget- ting to age-limited learning effects,

    M. Mermillod, A. Bugaiska, and P . Bonin, “The stability-plasticity dilemma: Investigating the continuum from catastrophic forget- ting to age-limited learning effects,”Frontiers in Psychology, vol. 4, p. 504, 2013

    work page 2013

  6. [3]

    Continual lifelong learning with neural networks: A review,

    G. I. Parisi, R. Kemker, J. L. Part, C. Kanan, and S. Wermter, “Continual lifelong learning with neural networks: A review,” Neural Networks, vol. 113, pp. 54–71, 2019

    2019

  7. [4]

    Class-incremental learning: Survey and performance evaluation on image classification,

    M. Masana, X. Liu, B. Twardowski, M. Menta, A. D. Bagdanov, and J. van de Weijer, “Class-incremental learning: Survey and performance evaluation on image classification,”IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 45, no. 5, pp. 5513– 5533, 2023

    work page 2023

  8. [6]

    End-to-end incremental learning,

    F. M. Castro, M. J. Mar ´ın-Jim´enez, N. Guil, C. Schmid, and K. Alahari, “End-to-end incremental learning,” inProc. Euro. Conf. Computer Vision, 2018, pp. 233–248

    work page 2018

Show all 47 references
  1. [7]

    Rie- mannian walk for incremental learning: Understanding forgetting and intransigence,

    A. Chaudhry, P . K. Dokania, T. Ajanthan, and P . H. S. Torr, “Rie- mannian walk for incremental learning: Understanding forgetting and intransigence,” inProc. Euro. Conf. Computer Vision, 2018, p. 556–572

    2018

  2. [8]

    Experience replay for continual learning,

    D. Rolnick, A. Ahuja, J. Schwarz, T. Lillicrap, and G. Wayne, “Experience replay for continual learning,” inProc. Advances in Neural Information Processing Systems, vol. 32, 2019

    2019

  3. [9]

    Gdumb: A simple approach that questions our progress in continual learning,

    A. Prabhu, P . H. S. Torr, and P . K. Dokania, “Gdumb: A simple approach that questions our progress in continual learning,” in Proc. Euro. Conf. Computer Vision, 2020, pp. 524–540

    2020

  4. [10]

    Class-incremental continual learning into the extended der- verse,

    M. Boschini, L. Bonicelli, P . Buzzega, A. Porrello, and S. Calder- ara, “Class-incremental continual learning into the extended der- verse,”IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 45, no. 5, pp. 5497–5512, 2023

    2023

  5. [11]

    Overcoming catastrophic forgetting in neural networks,

    J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska et al., “Overcoming catastrophic forgetting in neural networks,” Proceedings of the National Academy of Sciences, vol. 114, no. 13, pp. 3521–3526, 2017

    2017

  6. [12]

    Rotate your networks: Better weight consolida- tion and less catastrophic forgetting,

    X. Liu, M. Masana, L. Herranz, J. Van de Weijer, A. M. L ´opez, and A. D. Bagdanov, “Rotate your networks: Better weight consolida- tion and less catastrophic forgetting,” inProc. Int’l Conf. Pattern Recognition, 2018, pp. 2262–2268

    2018

  7. [13]

    Continual learning with ex- tended kronecker-factored approximate curvature,

    J. Lee, H. G. Hong, D. Joo, and J. Kim, “Continual learning with ex- tended kronecker-factored approximate curvature,” inProc. IEEE Conf. Computer Vision and Pattern Recognition, 2020, pp. 8998–9007

    2020

  8. [14]

    Continual learning through synaptic intelligence,

    F. Zenke, B. Poole, and S. Ganguli, “Continual learning through synaptic intelligence,” inProc. Int’l Conf. Machine Learning, vol. 70, 2017, pp. 3987–3995

    2017

  9. [15]

    Memory aware synapses: Learning what (not) to forget,

    R. Aljundi, F. Babiloni, M. Elhoseiny, M. Rohrbach, and T. Tuyte- laars, “Memory aware synapses: Learning what (not) to forget,” inProc. Euro. Conf. Computer Vision, 2018, p. 144–161

    2018

  10. [16]

    Learning without forgetting,

    Z. Li and D. Hoiem, “Learning without forgetting,”IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 40, no. 12, pp. 2935– 2947, 2018

    2018

  11. [18]

    Class-incremental learning via deep model consolidation,

    J. Zhang, J. Zhang, S. Ghosh, D. Li, S. Tasci, L. Heck, H. Zhang, and C.-C. J. Kuo, “Class-incremental learning via deep model consolidation,” inProc. IEEE Winter Conf. Applications of Computer Vision, 2020, pp. 1120–1129

    2020

  12. [19]

    Learning a unified classifier incrementally via rebalancing,

    S. Hou, X. Pan, C. C. Loy, Z. Wang, and D. Lin, “Learning a unified classifier incrementally via rebalancing,” inProc. IEEE Conf. Computer Vision and Pattern Recognition, 2019, pp. 831–839

    2019

  13. [20]

    Large scale incremental learning,

    Y. Wu, Y. Chen, L. Wang, Y. Ye, Z. Liu, Y. Guo, and Y. Fu, “Large scale incremental learning,” inProc. IEEE Conf. Computer Vision and Pattern Recognition, 2019, pp. 374–382

    2019

  14. [21]

    Maintaining discrimination and fairness in class incremental learning,

    B. Zhao, X. Xiao, G. Gan, B. Zhang, and S.-T. Xia, “Maintaining discrimination and fairness in class incremental learning,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2020, pp. 13 208–13 217

    2020

  15. [22]

    Podnet: Pooled outputs distillation for small-tasks incremental learning,

    A. Douillard, M. Cord, C. Ollion, T. Robert, and E. Valle, “Podnet: Pooled outputs distillation for small-tasks incremental learning,” inProc. Euro. Conf. Computer Vision, 2020, pp. 86–102

    2020

  16. [23]

    Overcoming catastrophic forgetting with hard attention to the task,

    J. Serra, D. Suris, M. Miron, and A. Karatzoglou, “Overcoming catastrophic forgetting with hard attention to the task,” inProc. Int’l Conf. Machine Learning, 2018, pp. 4548–4557

    2018

  17. [24]

    Achieving forgetting prevention and knowledge transfer in continual learning,

    Z. Ke, B. Liu, N. Ma, H. Xu, and L. Shu, “Achieving forgetting prevention and knowledge transfer in continual learning,” inProc. Advances in Neural Information Processing Systems, vol. 34, 2021, pp. 22 443–22 456

    2021

  18. [25]

    Ternary feature masks: Zero-forgetting for task-incremental learning,

    M. Masana, T. Tuytelaars, and J. van de Weijer, “Ternary feature masks: Zero-forgetting for task-incremental learning,” inProc. IEEE Conf. Computer Vision and Pattern Recognition, June 2021, pp. 3570–3579

    2021

  19. [26]

    Self-growing binary activation network: A novel deep learning model with dynamic architec- ture,

    Z. Zhang, Y. Chen, and C. Zhou, “Self-growing binary activation network: A novel deep learning model with dynamic architec- ture,”IEEE Trans. on Neural Networks and Learning Systems, vol. 35, no. 1, pp. 624–633, 2024

    2024

  20. [27]

    Ef- ficient feature transformations for discriminative and generative continual learning,

    V . K. Verma, K. J. Liang, N. Mehta, P . Rai, and L. Carin, “Ef- ficient feature transformations for discriminative and generative continual learning,” inProc. IEEE Conf. Computer Vision and Pattern Recognition, 2021, pp. 13 865–13 875

    2021

  21. [28]

    Continual learning by using information of each class holistically,

    W. Hu, Q. Qin, M. Wang, J. Ma, and B. Liu, “Continual learning by using information of each class holistically,” inProc. AAAI Conf. Artificial Intelligence, vol. 35, no. 9, 2021, pp. 7797–7805

    2021

  22. [29]

    Random path selection for incremental learning,

    J. Rajasegaran, M. Hayat, S. Khan, F. S. Khan, and L. Shao, “Random path selection for incremental learning,”Proc. Advances in Neural Information Processing Systems, 2019

    2019

  23. [31]

    Preserving ear- lier knowledge in continual learning with the help of all previous feature extractors,

    Z. Li, C. Zhong, S. Liu, R. Wang, and W.-S. Zheng, “Preserving ear- lier knowledge in continual learning with the help of all previous feature extractors,”arXiv preprint arXiv:2104.13614, 2021

    2021 arXiv

  24. [33]

    Transformer with task selection for continual learning,

    S.-K. Huang and C.-R. Huang, “Transformer with task selection for continual learning,” inProc. Int’l Conf. Machine Vision and Applications, 2023, pp. 1–5

    2023

  25. [34]

    Herding dynamical weights to learn,

    M. Welling, “Herding dynamical weights to learn,” inProc. Int’l Conf. Machine Learning, 2009, pp. 1121–1128

    2009

  26. [35]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” inProc. Int’l Conf. Learning Representa...

    2021

  27. [36]

    A continual learn- ing survey: Defying forgetting in classification tasks,

    M. De Lange, R. Aljundi, M. Masana, S. Parisot, X. Jia, A. Leonardis, G. Slabaugh, and T. Tuytelaars, “A continual learn- ing survey: Defying forgetting in classification tasks,”IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 44, no. 7, pp. 3366– 3385, 2022

    2022

  28. [37]

    A comprehensive survey of continual learning: Theory, method and application,

    L. Wang, X. Zhang, H. Su, and J. Zhu, “A comprehensive survey of continual learning: Theory, method and application,”IEEE Trans. on Pattern Analysis and Machine Intelligence, pp. 1–20, 2024

    2024

  29. [38]

    Distilling the knowledge in a neural network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” inNIPS Deep Learning and Representation Learning Workshop, 2015

    2015

  30. [39]

    Training data-efficient image transformers & distilla- tion through attention,

    H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. Jegou, “Training data-efficient image transformers & distilla- tion through attention,” inProc. Int’l Conf. Machine Learning, vol. 139, 2021, pp. 10 347–10 357

    2021

  31. [40]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” inProc. Int’l Conf. Computer Vision, 2021, pp. 9992– 10 002

    2021

  32. [41]

    Neighborhood attention transformer,

    A. Hassani, S. Walton, J. Li, S. Li, and H. Shi, “Neighborhood attention transformer,” inProc. IEEE Conf. Computer Vision and Pattern Recognition, 2023, pp. 6185–6194

    2023

  33. [42]

    Dilated neighborhood attention trans- former,

    A. Hassani and H. Shi, “Dilated neighborhood attention trans- former,”arXiv preprint arXiv:2209.15001, 2022

    2022 arXiv

  34. [43]

    Decoupled weight decay regulariza- tion,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regulariza- tion,” inProc. Int’l Conf. Learning Representations, 2019

    2019

  35. [44]

    ADMM- SRNet: Alternating direction method of multipliers based sparse representation network for one-class classification,

    C.-Y. Chiou, K.-T. Lee, C.-R. Huang, and P .-C. Chung, “ADMM- SRNet: Alternating direction method of multipliers based sparse representation network for one-class classification,”IEEE Trans. on Image Processing, vol. 32, pp. 2843–2856, 2023. SUBMITTED TO IEEE TRANSACTIONS ON P...

    2023

  36. [45]

    Mixup: Beyond empirical risk minimization,

    H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz, “Mixup: Beyond empirical risk minimization,” inProc. Int’l Conf. Learning Representations, 2018

    2018

  37. [46]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky, G. Hintonet al., “Learning multiple layers of features from tiny images,”Technical Report, 2009

    2009

  38. [47]

    Im- agenet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Im- agenet: A large-scale hierarchical image database,” inProc. IEEE Conf. Computer Vision and Pattern Recognition, 2009, pp. 248–255. Sheng-Kai Huangreceived the B.S. degrees from the Department of Computer Scienc...

    2009

  39. [2023]

    His research in- terests include computer vision, computer graphics, multimedia signal processing, image processing, and medical image processing

    In 2024, he joined the Department of Computer Science, National Y ang Ming Chiao Tung University, Hsinchu, Taiwan. His research in- terests include computer vision, computer graphics, multimedia signal processing, image processing, and medical image processing. He is a member ...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.