Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

BOFA claims that exemplar-free class-incremental learning with CLIP can be achieved by fine-tuning only CLIP's existing cross-modal bridge-layer, with updates constrained to an Orthogonal Safe Subspace derived from past-task feature scatter

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-03 22:11 UTC pith:OAQOBOCT

load-bearing objection BOFA has a clean idea and strong numbers, but the no-forgetting guarantee is weaker than claimed, the 'no extra parameters' claim is undercut by auxiliary classifiers, and the empirical reporting needs code and variance. the 4 major comments →

arxiv 2511.11421 v2 pith:OAQOBOCT submitted 2025-11-14 cs.CV cs.LG

BOFA: Bridge-Layer Orthogonal Low-Rank Fusion for CLIP-Based Class-Incremental Learning

Lan Li , Tao Hu , Da-Wei Zhou , Jia-Qi Yang , Han-Jia Ye , De-Chuan Zhan This is my paper
classification cs.CV cs.LG
keywords class-incremental learningCLIPcatastrophic forgettingorthogonal low-rank fusionsafe subspaceexemplar-free learninghybrid prototypesvision-language models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to show that catastrophic forgetting in CLIP-based class-incremental learning can be prevented without storing old data or adding any new modules. Its proposal, BOFA, fine-tunes only the existing linear projection layer that maps visual features into CLIP's shared embedding space, and restricts each new task's weight update to lie in a low-rank subspace spanned by the smallest eigenvectors of the cumulative scatter matrix of features from previous tasks. Because old features project near zero onto this subspace, the update leaves past-task representations approximately unchanged. The paper further argues that fusing textual prototypes with visual prototypes produced by this stably adapted layer yields a more discriminative classifier, and reports consistent accuracy gains over recent baselines on nine datasets.

Core claim

The central claim is that a CLIP model can continually learn new classes without replay and without extra parameters if all adaptation happens in the bridge-layer (the linear projection g2 after the frozen visual backbone) and each incremental update ΔW_new is confined to an Orthogonal Safe Subspace. This subspace is defined as the span of the k eigenvectors of S_old = X_old^T X_old with the smallest eigenvalues, the directions where past-task features have minimal projection. The update is implemented as ΔW = P*B, with P* frozen to that eigenbasis and B the only trainable matrix, initialized from a brief unconstrained fine-tuning oracle via B0 = P*^T ΔW~new. The cumulative scatter matrix is

What carries the argument

The Orthogonal Safe Subspace (OSS): the k-dimensional subspace spanned by the eigenvectors of the cumulative scatter matrix of past task features that have the smallest eigenvalues. It is the solution to minimizing ||X_old P||_F over orthonormal P (Proposition 1). Parameter updates are implemented as a modified LoRA, ΔW = P*B, where P* is fixed to the OSS basis and B is learned with an oracle-initialized B0 = P*^T ΔW~new, so the row space of the update lies in the approximate null space of old features. This construction is what carries the forgetting-prevention argument.

Load-bearing premise

The whole forgetting guarantee rests on treating X_old ΔW_new ≈ 0 as sufficient for preserving old-task accuracy, which ignores the trained classifier heads and the accumulated past updates, and assumes that the new task can actually be learned using only directions inside the k-dimensional safe subspace.

What would settle it

Run BOFA on a two-task sequence where the second task is from a very different visual domain (e.g., first task natural images, second task sketches). If the angle between the unconstrained fine-tuning update and the OSS basis is near 90 degrees (most gradient energy outside the subspace), and old-task accuracy still drops or new-task accuracy stays low, the central claim fails. More directly: if final old-task accuracy is not preserved when X_old ΔW_new ≈ 0 holds to numerical precision, the heuristic linking the interference term to forgetting is false.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If correct, exemplar-free CIL on CLIP needs no replay buffer and no additional trainable modules, removing the storage and latency overhead of adapters and prompts.
  • The final model after all tasks is exactly the original CLIP architecture with only the bridge-layer weights modified, so inference cost is unchanged from the base model.
  • The cumulative scatter matrix update means the method's memory footprint grows with the feature dimension, not with the number of training samples, making it feasible for large incremental streams.
  • The hybrid textual-visual prototype classifier could improve zero-shot and few-shot classification beyond the incremental setting, since it combines semantic and data-driven cues.
  • The same orthogonal-safe-subspace idea transfers to any CLIP-like model with a linear cross-modal projection layer.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable implication the paper does not pursue: the OSS constraint should degrade gracefully as tasks become more dissimilar, because new-task gradients then lie increasingly outside the low-energy subspace; measuring the cosine similarity between unconstrained updates and the OSS basis would quantify plasticity loss.
  • The no-extra-parameters claim is complicated by the hierarchical inference stage, which adds auxiliary per-class classifiers; these are stored and used at inference, so the 'no extra parameters or inference cost' statement applies to the CLIP backbone itself, not the full deployed system.
  • The forgetting analysis treats the bridge-layer in isolation; coupling the OSS constraint with a distillation loss on the classifier may be a natural extension to protect prototype-based predictions.
  • The final prototype refinement step recomputes visual prototypes with the final bridge-layer; in a strict streaming setting where later tasks arrive indefinitely, this can only be done if per-class mean features are retained, so the method may need to keep those means indefinitely.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes BOFA, an exemplar-free class-incremental learning method for CLIP. All trainable adaptation is confined to CLIP's existing cross-modal bridge layer (the linear projection after the vision backbone), avoiding extra parameters. To prevent forgetting, updates are restricted to an 'Orthogonal Safe Subspace' spanned by the k smallest eigenvectors of the cumulative feature scatter matrix, implemented as LoRA with the left factor fixed to this basis. Classification uses hybrid text/visual prototypes plus a hierarchical inference stage with auxiliary classifiers. The authors claim no extra parameters or inference cost, stable knowledge accumulation without replay, and state-of-the-art accuracy on nine benchmarks.

Significance. The core idea is attractive: adapting only a single existing projection layer and constraining updates to an approximate null space of older features is a clean, parameter-free-in-spirit design. The spectral result in Proposition 1 is standard and correct, and the paper carefully positions the method against prompt- and adapter-based CLIP continual learning. The empirical evaluation is broad (nine datasets, including comparisons to exemplar-based methods without using exemplars) and the reported gains are nontrivial. However, the manuscript is not yet sufficient: the forgetting analysis does not prove the claimed safety guarantee, the 'no extra inference cost' claim is contradicted by the hierarchical inference design, and the SOTA results rest on single-seed tables with a placeholder extended-version link and missing training details. These issues are fixable, so the paper warrants major revision rather than rejection.

major comments (4)
  1. [§3.2, 'Forgetting Analysis' and 'LoRA in the Orthogonal Safe Subspace'] Equation (2) is not enforced by the proposed parameterization. With ΔW_new = P* B, the interference is X_old P* B, whose norm is at most ||X_old P*||_F ||B||_F. Small ||X_old P*||_F does not imply small interference because B is trainable and is not norm-constrained; the data-driven initialization B0 = P*^T ΔW~ can itself be large, and the paper mentions no weight decay, projection-back step, or norm bound on B. Thus the claim that OSS 'ensures stable knowledge accumulation' overstates the guarantee. The analysis should either add an explicit bound on B, add a practical norm control mechanism, or be rephrased as a heuristic whose validity is verified empirically.
  2. [§3.3, 'Hierarchical Inference Strategy' and 'Discussion of BOFA'] The central efficiency claim 'no extra parameters or inference cost' is contradicted by the method itself. The hierarchical inference stage uses task-specific auxiliary classifiers (|Y|·d_o storage) to pre-select candidate classes, and these classifiers are part of the inference pipeline. The Discussion also admits storing the d_o×d_o cumulative scatter matrix and per-class mean features. These may be acceptable memory costs, but the paper's abstract and introduction say 'adding no extra parameters or inference cost' and 'preserves the original inference cost of CLIP.' This should be corrected: auxiliary classifiers add an extra inference pass, and the scatter matrix is an additional memory requirement.
  3. [Tables 1–2 and 'Implementation Details'] The SOTA claim is currently supported only by single point estimates. Tables 1 and 2 report no standard deviations, no number of seeds, and only one class order (seed 1993). CIL results are highly sensitive to class order and initial task composition. Give mean±std over at least 3–5 seeds/orders and, where possible, significance tests against RAPF and SimpleCIL. In addition, the 'extended version' link is a placeholder (https://aaai.org/example/extended-version) and the supplementary materials are not included; training details such as the oracle fine-tuning schedule, hierarchical candidate-set size, and exact evaluation protocol are missing, which prevents reproduction.
  4. [Eq. (4) and 'Implementation Details'] Two hyperparameters control the method: k (rank of the OSS/LoRA subspace) and λ (hybrid prototype interpolation weight). The paper says λ is chosen by grid search on the first task and fixed later, but no range or criterion is given, and k is not discussed at all. Since k governs the plasticity-stability trade-off (if k is too small, new-task gradients lying outside the subspace are ignored; if too large, old-feature interference increases), the lack of a sensitivity analysis or selection rule weakens the empirical claim. Report accuracy versus k and λ on at least one or two datasets, and specify how the final values were chosen.
minor comments (4)
  1. [Figure 4 caption] Typo: 'with out' should be 'without' in the caption.
  2. [Introduction] The 'Extended version—https://aaai.org/example/extended-version' line is a placeholder and should be replaced with a working link or removed.
  3. [Notation, §3.2] The symbol X_old is used both for the raw high-dimensional feature matrix and for x_o,old; make the distinction explicit to avoid confusion.
  4. [Table 1 headers] The column grouping like 'B0 Inc10 B50 Inc10' is difficult to parse. The two numbers under each configuration should be labeled explicitly (e.g., 'Avg' and 'Last') to avoid ambiguity.

Circularity Check

0 steps flagged

No significant circularity: the derivation is self-contained and benchmark claims are empirical, not reducible to fitted inputs.

full rationale

The central derivation is the construction of the Orthogonal Safe Subspace from the k smallest eigenvectors of the cumulative scatter matrix, minimizing ||X_old P||_F. Proposition 1 is a standard spectral characterization of the minimizer; it is proved in the appendix and is not a claim whose output is assumed as an input. The forgetting condition X_old ΔW_new ≈ 0 is posed as an ideal objective, and the actual implementation constrains only the row space of ΔW = P*B, not the magnitude of B. This is a possible correctness gap—the interference can be large if B grows—but it is not circularity: no predicted quantity is fitted and then reported as if independently obtained. The hyperparameters λ and k are tuned on the first task, but the reported benchmark accuracies are external comparisons, not quantities determined by those fits. The paper cites prior work from overlapping groups (e.g., PROOF, RAPF), but these citations are for baselines and experimental protocols, not for the load-bearing uniqueness or optimality of the OSS construction. No renamed known result is presented as a derivation of itself. Under the stated rules, a self-contained empirical method with standard spectral steps should receive a low score even if its forgetting guarantee is heuristic or incompletely analyzed.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The method introduces no new physical entities. Its free parameters are the prototype mixing weight λ, the OSS rank k, and the oracle fine-tuning schedule. The central axioms are the bridge-layer-only adaptation assumption and the plasticity of the OSS for new tasks, neither of which is analyzed beyond the empirical ablation.

free parameters (3)
  • λ (hybrid prototype interpolation weight) = not reported; grid-searched on first task's training set
    Eq. (4): p_c = (1−λ) z_t^c + λ z_i^c; selected on the first task's training set and fixed thereafter. Benchmark results depend on this choice.
  • k (rank of Orthogonal Safe Subspace / LoRA) = not reported in main text
    P* ∈ R^{d_o×k} is the k smallest eigenvectors of S_old; k controls the capacity/interference tradeoff, but its value is not stated.
  • Oracle fine-tuning schedule for ΔW~ = not specified
    B0 = P*^T ΔW~ requires briefly fine-tuning the entire bridge layer on the new task; duration and step count are omitted, and this affects results.
axioms (5)
  • domain assumption Minimizing ||X_old P||_F^2 and constraining ΔW_new to span(P) preserves old-task classification performance.
    Core forgetting-mitigation premise (Eq. 2 and 'Constructing an Orthogonal Safe Subspace'); equates reduced feature projection interference with preserved task accuracy.
  • domain assumption The frozen backbones g1 and g_t remain suitable; only the linear bridge layer needs adaptation.
    Fine-tuning only g2 assumes all downstream task-specific information can be encoded in this linear map (Section 'Fine-tuning the Cross-Modal Bridge-Layer').
  • domain assumption The k smallest eigenvectors of S_old provide a subspace that still permits learning new tasks.
    The OSS may have low signal for new task data; the paper provides only empirical t-SNE, not an analysis. Section 'LoRA in the Orthogonal Safe Subspace'.
  • standard math Eckart-Young/PCA optimality of the approximate null space (Proposition 1).
    Standard spectral theorem; proof deferred to Appendix A.2, which is not included in the manuscript.
  • standard math Cumulative scatter matrix S_new = S_old + X_new^T X_new captures all past-task feature geometry.
    Holds by algebra, but assumes X_old features are from frozen g1 and remain valid across tasks.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of BOFA: Bridge-Layer Orthogonal Low-Rank Fusion for CLIP-Based Class-Incremental Learning." pith.science (2026). https://pith.science/paper/OAQOBOCT

@misc{pith2026251111421,
  author       = {Pith},
  title        = {Pith review of: BOFA: Bridge-Layer Orthogonal Low-Rank Fusion for CLIP-Based Class-Incremental Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OAQOBOCT}},
  note         = {Machine review of arXiv:2511.11421}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Class-Incremental Learning (CIL) aims to continually learn new categories without forgetting previously acquired knowledge. Vision-language models such as CLIP offer strong transferable representations via multi-modal supervision, making them promising for CIL. However, applying CLIP to CIL poses two major challenges: (1) adapting to downstream tasks often requires additional learnable modules, increasing model complexity and susceptibility to forgetting; and (2) while multi-modal representations offer complementary strengths, existing methods have yet to fully realize their potential in effectively integrating visual and textual modalities. To address these issues, we propose BOFA (Bridge-layer Orthogonal Fusion for Adaptation), a novel framework for CIL. BOFA confines all model adaptation exclusively to CLIP's existing cross-modal bridge-layer, thereby adding no extra parameters or inference cost. To prevent forgetting within this layer, it leverages Orthogonal Low-Rank Fusion, a mechanism that constrains parameter updates to a low-rank ``safe subspace" mathematically constructed to be orthogonal to past task features. This ensures stable knowledge accumulation without data replay. Furthermore, BOFA employs a cross-modal hybrid prototype that synergizes stable textual prototypes with visual counterparts derived from our stably adapted bridge-layer, enhancing classification performance. Extensive experiments on standard benchmarks show that BOFA achieves superior accuracy and efficiency compared to existing methods.

Figures

Figures reproduced from arXiv: 2511.11421 by Da-Wei Zhou, De-Chuan Zhan, Han-Jia Ye, Jia-Qi Yang, Lan Li, Tao Hu.

Figure 1
Figure 1. Figure 1: Overview of Orthogonal Low-Rank Fusion, where an OSS P ∗ is constructed from past task features to constrain the low￾rank update for a new task, thereby minimizing interference with prior knowledge. null space as defined above. With this parameterization, the weight update is given by ∆W = P∗B, where only B re￾mains trainable. However, when A is frozen to the orthogonal safe sub￾space basis, simply initial… view at source ↗
Figure 2
Figure 2. Figure 2: Incremental performance of different methods. Accuracy is reported at each incremental stage. BOFA consistently outperforms all baselines, with the final gap to the strongest competitor noted at the end of each curve. Additional results are available in the supplementary material. Cars B0 Inc10 CUB B0 Inc20 UCF B0 Inc10 Food B0 Inc10 0 20 40 60 80 BOFA RAPF`s Fusion Sequential Fine-tuning LoRA [PITH_FULL_… view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Continual Learning with Vision-Language Models via Semantic-Geometry Preservation

    cs.CV 2026-03 conditional novelty 6.0

    SeGP-CL reduces catastrophic forgetting in CLIP-based continual learning by distilling cross-modal geometry around adversarial anchors at the old-new class boundary plus regularizing the text-space reference frame.

Reference graph

Works this paper leans on

67 extracted references · 4 linked inside Pith · cited by 1 Pith paper

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Aggarwal, C. C. 2018. A survey of stream clustering algorithms. In Data Clustering, 231--258. Chapman and Hall/CRC

    2018

  4. [4]

    Aljundi, R.; Babiloni, F.; Elhoseiny, M.; Rohrbach, M.; and Tuytelaars, T. 2018. Memory aware synapses: Learning what (not) to forget. In ECCV, 139--154

    2018

  5. [5]

    Aljundi, R.; Kelchtermans, K.; and Tuytelaars, T. 2019. Task-free continual learning. In CVPR, 11254--11263

    2019

  6. [6]

    Barbu, A.; Mayo, D.; Alverio, J.; Luo, W.; Wang, C.; Gutfreund, D.; Tenenbaum, J.; and Katz, B. 2019. Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models. NeurIPS, 32

    2019

  7. [7]

    Bossard, L.; Guillaumin, M.; and Van Gool, L. 2014. Food-101--mining discriminative components with random forests. In ECCV, 446--461. Springer

    2014

  8. [8]

    K.; Ajanthan, T.; and Torr, P

    Chaudhry, A.; Dokania, P. K.; Ajanthan, T.; and Torr, P. H. 2018. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In ECCV, 532--547

    2018

  9. [9]

    Chen, S.; Ge, C.; Tong, Z.; Wang, J.; Song, Y.; Wang, J.; and Luo, P. 2022. Adaptformer: Adapting vision transformers for scalable visual recognition. NeurIPS, 35: 16664--16678

    2022

  10. [10]

    De Lange, M.; Aljundi, R.; Masana, M.; Parisot, S.; Jia, X.; Leonardis, A.; Slabaugh, G.; and Tuytelaars, T. 2021. A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence, 44(7): 3366--3385

    2021

  11. [11]

    Dong, S.; Hong, X.; Tao, X.; Chang, X.; Wei, X.; and Gong, Y. 2021. Few-Shot Class-Incremental Learning via Relation Knowledge Distillation. In AAAI, 1255--1263

    2021

  12. [12]

    Douillard, A.; Cord, M.; Ollion, C.; Robert, T.; and Valle, E. 2020. Podnet: Pooled outputs distillation for small-tasks incremental learning. In ECCV, 86--102

    2020

  13. [13]

    Douillard, A.; Ram \'e , A.; Couairon, G.; and Cord, M. 2022. Dytox: Transformers for continual learning with dynamic token expansion. In CVPR, 9285--9295

    2022

  14. [14]

    French, R. M. 1999. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 3(4): 128--135

    1999

  15. [15]

    M.; and Ferrara, A

    French, R. M.; and Ferrara, A. 1999. Modeling time perception in rats: Evidence for catastrophic interference in animal learning. In Proceedings of the 21st Annual Conference of the Cognitive Science Conference, 173--178. Citeseer

    1999

  16. [16]

    Gao, Q.; Zhao, C.; Ghanem, B.; and Zhang, J. 2022. R-DFCIL: Relation-Guided Representation Learning for Data-Free Class Incremental Learning. In ECCV, 423--439

    2022

  17. [17]

    Hendrycks, D.; Basart, S.; Mu, N.; Kadavath, S.; Wang, F.; Dorundo, E.; Desai, R.; Zhu, T.; Parajuli, S.; Guo, M.; et al. 2021. The many faces of robustness: A critical analysis of out-of-distribution generalization. In ICCV, 8340--8349

    2021

  18. [18]

    Hinton, G.; Vinyals, O.; and Dean, J. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531

    Pith/arXiv arXiv 2015

  19. [19]

    J.; yelong shen; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W

    Hu, E. J.; yelong shen; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W. 2022. Lo RA : Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations

    2022

  20. [20]

    Huang, L.; Cao, X.; Lu, H.; and Liu, X. 2024. Class-incremental learning with clip: Adaptive representation adjustment and parameter fusion. In ECCV

    2024

  21. [21]

    Ilharco, G.; Wortsman, M.; Wightman, R.; Gordon, C.; Carlini, N.; Taori, R.; Dave, A.; Shankar, V.; Namkoong, H.; Miller, J.; Hajishirzi, H.; Farhadi, A.; and Schmidt, L. 2021. OpenCLIP

    2021

  22. [22]

    Jia, C.; Yang, Y.; Xia, Y.; Chen, Y.-T.; Parekh, Z.; Pham, H.; Le, Q.; Sung, Y.-H.; Li, Z.; and Duerig, T. 2021. Scaling up visual and vision-language representation learning with noisy text supervision. In ICML, 4904--4916

    2021

  23. [23]

    J.; Hariharan, B.; and Lim, S

    Jia, M.; Tang, L.; Chen, B.; Cardie, C.; Belongie, S. J.; Hariharan, B.; and Lim, S. 2022. Visual Prompt Tuning. In ECCV, 709--727

    2022

  24. [24]

    Jung, D.; Han, D.; Bang, J.; and Song, H. 2023. Generating instance-level prompts for rehearsal-free continual learning. In ICCV, 11847--11857

    2023

  25. [25]

    Krause, J.; Stark, M.; Deng, J.; and Fei-Fei, L. 2013. 3d object representations for fine-grained categorization. In ICCV Workshop, 554--561

    2013

  26. [26]

    Krizhevsky, A. 2009. Learning Multiple Layers of Features from Tiny Images

    2009

  27. [27]

    Li, J.; Li, D.; Savarese, S.; and Hoi, S. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML, 19730--19742

    2023

  28. [28]

    Li, L.; Peng, J.; Chen, H.; Gao, C.; and Yang, X. 2024. How to configure good in-context sequence for visual question answering. In CVPR, 26710--26720

    2024

  29. [29]

    Li, Z.; and Hoiem, D. 2016. Learning Without Forgetting. In ECCV, 614--629

    2016

  30. [30]

    Lu, Y.; Wang, M.; and Deng, W. 2022. Augmented Geometric Distillation for Data-Free Incremental Person ReID. In CVPR, 7329--7338

    2022

  31. [31]

    Luo, Z.; Liu, Y.; Schiele, B.; and Sun, Q. 2023. Class-incremental exemplar compression for class-incremental learning. In CVPR, 11371--11380

    2023

  32. [32]

    Maji, S.; Rahtu, E.; Kannala, J.; Blaschko, M.; and Vedaldi, A. 2013. Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151

    Pith/arXiv arXiv 2013

  33. [33]

    D.; and Van De Weijer, J

    Masana, M.; Liu, X.; Twardowski, B.; Menta, M.; Bagdanov, A. D.; and Van De Weijer, J. 2022. Class-incremental learning: survey and performance evaluation on image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5): 5513--5533

    2022

  34. [34]

    Masoudnia, S.; and Ebrahimpour, R. 2014. Mixture of experts: a literature survey. Artificial Intelligence Review, 42: 275--293

    2014

  35. [35]

    D.; Gong, D.; Parveneh, A.; Abbasnejad, E.; and Hengel, A

    McDonnell, M. D.; Gong, D.; Parveneh, A.; Abbasnejad, E.; and Hengel, A. v. d. 2023. RanPAC: Random Projections and Pre-trained Models for Continual Learning. In NeurIPS

    2023

  36. [36]

    Park, J.; Kang, M.; and Han, B. 2021. Class-incremental learning for action recognition in videos. In ICCV, 13698--13707

    2021

  37. [37]

    Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; et al. 2019. Pytorch: An imperative style, high-performance deep learning library. In NeurIPS, 8026--8037

    2019

  38. [38]

    Qi, Z.-H.; Zhou, D.-W.; Yao, Y.; Ye, H.-J.; and Zhan, D.-C. 2025. Adaptive adapter routing for long-tailed class-incremental learning. Machine Learning, 114(3): 1--20

    2025

  39. [39]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In ICML, 8748--8763

    2021

  40. [40]

    Rebuffi, S.-A.; Kolesnikov, A.; Sperl, G.; and Lampert, C. H. 2017. icarl: Incremental classifier and representation learning. In CVPR, 2001--2010

    2017

  41. [41]

    H.; Bai, S.; and Tan, V

    Shi, Y.; Zhou, K.; Liang, J.; Jiang, Z.; Feng, J.; Torr, P. H.; Bai, S.; and Tan, V. Y. 2022. Mimicking the Oracle: An Initial Phase Decorrelation Approach for Class Incremental Learning. In CVPR, 16722--16731

    2022

  42. [42]

    S.; Karlinsky, L.; Gutta, V.; Cascante-Bonilla, P.; Kim, D.; Arbelle, A.; Panda, R.; Feris, R.; and Kira, Z

    Smith, J. S.; Karlinsky, L.; Gutta, V.; Cascante-Bonilla, P.; Kim, D.; Arbelle, A.; Panda, R.; Feris, R.; and Kira, Z. 2023. Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning. In CVPR, 11909--11919

    2023

  43. [43]

    Snell, J.; Swersky, K.; and Zemel, R. 2017. Prototypical networks for few-shot learning. In NIPS, 4080--4090

    2017

  44. [44]

    R.; and Shah, M

    Soomro, K.; Zamir, A. R.; and Shah, M. 2012. UCF101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402

    Pith/arXiv arXiv 2012

  45. [45]

    Sun, H.-L.; Zhou, D.-W.; Zhao, H.; Gan, L.; Zhan, D.-C.; and Ye, H.-J. 2025. MOS: Model Surgery for Pre-Trained Model-Based Class-Incremental Learning. In AAAI

    2025

  46. [46]

    Van der Maaten, L.; and Hinton, G. 2008. Visualizing data using t-SNE. JMLR, 9(11)

    2008

  47. [47]

    Wah, C.; Branson, S.; Welinder, P.; Perona, P.; and Belongie, S. 2011. The Caltech-UCSD Birds-200-2011 Dataset . Technical Report CNS-TR-2011-001, California Institute of Technology

    2011

  48. [48]

    Wang, F.-Y.; Zhou, D.-W.; Liu, L.; Ye, H.-J.; Bian, Y.; Zhan, D.-C.; and Zhao, P. 2023 a . BEEF : Bi-Compatible Class-Incremental Learning via Energy-Based Expansion and Fusion. In ICLR

    2023

  49. [49]

    Wang, R.; Duan, X.; Kang, G.; Liu, J.; Lin, S.; Xu, S.; L \"u , J.; and Zhang, B. 2023 b . Attriclip: A non-incremental learner for incremental knowledge learning. In CVPR, 3654--3663

    2023

  50. [50]

    Wang, Y.; Huang, Z.; and Hong, X. 2022. S-prompts learning with pre-trained transformers: An occam’s razor for domain incremental learning. NeurIPS, 5682--5695

    2022

  51. [51]

    Wang, Z.; Zhang, Z.; Ebrahimi, S.; Sun, R.; Zhang, H.; Lee, C.-Y.; Ren, X.; Su, G.; Perot, V.; Dy, J.; et al. 2022 a . Dualprompt: Complementary prompting for rehearsal-free continual learning. In ECCV, 631--648

    2022

  52. [52]

    Wang, Z.; Zhang, Z.; Lee, C.-Y.; Zhang, H.; Sun, R.; Ren, X.; Su, G.; Perot, V.; Dy, J.; and Pfister, T. 2022 b . Learning to prompt for continual learning. In CVPR, 139--149

    2022

  53. [53]

    A.; Oliva, A.; and Torralba, A

    Xiao, J.; Hays, J.; Ehinger, K. A.; Oliva, A.; and Torralba, A. 2010. Sun database: Large-scale scene recognition from abbey to zoo. In CVPR, 3485--3492. IEEE

    2010

  54. [54]

    Xu, J.; and Zhu, Z. 2018. Reinforced continual learning. In NeurIPS, 899--908

    2018

  55. [55]

    Yang, X.; Li, Z.; Xu, H.; Zhang, H.; Ye, Q.; Li, C.; Yan, M.; Zhang, Y.; Huang, F.; and Huang, S. 2023. Learning trajectory-word alignments for video-language tasks. In ICCV, 2504--2514

    2023

  56. [56]

    Yoon, J.; Yang, E.; Lee, J.; and Hwang, S. J. 2018. Lifelong Learning with Dynamically Expandable Networks. In ICLR

    2018

  57. [57]

    Yu, J.; Wang, Z.; Vasudevan, V.; Yeung, L.; Seyedhosseini, M.; and Wu, Y. 2024 a . Coca: Contrastive captioners are image-text foundation models. Transactions on Machine Learning Research

    2024

  58. [58]

    Yu, J.; Zhuge, Y.; Zhang, L.; Hu, P.; Wang, D.; Lu, H.; and He, Y. 2024 b . Boosting continual learning of vision-language models via mixture-of-experts adapters. In CVPR, 23219--23230

    2024

  59. [59]

    Yuan, L.; Chen, D.; Chen, Y.-L.; Codella, N.; Dai, X.; Gao, J.; Hu, H.; Huang, X.; Li, B.; Li, C.; et al. 2021. Florence: A new foundation model for computer vision. arXiv preprint arXiv:2111.11432

    Pith/arXiv arXiv 2021

  60. [60]

    Zhao, B.; Xiao, X.; Gan, G.; Zhang, B.; and Xia, S.-T. 2020. Maintaining Discrimination and Fairness in Class Incremental Learning. In CVPR, 13208--13217

    2020

  61. [61]

    Zheng, B.; Zhou, D.-W.; Ye, H.-J.; and Zhan, D.-C. 2025. Task-Agnostic Guided Feature Expansion for Class-Incremental Learning. In CVPR

    2025

  62. [62]

    Zhou, D.-W.; Cai, Z.-W.; Ye, H.-J.; Zhan, D.-C.; and Liu, Z. 2025 a . Revisiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need. International Journal of Computer Vision, 133: 1012--1032

    2025

  63. [63]

    Zhou, D.-W.; Cai, Z.-W.; Ye, H.-J.; Zhang, L.; and Zhan, D.-C. 2025 b . Dual Consolidation for Pre-Trained Model-Based Domain-Incremental Learning. In CVPR

    2025

  64. [64]

    Zhou, D.-W.; Wang, Q.-W.; Ye, H.-J.; and Zhan, D.-C. 2023. A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learning. In ICLR

    2023

  65. [65]

    Zhou, D.-W.; Zhang, Y.; Wang, Y.; Ning, J.; Ye, H.-J.; Zhan, D.-C.; and Liu, Z. 2025 c . Learning without Forgetting for Vision-Language Models. IEEE Transactions on Pattern Analysis and Machine Intelligence

    2025

  66. [66]

    C.; and Liu, Z

    Zhou, K.; Yang, J.; Loy, C. C.; and Liu, Z. 2022. Learning to prompt for vision-language models. IJCV, 130(9): 2337--2348

    2022

  67. [67]

    Zhu, F.; Zhang, X.-Y.; Wang, C.; Yin, F.; and Liu, C.-L. 2021. Prototype Augmentation and Self-Supervision for Incremental Learning. In CVPR, 5871--5880

    2021

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.