Pith. sign in

REVIEW 3 major objections 4 minor 71 references

This paper argues that catastrophic forgetting in continually adapted vision-language models is concentrated at the old-new semantic boundary, and that probing this interface with a handful of adversarial anchors is enough to preserve old k

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 18:17 UTC pith:QRBRXMIU

load-bearing objection Serious and worth a referee: adversarial anchors are a fresh idea, but the central interface-drift claim rests on one error-bar-free figure and the missing distributional check keeps the mechanism plausible rather than proven. the 3 major comments →

arxiv 2603.12055 v3 pith:QRBRXMIU submitted 2026-03-12 cs.CV cs.LG

Continual Learning with Vision-Language Models via Semantic-Geometry Preservation

Chiyuan He , Zihuan Qiu , Fanman Meng , Runtong Zhang , Linfeng Xu , Qingbo Wu , Hongliang Li This is my paper
classification cs.CV cs.LG
keywords continual learningvision-language modelsCLIPcatastrophic forgettingsemantic geometryadversarial anchorsknowledge distillationprototype transfer
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

SeGP-CL is built on a specific diagnosis: when a vision-language model like CLIP is tuned to a new batch of classes, the damage to previously learned knowledge does not spread evenly across the embedding space, but clusters in the neighborhood where old and new class semantics overlap, because shared visual patterns get re-assigned to the new texts. To protect exactly this vulnerable region, the paper constructs adversarial anchors — new-task images pushed by dual-targeted projected gradient descent toward an old class's text embedding while held close to that class's raw visual prototype — and then applies anchor-guided cross-modal geometry distillation, plus a light regularization of the text-space neighborhood graph. After each task, it estimates the drift revealed by the anchors, transfers the old visual prototypes accordingly, and fuses cross-modal and prototype-based logits at inference. The paper reports state-of-the-art final-task accuracy and the lowest forgetting on five benchmarks, while retaining zero-shot robustness. A sympathetic reader would take this as evidence that forgetting in VLMs is a localized geometric event, and that targeted probing at the boundary is a more economical remedy than replay, synthesis, or global regularization.

Core claim

The central discovery is that forgetting in continually adapted vision-language models is concentrated near the old-new semantic interface, and that a compact set of adversarial anchors generated by Dual-targeted Projected Gradient Descent (DPGD) can probe and protect that region without storing old images. Each anchor starts from a new-task seed with affinity to an old class, is pushed toward the old class's text embedding, and is held near the old class's raw visual prototype to compensate for the modality gap. Anchor-guided cross-modal geometry distillation (ACGD) then preserves the teacher's old-class distribution on these anchors, while text semantic-geometry regularization (TSGR) keeps

What carries the argument

The load-bearing object is the adversarial anchor, produced by Dual-targeted Projected Gradient Descent (DPGD). DPGD takes a new-task image that already resembles an old class, then minimizes a dual-target objective: a text-targeted term that pushes the image's CLIP embedding toward the old class's text embedding, plus a raw-visual term that pulls its raw feature toward the old class's stored visual prototype, all inside a small ℓ∞ budget. Because of the modality gap, the raw-visual term is what keeps the anchor visually plausible; the text term is what places it in the vulnerable boundary zone. These anchors then support three mechanisms: ACGD distills the teacher's old-class distribution o

Load-bearing premise

The argument depends on the adversarial anchors being faithful stand-ins for the real old-class visual neighborhood — if DPGD anchors drift away from genuine old-class image features, the distillation constrains a region that is not where actual forgetting occurs, and the prototype-transfer step inherits that bias.

What would settle it

Compare SeGP-CL to a variant that generates DPGD anchors toward a randomly chosen old class instead of the seed-selected boundary class, keeping the budget, distillation, and prototype transfer identical; if Last accuracy and Forgetting do not change, then boundary targeting is not the operative mechanism, and the adversarial anchors are merely acting as generic regularizers.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the boundary-probing claim is correct, a few dozen adversarial anchors per old class are sufficient to preserve old knowledge, so exemplar-free VLM continual learning can be both memory-light and effective.
  • Distillation on targeted anchors should outperform both distillation on raw new-task data, which hurts new-task learning, and distillation on generic reference data, which is less targeted — the paper reports +5.8 Last on CIFAR100 over naive LoRA fine-tuning.
  • Preserving the canonical cross-modal geometry should keep zero-shot transfer intact even after 10 tasks of adaptation, which the paper verifies on Food101, Oxford-Pets, and ImageNet-full.
  • Anchor-based drift estimation gives a principled way to transfer old visual prototypes under the exemplar-free constraint, supporting the dual-path inference that combines CLIP logits with prototype logits.
  • Because the cross-modal constraints regularize the visual encoder too, the visual-branch-only evaluation should also improve — and the paper reports lower forgetting than a frozen-CLIP visual classifier.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the 'drift concentrates at the semantic interface' claim could be tested independently by measuring per-neighborhood JSD shifts and correlating them with per-class forgetting across the task sequence; a weak correlation would indicate the mechanism is less localized than claimed.
  • Beyond the paper: the anchor-selection heuristic (top-K seeds by teacher similarity to old text) could be replaced by diversity-aware selection or per-class budgets, and the method might extend to multi-modal domains by choosing a different seed distribution.
  • Beyond the paper: a natural stress test is to apply SeGP-CL to a distribution-shift setting (e.g., adapting from ImageNet to Sketch or to a medical domain), where the notion of 'shared visual patterns' between old and new classes is less reliable; if forgetting returns, the method's assumption of appearance-based overlap is exposed.
  • Beyond the paper: the finding suggests that for any continual learning scheme on dual-encoder models, protecting the decision boundary between old and new is more cost-effective than regularizing the whole old manifold; this could motivate similar boundary-probing techniques in prompt-learning and adapter-based continual learning.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes SeGP-CL, an exemplar-free continual learning method for CLIP that explicitly preserves the cross-modal semantic geometry inherited from pretraining and previous tasks. The central claim is that forgetting arises from harmful drift concentrated near the old-new semantic interface, where shared visual patterns are re-explained by new textual semantics. To protect this region without old data, the method constructs DPGD adversarial anchors from new-task seeds, distills old-class cross-modal logits on these anchors (ACGD), regularizes the textual semantic geometry via a k-NN subgraph (TSGR), and after training estimates anchor-induced raw-space drift to transfer old visual prototypes, followed by dual-path prediction. Experiments on five class-incremental benchmarks report state-of-the-art Avg/Last accuracy, improved FWT/BWT/Forgetting, and retained zero-shot robustness, together with extensive ablations of components, distillation temperatures, budgets, and costs.

Significance. If the mechanism holds, the paper makes a useful contribution: it identifies a structural locus of forgetting — the old-new semantic interface — and provides a targeted, exemplar-free way to constrain updates there, rather than applying generic distillation or reference-data regularization. The strengths are genuine: the method is described in sufficient detail for re-implementation, code is released, ablations are broad (Tables III, IV, VI; Figs. 6-8), no target result is fitted into the loss, and hyperparameters are tuned on validation splits. The reported gains, e.g., CIFAR100 Last 84.6 vs. 80.6 for MG-CLIP and CUB-200 Last 80.1 vs. 76.2 for RAPF, are substantial. However, the central mechanistic story — that DPGD anchors faithfully probe old-class raw-visual neighborhoods and that the gains come from protecting the old-new interface — is not directly validated. The evidence offered is indirect (one JSD curve without error bars) and the anchor-faithfulness assumption is load-bearing for ACGD and for the prototype-transfer step. The paper therefore needs additional distributional checks before the central claim is established.

major comments (3)
  1. [Sec. III-C, Eq. (13)] The paper's core mechanism assumes that DPGD anchors are faithful probes of old-class raw-visual neighborhoods. Eq. (13) combines a text-targeted cross-modal loss (11) with a single per-class raw-prototype attraction term (12, λp=0.5). This constrains the anchor toward the old class mean, not the old-class visual manifold; a new-task image can be pushed toward the mean without entering the old-class distribution. Fig. 6 even shows the text branch oscillating and converging differently from the visual branch, and Fig. 11 shows that different ε produce qualitatively different anchors, but no experiment measures whether anchors actually resemble old-class features in raw space. Because old test data are available for evaluation, a quantitative check is feasible: e.g., compare the feature distribution of anchors to real old-class features (distance, precision/recall), and ablate DPGD against
  2. [Sec. IV-B, Fig. 2(a)] The central empirical observation — that drift concentrates near the old-new semantic interface — is supported by a single JSD curve without error bars, without a statistical comparison, and without reporting how it is computed across tasks or classes. This is the motivation for the entire anchor machinery, so the evidence should be more than illustrative. I request quantification: error bars over seeds, a comparison of JSD at the interface vs. core old regions with a statistical test, and at least one more benchmark to show the phenomenon is not dataset-specific.
  3. [Sec. III-E, Eqs. (27)-(29)] The prototype-transfer step inherits any bias in the anchor drift estimate. The weighted average displacement (27) and magnitude modulation (28) assume that anchor-induced raw-space translation approximates the true translation of old-class prototypes. This is not validated. Since old-class test data are available at evaluation time, the authors could compute the true prototype drift and compare it with the estimated drift, or at least study prototype-transfer performance when using the true drift as an oracle. Without such a check, the PT gains in Table IV (Last +0.4 on CIFAR100, +0.7 on UCF) may come from an incidental regularizer rather than from correct drift compensation.
minor comments (4)
  1. [Table I / Table V] No standard deviations or multiple-seed results are reported for the main tables. Given that several key claims are based on 1-4 point improvements, error bars or at least a statement about runs would increase confidence.
  2. [Eq. (33)] In the definition of FWT, the indexing R_{j-1,j} appears inconsistent with the surrounding notation (which uses R_{i,j} for accuracy on task j after training on task i). Please clarify.
  3. [Table III] The 'Naive (baseline)' row in Table III presents values in parentheses that are ambiguous; it would be clearer to state the baseline Avg/Last explicitly and then report deltas.
  4. [Sec. IV-H] The limitation paragraph mentions historical memory of raw texts and visual prototypes but gives no numbers. Since the paper emphasizes exemplar-freedom, a brief quantification of this extra storage/computation (e.g., KB per task) would be useful.

Circularity Check

0 steps flagged

No significant circularity: all regularizers are computed from teacher/pretrained reference states, and no claimed prediction reduces to a fitted input or self-citation chain.

full rationale

The paper's core machinery—DPGD anchor construction (Eqs. 9–14), ACGD distillation (Eq. 18), TSGR (Eq. 21), prototype transfer (Eqs. 24–29), and dual-path inference (Eqs. 30–31)—is computed from frozen teacher snapshots, pretrained text-encoder reference G0, and new-task data, not from the benchmark accuracies it reports. No target result is built into the definition of the loss: the regularization terms match teacher and student distributions on anchors and text subgraphs, which are standard distillation objectives with external content. Prototype transfer is a heuristic drift estimate based on anchors; it could be unvalidated or unsuccessful if anchors are not faithful probes of old-class neighborhoods, but that is a correctness/validation concern, not a demonstration that the output equals the input by construction. Self-citations in the paper (e.g., refs. [8], [9], [17], [31]) appear as background or competitor methods and are not load-bearing for the central derivation. The paper does not invoke a uniqueness theorem, does not smuggle in an ansatz via self-citation, and does not rename a known result as a new prediction. Hyperparameter choices (λ_p, λ_ACGD, λ_GR, β) are tuned on validation splits, which is standard practice and does not by itself constitute circularity. Therefore no circular step can be exhibited, and the appropriate score is 0.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 1 invented entities

The method introduces seven hand-set hyperparameters that co-determine the headline numbers, two working assumptions that are load-bearing (anchor representativeness and text-space stability), and one new entity (adversarial anchors) with no external validation. The paper reports ablations for several of these, which is honest, but the final SOTA claim depends on all of them together being well-tuned.

free parameters (7)
  • DPGD perturbation budget epsilon = 4/255 (Fig. 11); step gamma=1.5e-3
    Chosen by ablating epsilon; larger radii hurt performance (Fig. 11), so the operating point is tuned on the benchmark rather than derived.
  • lambda_p (visual anchoring weight) = 0.5
    Set in Sec. IV-A; no sensitivity analysis exposed, yet it controls how much anchors stay near the raw prototype vs the text target.
  • lambda_ACGD / lambda_GR (loss weights) = 5 / 1
    Chosen by hand; no ablation reported in the paper.
  • tau_A (ACGD temperature) = 20
    Ablated in Fig. 7; the claimed best value is an empirical choice.
  • tau_T (TSGR temperature) = 0.05
    Ablated in Fig. 7.
  • beta (visual-branch ensembling weight) = 0.5
    Set at inference (Sec. IV-A); its value co-determines the reported Last accuracy, so the final numbers depend on this hand choice.
  • K_seed, K_adv, k (subgraph) = 5, 10, 10
    K_adv and k are ablated (Figs. 6, 8); K_seed is stated without ablation.
axioms (5)
  • domain assumption CLIP's pretrained text encoder provides a stable reference frame across tasks and text embeddings are more stable than visual ones.
    TSGR rests on resetting LoRA to recover a shared text space (Sec. III-D, Eq. 21); the paper cites [57] for this, but it is a working assumption.
  • ad hoc to paper Adversarial anchors generated on new-task seeds with DPGD are representative of old-class raw-visual drift.
    Prototype transfer (Eq. 27-29) assumes the anchor-induced displacement estimates the true old-class prototype drift; with no old data, this is unverified.
  • ad hoc to paper The old-new interface is the dominant forgetting locus.
    Fig. 2a measures JSD shift in one setup without statistics; the entire anchor-targeting design depends on this being true and measurable.
  • domain assumption The LoRA-updated visual/text encoders stay close enough to the teacher for distillation to be valid.
    ACGD (Eq. 18) assumes student-snapshot distributions are comparable and that the teacher is trustworthy after multiple tasks; no check of teacher-student distance is provided.
  • domain assumption The pretrained CLIP text embeddings and hand-crafted prompt template 'A photo of a {ClassName}' adequately represent class-level semantic geometry.
    The paper's own Limitations (Sec. IV-H) admits the semantic-relevance structure depends on prompt quality.
invented entities (1)
  • DPGD adversarial anchors no independent evidence
    purpose: Probe vulnerable old-new semantic boundary and serve as distillation data and drift estimators
    These are optimized perturbations of new-task images constrained by the teacher model. They have no external falsifiable handle: the paper does not show that anchors transfer to other backbones or that they match real old-class data (old data is unavailable by design).

pith-pipeline@v1.3.0-alltime-deepseek · 21460 in / 9056 out tokens · 69412 ms · 2026-08-02T18:17:47.957014+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Continual Learning with Vision-Language Models via Semantic-Geometry Preservation." pith.science (2026). https://pith.science/paper/QRBRXMIU

@misc{pith2026260312055,
  author       = {Pith},
  title        = {Pith review of: Continual Learning with Vision-Language Models via Semantic-Geometry Preservation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QRBRXMIU}},
  note         = {Machine review of arXiv:2603.12055}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Continual learning of pretrained vision-language models (VLMs) is prone to catastrophic forgetting, yet current approaches adapt to new tasks without explicitly preserving the cross-modal semantic geometry inherited from pretraining and previous stages, allowing new-task supervision to induce geometric distortion. We observe that the most pronounced drift tends to concentrate in vulnerable neighborhoods near the old-new semantic interface, where shared visual patterns are easily re-explained by new textual semantics. To address this under an exemplar-free constraint, we propose Semantic Geometry Preservation for Continual Learning (SeGP-CL). SeGP-CL first probes the drift-prone region by constructing a compact set of adversarial anchors with dual-targeted projected gradient descent (DPGD), which drives selected new-task seeds toward old-class semantics while remaining faithful in raw visual space. During training, we preserve cross-modal structure by anchor-guided cross-modal geometry distillation (ACGD), and stabilize the textual reference frame across tasks via a lightweight text semantic-geometry regularization (TSGR). After training, we estimate anchor-induced raw-space drift to transfer old visual prototypes and perform dual-path inference by fusing cross-modal and visual cues. Extensive experiments on five continual learning benchmarks demonstrate that SeGP-CL consistently improves stability and forward transfer, achieving state-of-the-art performance while better preserving semantic geometry of VLMs. Code is available at: https://github.com/chiyuan-IVIPLab/SeGP-CL.

Figures

Figures reproduced from arXiv: 2603.12055 by Chiyuan He, Fanman Meng, Hongliang Li, Linfeng Xu, Qingbo Wu, Runtong Zhang, Zihuan Qiu.

Figure 1
Figure 1. Figure 1: Boundary vulnerability in VLM-based continual learning and our [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Empirical evidence of boundary vulnerability and comparison of [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of proposed SeGP-CL. 1) Anchor construction: Dual-targeted projected gradient descent (DPGD) iteratively perturbs seed samples to synthesize adversarial anchors that are simultaneously guided in raw visual space and CLIP feature space. 2) Continual learning: A LoRA-tuned VLM is optimized on task batches with CE loss, while anchor batches and history texts impose semantic-geometry preservation via … view at source ↗
Figure 4
Figure 4. Figure 4: After training: We estimate raw-space drift and transfer visual [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison with state-of-the-art CL methods in terms of per-task accuracy and global transfer. All results are achieved on the same CLIP ViT-B/16 [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 8
Figure 8. Figure 8: Ablation of anchor batch size and number of subgraph node. [PITH_FULL_IMAGE:figures/full_fig_p009_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Evaluation of cross-modal transfer robustness on CIFAR100. We [PITH_FULL_IMAGE:figures/full_fig_p009_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: DPGD optimization trajectories of adversarial anchors in visual [PITH_FULL_IMAGE:figures/full_fig_p011_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Effect of perturbation budget on adversarial anchors. We visualize [PITH_FULL_IMAGE:figures/full_fig_p011_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

71 extracted references · 3 linked inside Pith

  1. [1]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clarket al., “Learning transferable visual models from natural language supervision,” inInternational conference on machine learning. PMLR, 2021, pp. 8748–8763

    2021

  2. [2]

    Scaling up visual and vision-language representation learning with noisy text supervision,

    C. Jia, Y . Yang, Y . Xia, Y .-T. Chen, Z. Parekh, H. Pham, Q. Le, Y .-H. Sung, Z. Li, and T. Duerig, “Scaling up visual and vision-language representation learning with noisy text supervision,” inInternational conference on machine learning. PMLR, 2021, pp. 4904–4916

    2021

  3. [3]

    Csta: Spatial-temporal causal adaptive learning for exemplar-free video class-incremental learning,

    T. Chen, H. Liu, C. Hong Lim, J. See, X. Gao, J. Hou, and W. Lin, “Csta: Spatial-temporal causal adaptive learning for exemplar-free video class-incremental learning,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 35, no. 11, pp. 11 488–11 501, 2025

    2025

  4. [4]

    Achieving plasticity- stability trade-off in continual learning through adaptive orthogonal projection,

    D. Cheng, Y . Hu, N. Wang, D. Zhang, and X. Gao, “Achieving plasticity- stability trade-off in continual learning through adaptive orthogonal projection,”IEEE Transactions on Circuits and Systems for Video Technology, 2025

    2025

  5. [5]

    Joint memory optimiza- tion for continual learning,

    Z. Ma, Y . Ma, X. Hong, H. Li, and S. Zhang, “Joint memory optimiza- tion for continual learning,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 35, no. 7, pp. 6961–6973, 2025. 12

    2025

  6. [6]

    Learning without forgetting for vision-language models,

    D.-W. Zhou, Y . Zhang, Y . Wang, J. Ning, H.-J. Ye, D.-C. Zhan, and Z. Liu, “Learning without forgetting for vision-language models,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 47, no. 6, pp. 4489–4504, 2025

    2025

  7. [7]

    Class-aware prompting for federated few-shot class- incremental learning,

    F.-Y . Liang, Y .-W. Zhan, J. Liu, C.-Y . Zhang, Z.-D. Chen, X. Luo, and X.-S. Xu, “Class-aware prompting for federated few-shot class- incremental learning,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 35, no. 9, pp. 8520–8532, 2025

    2025

  8. [8]

    Mingle: Mixture of null-space gated low-rank experts for test-time continual model merging,

    Z. Qiu, Y . Xu, C. He, F. Meng, L. Xu, Q. Wu, and H. Li, “Mingle: Mixture of null-space gated low-rank experts for test-time continual model merging,” inAdvances in Neural Information Processing Systems (NeurIPS), 2025, poster (OpenReview). [Online]. Available: https://openreview.net/forum?id=8DCyv8x58O

    2025

  9. [9]

    Null-space filtering for data-free continual model merging: Preserving stability, promoting plasticity,

    Z. Qiu, L. Wang, Y . Cao, R. Zhang, B. Su, Y . Xu, F. Meng, L. Xu, Q. Wu, and H. Li, “Null-space filtering for data-free continual model merging: Preserving stability, promoting plasticity,” inInternational Conference on Learning Representations (ICLR), 2026, poster. [Online]. Available: https://openreview.net/forum?id=HDIf3fYqPP

    2026

  10. [10]

    New insights on relieving task-recency bias for online class incremental learning,

    G. Liang, Z. Chen, Z. Chen, S. Ji, and Y . Zhang, “New insights on relieving task-recency bias for online class incremental learning,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 34, no. 5, pp. 3451–3464, 2024

    2024

  11. [11]

    Continual learning of image classes with language guidance from a vision-language model,

    W. Zhang, Y . Huang, W. Zhang, T. Zhang, Q. Lao, Y . Yu, W.-S. Zheng, and R. Wang, “Continual learning of image classes with language guidance from a vision-language model,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 34, no. 12, pp. 13 152–13 163, 2024

    2024

  12. [12]

    Ex- ternal knowledge injection for clip-based class-incremental learning,

    D.-W. Zhou, K.-W. Li, J. Ning, H.-J. Ye, L. Zhang, and D.-C. Zhan, “Ex- ternal knowledge injection for clip-based class-incremental learning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2025, pp. 3314–3325

    2025

  13. [13]

    Class-incremental learning with clip: Adaptive representation adjustment and parameter fusion,

    L. Huang, X. Cao, H. Lu, and X. Liu, “Class-incremental learning with clip: Adaptive representation adjustment and parameter fusion,” inComputer Vision – ECCV 2024, 2024, pp. 214–231. [Online]. Available: https://dblp.org/rec/conf/eccv/HuangCLL24

    2024

  14. [14]

    Overcoming generic knowledge loss with selective parameter update,

    W. Zhang, P. Janson, R. Aljundi, and M. Elhoseiny, “Overcoming generic knowledge loss with selective parameter update,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, 2024, pp. 24 046–24 056

    2024

  15. [15]

    Slca: Slow learner with classifier alignment for continual learning on a pre-trained model,

    G. Zhang, L. Wang, G. Kang, L. Chen, and Y . Wei, “Slca: Slow learner with classifier alignment for continual learning on a pre-trained model,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 19 148–19 158

    2023

  16. [16]

    Boosting continual learning of vision-language models via mixture-of-experts adapters,

    J. Yu, Y . Zhuge, L. Zhang, P. Hu, D. Wang, H. Lu, and Y . He, “Boosting continual learning of vision-language models via mixture-of-experts adapters,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 23 219–23 230

    2024

  17. [17]

    Desclip: Robust continual learning via general attribute descriptions for vlm-based visual recognition,

    C. He, Z. Qiu, F. Meng, L. Xu, Q. Wu, and H. Li, “Desclip: Robust continual learning via general attribute descriptions for vlm-based visual recognition,”IEEE Transactions on Multimedia, pp. 1–16, 2026

    2026

  18. [18]

    Language guided concept bottleneck models for interpretable continual learning,

    L. Yu, H. Han, Z. Tao, H. Yao, and C. Xu, “Language guided concept bottleneck models for interpretable continual learning,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, 2025, pp. 14 976–14 986

    2025

  19. [19]

    Hierarchical semantic tree anchoring for clip-based class-incremental learning,

    T. Hu, L. Li, Z.-H. Xie, and D.-W. Zhou, “Hierarchical semantic tree anchoring for clip-based class-incremental learning,”arXiv preprint arXiv:2511.15633, 2025

    arXiv 2025

  20. [20]

    Difference vector equalization for robust fine-tuning of vision-language models,

    S. Suzuki, S. Yamaguchi, S. Takeda, T. Yamane, N. Makishima, N. Kawata, M. Ihori, T. Tanaka, S. Orihashi, and R. Masumura, “Difference vector equalization for robust fine-tuning of vision-language models,”arXiv preprint arXiv:2511.09973, 2025

    arXiv 2025

  21. [21]

    Mind the gap: Preserving and compensating for the modality gap in clip- based continual learning,

    L. Huang, X. Cao, H. Lu, Y . Meng, F. Yang, and X. Liu, “Mind the gap: Preserving and compensating for the modality gap in clip- based continual learning,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2025, pp. 3777–3786

    2025

  22. [22]

    Preventing zero-shot transfer degradation in continual learning of vision-language models,

    Z. Zheng, M. Ma, K. Wang, Z. Qin, X. Yue, and Y . You, “Preventing zero-shot transfer degradation in continual learning of vision-language models,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 19 125–19 136

    2023

  23. [23]

    Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts,

    S. Changpinyo, P. Sharma, N. Ding, and R. Soricut, “Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 3558–3568

    2021

  24. [24]

    Imagenet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009, pp. 248–255

    2009

  25. [25]

    Divergence measures based on the shannon entropy,

    J. Lin, “Divergence measures based on the shannon entropy,”IEEE Transactions on Information Theory, vol. 37, no. 1, pp. 145–151, 1991

    1991

  26. [26]

    Text-guided attention is all you need for zero-shot robustness in vision-language models,

    L. Yu, H. Zhang, and C. Xu, “Text-guided attention is all you need for zero-shot robustness in vision-language models,” inAdvances in Neural Information Processing Systems, 2024

    2024

  27. [27]

    Evaluating the adversarial robustness of vision-language models via internal feature perturbations,

    C. Liu, Y . Wang, H. Cao, B. Liu, and D. Jiang, “Evaluating the adversarial robustness of vision-language models via internal feature perturbations,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 36, no. 3, pp. 3938–3950, 2026

    2026

  28. [28]

    Attention-guided hierarchical defense for multimodal attacks in vision-language models,

    C. Chen, Z. Li, Y . Zhang, X. Guo, and W. Zhang, “Attention-guided hierarchical defense for multimodal attacks in vision-language models,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2024, pp. 5209–5218

    2024

  29. [29]

    Synthetic data is an elegant gift for continual vision-language models,

    B. Wu, W. Shi, J. Wang, and M. Ye, “Synthetic data is an elegant gift for continual vision-language models,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2025, pp. 2813–2823

    2025

  30. [30]

    Lora-loop: Closing the synthetic re- play cycle for continual vlm learning,

    K. Wang, D. Kim, and M. Betke, “Lora-loop: Closing the synthetic re- play cycle for continual vlm learning,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, Oc- tober 2025, pp. 445–454

    2025

  31. [31]

    Dual-consistency model inversion for non-exemplar class incremental learning,

    Z. Qiu, Y . Xu, F. Meng, H. Li, L. Xu, and Q. Wu, “Dual-consistency model inversion for non-exemplar class incremental learning,” in2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, 2024, pp. 24 025–24 035

    2024

  32. [32]

    Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning,

    V . W. Liang, Y . Zhang, Y . Kwon, S. Yeung, and J. Y . Zou, “Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning,”Advances in Neural Information Processing Systems, vol. 35, pp. 17 612–17 625, 2022

    2022

  33. [33]

    Clip-lora: Low-rank adaptation for clip,

    M. Zanella and I. Ben Ayed, “Clip-lora: Low-rank adaptation for clip,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2024

    2024

  34. [34]

    Learning to prompt for vision- language models,

    K. Zhou, J. Yang, C. C. Loy, and Z. Liu, “Learning to prompt for vision- language models,”International Journal of Computer Vision, vol. 130, no. 9, pp. 2337–2348, 2022

    2022

  35. [35]

    Conditional prompt learning for vision-language models,

    K. Zhou, J. Yang, C. C. Loy, and Z. Liu, “Conditional prompt learning for vision-language models,” inProceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, 2022, pp. 16 816– 16 825

    2022

  36. [36]

    Graphadapter: Tuning vision-language models with dual knowledge graph,

    X. Li, D. Lian, Z. Lu, J. Bai, Z. Chen, and X. Wang, “Graphadapter: Tuning vision-language models with dual knowledge graph,”Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  37. [37]

    Vmt-adapter: Parameter- efficient transfer learning for multi-task dense scene understanding,

    Y . Xin, J. Du, Q. Wang, Z. Lin, and K. Yan, “Vmt-adapter: Parameter- efficient transfer learning for multi-task dense scene understanding,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 14, 2024, pp. 16 085–16 093

    2024

  38. [38]

    Lora: Low-rank adaptation of large language models

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chenet al., “Lora: Low-rank adaptation of large language models.” ICLR, vol. 1, no. 2, p. 3, 2022

    2022

  39. [39]

    A model or 603 exemplars: Towards memory-efficient class-incremental learning,

    D.-W. Zhou, Q.-W. Wang, H.-J. Ye, and D.-C. Zhan, “A model or 603 exemplars: Towards memory-efficient class-incremental learning,” inThe Eleventh International Conference on Learning Representations, 2022

    2022

  40. [40]

    Ceat: Continual expansion and absorption transformer for non-exemplar class- incremental learning,

    S. Dong, X. Gao, Y . He, Z. Zhou, A. C. Kot, and Y . Gong, “Ceat: Continual expansion and absorption transformer for non-exemplar class- incremental learning,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 35, no. 4, pp. 3146–3159, 2024

    2024

  41. [41]

    Learning without forgetting,

    Z. Li and D. Hoiem, “Learning without forgetting,”IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 12, pp. 2935– 2947, 2017

    2017

  42. [42]

    Overcoming catastrophic forgetting in neural networks,

    J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska et al., “Overcoming catastrophic forgetting in neural networks,”Pro- ceedings of the national academy of sciences, vol. 114, no. 13, pp. 3521–3526, 2017

    2017

  43. [43]

    Learning to prompt for continual learning,

    Z. Wang, Z. Zhang, C.-Y . Lee, H. Zhang, R. Sun, X. Ren, G. Su, V . Perot, J. Dy, and T. Pfister, “Learning to prompt for continual learning,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 139–149

    2022

  44. [44]

    Coda-prompt: Contin- ual decomposed attention-based prompting for rehearsal-free continual learning,

    J. S. Smith, L. Karlinsky, V . Gutta, P. Cascante-Bonilla, D. Kim, A. Arbelle, R. Panda, R. Feris, and Z. Kira, “Coda-prompt: Contin- ual decomposed attention-based prompting for rehearsal-free continual learning,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 11 909–11 919

    2023

  45. [45]

    Expandable subspace ensemble for pre-trained model-based class-incremental learning,

    D.-W. Zhou, H.-L. Sun, H.-J. Ye, and D.-C. Zhan, “Expandable subspace ensemble for pre-trained model-based class-incremental learning,” in 13 Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 23 554–23 564

    2024

  46. [46]

    Inflora: Interference-free low-rank adaptation for continual learning,

    Y .-S. Liang and W.-J. Li, “Inflora: Interference-free low-rank adaptation for continual learning,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 23 638–23 647

    2024

  47. [47]

    Consistent prompting for rehearsal- free continual learning,

    Z. Gao, J. Cen, and X. Chang, “Consistent prompting for rehearsal- free continual learning,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 28 463–28 473

    2024

  48. [48]

    Revisiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need,

    D.-W. Zhou, Z.-W. Cai, H.-J. Ye, D.-C. Zhan, and Z. Liu, “Revisiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need,”International Journal of Computer Vision, vol. 133, no. 3, pp. 1012–1032, 2025

    2025

  49. [49]

    Exploiting the semantic knowledge of pre-trained text-encoders for continual learning,

    L. Yu, Z. Tao, D. Goswami, H. Yao, B. Twardowski, J. Van de Weijer, C. Xuet al., “Exploiting the semantic knowledge of pre-trained text-encoders for continual learning,”arXiv preprint arXiv:2408.01076, 2024

    Pith/arXiv arXiv 2024

  50. [50]

    Select and distill: Selective dual-teacher knowledge transfer for continual learning on vision-language models,

    Y .-C. Yu, C.-P. Huang, J.-J. Chen, K.-P. Chang, Y .-H. Lai, F.-E. Yang, and Y .-C. F. Wang, “Select and distill: Selective dual-teacher knowledge transfer for continual learning on vision-language models,” inComputer Vision – ECCV 2024, 2024, pp. 219–236. [Online]. Available: https://dblp.org/rec/conf/eccv/YuHCCLYW24

    2024

  51. [51]

    Adapt without forgetting: Distill proximity from dual teachers in vision-language models,

    M. Zheng, Y . Tang, Z. Hao, K. Han, Y . Wang, and C. Xu, “Adapt without forgetting: Distill proximity from dual teachers in vision-language models,” inComputer Vision – ECCV 2024, 2024, pp. 109–125. [Online]. Available: https://dblp.org/rec/conf/eccv/ZhengTHHWX24

    2024

  52. [52]

    Explaining and harnessing adversarial examples,

    I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” inInternational Conference on Learning Repre- sentations (ICLR), 2015, poster

    2015

  53. [53]

    Adversarial examples in the physical world,

    A. Kurakin, I. J. Goodfellow, and S. Bengio, “Adversarial examples in the physical world,” inArtificial intelligence safety and security. Chapman and Hall/CRC, 2018, pp. 99–112

    2018

  54. [54]

    Towards deep learning models resistant to adversarial attacks,

    A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” in International Conference on Learning Representations (ICLR), 2018, poster. [Online]. Available: https://openreview.net/forum?id=rJzIBfZAb

    2018

  55. [55]

    Bofa: Bridge-layer orthogonal low-rank fusion for clip-based class-incremental learning,

    L. Li, T. Hu, D.-W. Zhou, H.-J. Ye, and D.-C. Zhan, “Bofa: Bridge-layer orthogonal low-rank fusion for clip-based class-incremental learning,” arXiv preprint arXiv:2511.11421, 2025

    Pith/arXiv arXiv 2025

  56. [56]

    On information and sufficiency,

    S. Kullback and R. A. Leibler, “On information and sufficiency,”The annals of mathematical statistics, vol. 22, no. 1, pp. 79–86, 1951

    1951

  57. [57]

    Dynamic multi-layer null space projection for vision-language continual learning,

    B. Kang, L. Wang, Z. Wu, T. Feng, Y . Li, Y . Gao, and W. Li, “Dynamic multi-layer null space projection for vision-language continual learning,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2025, pp. 2077–2086

    2025

  58. [58]

    Exemplar-free continual representation learning via learnable drift compensation,

    A. Gomez-Villa, D. Goswami, K. Wang, A. D. Bagdanov, B. Twar- dowski, and J. van de Weijer, “Exemplar-free continual representation learning via learnable drift compensation,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 473–490

    2024

  59. [59]

    Resurrecting old classes with new data for exemplar-free continual learning,

    D. Goswami, A. Soutif-Cormerais, Y . Liu, S. Kamath, B. Twardowski, J. Van De Weijeret al., “Resurrecting old classes with new data for exemplar-free continual learning,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 28 525–28 534

    2024

  60. [60]

    Clip model is an efficient continual learner,

    V . Thengane, S. Khan, M. Hayat, and F. Khan, “Clip model is an efficient continual learner,”arXiv preprint arXiv:2210.03114, 2022

    Pith/arXiv arXiv 2022

  61. [61]

    Clap4clip: Continual learning with probabilistic finetuning for vision-language models,

    S. Jha, D. Gong, and L. Yao, “Clap4clip: Continual learning with probabilistic finetuning for vision-language models,” inAdvances in Neural Information Processing Systems, 2024

    2024

  62. [62]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky, G. Hintonet al., “Learning multiple layers of features from tiny images,”Toronto, ON, Canada, 2009

    2009

  63. [63]

    The caltech-ucsd birds-200-2011 dataset,

    C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie, “The caltech-ucsd birds-200-2011 dataset,”California Institute of Technology, 2011

    2011

  64. [64]

    The many faces of robustness: A critical analysis of out-of-distribution generalization,

    D. Hendrycks, K. Zhao, S. Basart, J. Steinhardt, and D. Song, “The many faces of robustness: A critical analysis of out-of-distribution generalization,” inICCV, 2021

    2021

  65. [65]

    Ucf101: A dataset of 101 human actions classes from videos in the wild,

    K. Soomro, A. R. Zamir, and M. Shah, “Ucf101: A dataset of 101 human actions classes from videos in the wild,” Center for Research in Computer Vision, University of Central Florida, Tech. Rep. CRCV-TR-12-01, 2012. [Online]. Available: https://www.crcv.ucf.edu/ papers/UCF101 CRCV-TR-12-01.pdf

    2012

  66. [66]

    Large-scale machine learning with stochastic gradient de- scent,

    L. Bottou, “Large-scale machine learning with stochastic gradient de- scent,” inProceedings of the 19th International Conference on Compu- tational Statistics (COMPSTAT). Springer, 2010, pp. 177–186

    2010

  67. [67]

    High- resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10 684–10 695

    2022

  68. [68]

    Food-101–mining dis- criminative components with random forests,

    L. Bossard, M. Guillaumin, and L. Van Gool, “Food-101–mining dis- criminative components with random forests,” inEuropean conference on computer vision. Springer, 2014, pp. 446–461

    2014

  69. [69]

    Cats and dogs,

    O. M. Parkhi, A. Vedaldi, A. Zisserman, and C. Jawahar, “Cats and dogs,” in2012 IEEE conference on computer vision and pattern recog- nition. IEEE, 2012, pp. 3498–3505

    2012

  70. [70]

    Ad- vancing cross-domain discriminability in continual learning of vision- language models,

    Y . Xu, Y . Chen, J. Nie, Y . Wang, H. Zhuang, and M. Okumura, “Ad- vancing cross-domain discriminability in continual learning of vision- language models,”Advances in Neural Information Processing Systems, vol. 37, pp. 51 552–51 576, 2024. Chiyuan Herecieved his M.S. degree in Information and Communication Engineering at the University of Electronic Scie...

    2024

  71. [2005]

    Video segmentation and its applications

    From 2005 to 2006, he joined the visual signal processing and communication laboratory (VSPC) of the Chinese University of Hong Kong (CUHK) as a Research Associate. From 2006 to 2008, he was a Postdoctoral Fellow at the same laboratory in CUHK. He is currently a Professor in the School of Information and Communication Engineering, University of Electronic...

    2005