Pith. sign in

REVIEW 4 major objections 5 minor 76 references

Learning Yourself: Class-Incremental Semantic Segmentation with Language-Inspired Bootstrapped Disentanglement

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims the real failure in class-incremental semantic segmentation is 'catastrophic semantic entanglement', and that a language-guided disentanglement framework using CLIP text embeddings as fixed topological templates can undo it

desk verdict Solid but not paradigm-shifting CISS paper: the CLIP-guided disentanglement recipe is new and the ablations are honest, but missing hyperparameters and thin margins mean the SOTA claim needs careful refereeing. read the letter →

arxiv 2509.00527 v1 pith:IVESAQA7 submitted 2025-08-30 cs.CV

Ruitao Wu , Yifan Zhao , Jia Li This is my paper
classification cs.CV
keywords class-incrementalsemanticsegmentationcatastrophicentanglementCLIPprototypedisentanglementbackgroundshiftvision-languagemodelsknowledgedistillationcontrastivelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that class-incremental semantic segmentation fails not merely because models forget, but because incremental training entangles class prototypes with visual features and entangles the growing list of foreground classes with the ever-changing background. To undo that, the authors build a framework called LBD that borrows the semantic structure of a pre-trained vision-language model: hand-crafted text descriptions act as fixed templates whose pairwise geometry the learned class embeddings must preserve, while multiple learnable background prototypes are pushed apart from new classes by a mask-pooling contrastive loss. On Pascal VOC and ADE20k the framework outperforms prior methods, with the largest margins in multi-step settings—for example, 78.1 All mIoU on the VOC 15-1 overlapped scenario. If the claim holds, the standard recipe of replay and distillation is incomplete: continual segmentation needs a stable external reference for what classes mean.

What carries the argument

The load-bearing object is the topological template: a set of static text embeddings that CLIP produces from hand-crafted descriptions of each class. The paper treats inter-class similarity as a graph; the stability loss (Eq. 3) keeps distances and angles among learned class embeddings close to the template geometry, the plasticity loss (Eq. 4) pushes apart only the k most similar pairs, and the dense loss (Eq. 5) aligns patch-level score maps with template score maps. The background counterpart is a manifold of multiple learnable background prototypes whose pixel logits are max-fused; a new class inherits its prompt weights from the most similar background prototype, and a contrastive loss

What would settle it

Take a pair of classes whose CLIP text embeddings are close but whose visual appearance is clearly different (e.g., 'stairs' vs 'stairway' as flagged by the authors) and compare per-class IoU with the language-guided disentanglement loss on versus off; if enforcing the text topology hurts that pair more than it helps others, the central anchor is misaligned. Alternatively, shuffle or randomize the text templates and rerun; if the reported gains survive randomization, the language topology is not the active ingredient being claimed.

Watch

Extended reading notes

Core claim

The paper's central claim is that a two-stage abstraction of class-incremental semantic segmentation—visual feature extraction followed by matching against class prototypes—exposes a common failure mode they call catastrophic semantic entanglement. Prototype-feature entanglement is the semantic misalignment that appears when new prototypes are learned from sparse new data and drift or collide with old prototypes and features; background-increment entanglement is the confusion that appears when pixels previously labeled background become new classes, so a single static background prototype absorbs mixed semantics. The proposed solution, LBD, attacks both: language-guided prototypical disentan

Load-bearing premise

The load-bearing premise is that CLIP's text embedding space already encodes the correct inter-class relationships, so fixing the learned prototypes to that geometry helps segmentation; the paper's own limitations note that CLIP handles rare concepts poorly, which is exactly where this premise is weakest.

Editorial extensions

If this is right

  • If catastrophic semantic entanglement is the true bottleneck, replaying old pixels or distilling old logits alone cannot fix class-incremental segmentation; the model must also keep the geometry of class prototypes anchored to a stable reference.
  • Using CLIP's text embeddings as a static template lets learned class prototypes inherit a stable inter-class topology, so old classes are less likely to be pulled into new ones during later steps.
  • Representing the background with several learnable prototypes instead of one, and separating them contrastively from new classes, reduces background shift in long multi-step schedules.
  • Within this framework, parameter-efficient tuning that leaves the visual encoder frozen is insufficient: prompt-plus-adapter training reaches 72.1 All mIoU on VOC 15-1, while also fine-tuning the encoder reaches 78.1.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the benefit of this approach should grow with the number of incremental steps, since entanglement accumulates; the reported multi-step gains point that way but the causal link could be tested directly by varying step count while holding data constant.
  • Beyond the paper: the same text-as-topology idea could transfer to other continual dense tasks such as panoptic segmentation or open-vocabulary segmentation, where the background and class vocabularies also drift over time.
  • Beyond the paper: because the new disentanglement losses are training-only and detached at inference, the method's deployment cost is nearly unchanged, which makes it attractive for models that must keep adding classes in the field.
  • Beyond the paper: a natural stress test is to replace the hand-crafted text templates with learned or dataset-specific descriptions; if the gains shrink, the fixed template itself matters more than the underlying language semantics.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes LBD, a CLIP-based framework for Class-Incremental Semantic Segmentation. It reframes CISS difficulties as two forms of 'catastrophic semantic entanglement': prototype-feature entanglement and background-increment entanglement. To address these, it introduces Language-guided Prototypical Disentanglement (Eqs. 3-6), which regularizes learned class embeddings and dense logits against hand-crafted CLIP text templates, and Manifold Mutual Background Disentanglement (Eqs. 7-10), which uses multiple background prototypes and a mask-pooling contrastive loss. Experiments are reported on Pascal VOC and ADE20k under disjoint/overlapped protocols, alongside component ablations and a CLIP-backbone control experiment.

Significance. If the claims hold, the paper makes a useful contribution by showing that frozen, hand-crafted text-template geometry can act as an external anchor for continual semantic segmentation. The control experiment in Tab. 2 is a genuine strength: it goes some way toward separating the benefit of the proposed losses from the benefit of simply using a CLIP backbone. The component ablations in Tab. 4 are also informative. However, the reported gains over strong baselines are modest in several settings, and key hyperparameters and uncertainty estimates are missing, so the significance is currently tempered.

major comments (4)
  1. [Eq. (4), Sec. 3.3] The loss as written is inconsistent with its stated goal. L_plasticity = Σ_{(i,j)∈E} (1 - cos(t_i,t_j)) · 1_{i∈Ct}, with E = Top-k({cos(t_i,t_j)}), is minimized when cos(t_i,t_j) → 1, i.e., it attracts the selected prototype pairs toward each other. This is a similarity-enforcing term, not the 'orthogonality constraint' described in the text. If the intent is to push apart the most similar/confusable prototypes, the loss should be e.g. cos(t_i,t_j) or cos²(t_i,t_j); if the intent is instead to preserve local similarity, the surrounding text should be rewritten. As it stands, one of the three central losses behaves opposite to the claimed mechanism, and the sign error is load-bearing for the LPD ablation.
  2. [Eqs. (4)-(6), Sec. 4.1] Two hyperparameters that control the geometric losses are never reported: the neighborhood size k in Eq. (4) and the distillation temperature T in Eq. (5). These are not cosmetic: Eq. (4) only applies to the k most similar pairs, and Eq. (5) is multiplied by T². The values α=1, β=0.2, n=4 are stated but no sensitivity analysis is given. Without k and T, and without a sensitivity study over α/β/n, the reader cannot assess how robust the reported SOTA is to the strength of the central disentanglement losses.
  3. [Tabs. 1 and 3, Sec. 4.2] All results are reported as single runs, with no error bars, no multiple seeds, and no code release. Several claimed improvements are small relative to expected run-to-run variance in incremental segmentation (e.g., ADE20k 50-50 All: 45.8 vs 45.4; VOC 15-5 Disjoint All: 78.0 vs 78.2, where Ours is actually lower). A single run cannot establish a SOTA claim at these margins. Please provide at least three seeds with mean±std, or release the code so the comparisons can be reproduced.
  4. [Supp. Sec. 4 (Limitations); Eqs. (3)-(5)] The manuscript's own Limitations section concedes that CLIP fails to capture semantic relationships between rare concepts and other classes. Since Eqs. (3) and (5) force the learned embeddings and patch logits to reproduce template-based distances, angles, and logits, a wrong text-template topology would not merely fail to help—it would actively inject a misaligned geometry into the classifier. The paper should address this risk directly, e.g., with a per-class analysis of gains/losses for rare or visually similar classes, or with an ablation that perturbs or replaces the text templates. This is important because the empirical margins in Tabs. 1 and 3 are thin enough that template misspecification could dominate the comparison.
minor comments (5)
  1. [Eq. (2)] The notation S_i = t'_i v' should specify the inner-product/transpose convention; currently the dimensions are implicit. Also, S is later said to lie in R^{(N+1)×H×W}, but Eq. (5) writes S ∈ R^{N×M} without explaining how the background class is handled.
  2. [Eq. (9)] The notation \widehat{S}^i_bkg appears to be typeset as '[S^i_bkg' in several places. Please define this symbol and distinguish it from the i-th background mask S^i_bkg.
  3. [Eq. (4)] Even apart from the sign issue, 'Top-k({cos(t_i,t_j)})' selects the k largest cosine similarities, which correspond to the shortest k edges only if edge weights are distances. Please clarify the graph edge definition to avoid ambiguity.
  4. [Tab. 3] The claim of 'state-of-the-art on both datasets' is slightly overstated: in VOC 15-5 Disjoint, the All mIoU (78.0) is below the reproduced MBS† result (78.2). Please temper the wording or explain why this setting is an exception.
  5. [Tab. 5 / Supp. Sec. 2] The computational-cost comparison reports DenseCLIP numbers but does not state whether they are taken from the paper or re-run under the same protocol. Please clarify.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: LBD's losses are training-time regularizers anchored to frozen CLIP text templates, and the SOTA claims are evaluated on external benchmarks, not on the loss terms themselves.

full rationale

The paper's core derivation chain is: abstract CISS into feature extraction plus prototype matching, identify catastrophic semantic entanglement, and propose LBD with two disentanglement modules. The load-bearing equations are Eq. (3) (stability loss), Eq. (4) (plasticity/orthogonality), Eq. (5) (dense KL distillation), and Eq. (10) (background contrastive loss). Each is a training objective, not a prediction defined by its own outputs. L_stability forces learned class embeddings to match pairwise distances and angles of hand-crafted CLIP text templates t*; L_dense aligns patch logits with template logits. These are regularizers anchored to a frozen, externally pretrained text encoder, so they do not reduce to any fitted quantity. The reported improvements are measured as mIoU on Pascal VOC and ADE20k against published and reproduced baselines, which is independent of the loss definitions. The ablation table (Tab. 4) shows incremental mIoU gains from prompts, LPD, manifold background, and MBD, so the contribution is empirically grounded rather than definitionally forced. The 'Is All the Credit Owed to CLIP?' experiment explicitly tests the CLIP-backbone confound and finds that CLIP alone is insufficient, further supporting that the gains are not tautological. The only self-citation is reference [62] in a general list of segmentation works (Zhao and Li), and it is not load-bearing. The Limitations section concedes that CLIP fails on rare concepts; this is a robustness/correctness caveat, not evidence of circularity. Nothing in the paper fits the patterns of self-definitional prediction, fitted input renamed as prediction, or self-citation chains: the central claim is an empirical performance claim validated on external benchmarks.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The framework rests on the frozen CLIP text encoder providing valid class semantics, a fixed small manifold for background, reliable pseudo-labels from the previous model, and hand-crafted descriptions as generalized templates. These are domain assumptions rather than standard math; the central claim depends on them.

free parameters (4)
  • background prototype count n = 4
    Hand-chosen; Eq. 7-10 depend on n; no sensitivity analysis; Sec 4.1.
  • loss weights alpha and beta = alpha=1, beta=0.2
    Eq. 6 weights; chosen by hand; no sensitivity analysis; Sec 4.1.
  • Top-k neighborhood size for L_plasticity = unspecified
    Eq. 4 uses Top-k over class embedding similarities, but k is never given; determines which pairs are pushed orthogonal.
  • distillation temperature T (Eq. 5) = unspecified
    L_dense uses temperature-scaled softmax KL divergence; T is not reported, changing distillation strength.
assumptions (4)
  • domain assumption CLIP text embeddings encode a reliable inter-class semantic topology that should be preserved during incremental learning.
    Used in Section 3.3; L_stability and L_dense assume text-template geometry is the correct target for prototype disentanglement.
  • domain assumption Background can be factorized into n=4 orthogonal learnable prototypes whose max logit approximates the true background.
    Section 3.4, Eq. 7; assumes a fixed small set of prototypes captures evolving background semantics.
  • domain assumption Pseudo-labels from the previous model for old classes are reliable enough to train on.
    Sec 3.2 adopts pseudo-labeling [11]; errors in pseudo-labels propagate through distillation, an unverified premise.
  • domain assumption Hand-crafted text descriptions provide 'generalized knowledge' suitable for the target datasets.
    The nine templates in Appendix Section 1.2 are manually chosen; not validated as appropriate for VOC/ADE20k.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning Yourself: Class-Incremental Semantic Segmentation with Language-Inspired Bootstrapped Disentanglement." pith.science (2026). https://pith.science/paper/IVESAQA7

@misc{pith2026250900527,
  author       = {Pith},
  title        = {Pith review of: Learning Yourself: Class-Incremental Semantic Segmentation with Language-Inspired Bootstrapped Disentanglement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IVESAQA7}},
  note         = {Machine review of arXiv:2509.00527}
}
read the original abstract

Class-Incremental Semantic Segmentation (CISS) requires continuous learning of newly introduced classes while retaining knowledge of past classes. By abstracting mainstream methods into two stages (visual feature extraction and prototype-feature matching), we identify a more fundamental challenge termed catastrophic semantic entanglement. This phenomenon involves Prototype-Feature Entanglement caused by semantic misalignment during the incremental process, and Background-Increment Entanglement due to dynamic data evolution. Existing techniques, which rely on visual feature learning without sufficient cues to distinguish targets, introduce significant noise and errors. To address these issues, we introduce a Language-inspired Bootstrapped Disentanglement framework (LBD). We leverage the prior class semantics of pre-trained visual-language models (e.g., CLIP) to guide the model in autonomously disentangling features through Language-guided Prototypical Disentanglement and Manifold Mutual Background Disentanglement. The former guides the disentangling of new prototypes by treating hand-crafted text features as topological templates, while the latter employs multiple learnable prototypes and mask-pooling-based supervision for background-incremental class disentanglement. By incorporating soft prompt tuning and encoder adaptation modifications, we further bridge the capability gap of CLIP between dense and sparse tasks, achieving state-of-the-art performance on both Pascal VOC and ADE20k, particularly in multi-step scenarios.

Figures

Figures reproduced from arXiv: 2509.00527 by the authors.

Figure 1
Figure 1. Illustration of Catastrophic Semantic Entanglement (Orange) and our countermeasures (Green). (a) Prototype￾Feature Entanglement caused by the inter-class topology disrup￾tion. (b) Background-Increment Entanglement caused by the dy￾namically evolving foreground. We address the two issues through language-guided prototypical disentanglement (Sec. 3.3) and man￾ifold mutual background disentanglement (Sec. 3.4). an imag… view at source ↗
Figure 2
Figure 2. Illustration of our Language-inspired Bootstrapped Disentanglement pipeline. (a) The overall architecture of CISS, includ￾ing the CLIP encoder and learnable prompts. (b) Mutual Background Disentanglement. CLIP-derived old class masks and ground-truth labels are used to calculate pooling-based features to achieve mutual disentanglement of the background and new classes. (c) Manifold Background Representation with Sel… view at source ↗
Figure 3
Figure 3. Illustration of Language-guided protoypical disen￾tangle. Macroscopically, the topological structure of prototypes must be maintained. Microscopically, the local semantic plastic￾ity must be ensured. The former is achieved through relationship distillation between class embeddings and templates, while the lat￾ter relies on maximum similarity constraints. Cross-modal dense learning further maintains the generalizatio… view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Visualization of the confusion matrix. After ap [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 4
Figure 4. Figure 4: Visualization of class embeddings across different iters [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

76 extracted references · 73 canonical work pages

  1. [1]

    Decomposed Knowledge Distillation for Class-Incremental Semantic Segmentation

    Donghyeon Baek, Youngmin Oh, Sanghoon Lee, Junghyup Lee, and Bumsub Ham. Decomposed knowledge distilla- tion for class-incremental semantic segmentation. ArXiv, abs/2210.05941, 2022. 1

    work page Pith review arXiv 2022

  2. [2]

    Rainbow memory: Continual learning with a memory of diverse samples

    Jihwan Bang, Heesu Kim, Youngjoon Yoo, Jung-Woo Ha, and Jonghyun Choi. Rainbow memory: Continual learning with a memory of diverse samples. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, vir- tual, June 19-25, 2021, pages 8218–8227. Computer Vision Foundation / IEEE, 2021. 2

    work page 2021

  3. [3]

    Modeling the back- ground for incremental learning in semantic segmentation

    Fabio Cermelli, Massimiliano Mancini, Samuel Rota Bul `o, Elisa Ricci, and Barbara Caputo. Modeling the back- ground for incremental learning in semantic segmentation. In 2020 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, CVPR 2020, Seattle, WA, USA, June 13- 19, 2020, pages 9230–9239. Computer Vision Foundation / IEEE, 2020. 2, 7, 8

    work page 2020

  4. [4]

    Incremental learning in semantic segmentation from image labels

    Fabio Cermelli, Dario Fontanel, Antonio Tavera, Marco Cic- cone, and Barbara Caputo. Incremental learning in semantic segmentation from image labels. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 4361–4371. IEEE, 2022. 2

    work page 2022

  5. [5]

    Com- former: Continual learning in semantic and panoptic seg- mentation

    Fabio Cermelli, Matthieu Cord, and Arthur Douillard. Com- former: Continual learning in semantic and panoptic seg- mentation. In IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023 , pages 3010–3020. IEEE, 2023. 3

    work page 2023

  6. [6]

    Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L. Yuille. Deeplab: Semantic im- age segmentation with deep convolutional nets, atrous con- volution, and fully connected crfs.IEEE Trans. Pattern Anal. Mach. Intell., 40(4):834–848, 2018. 1

    work page 2018

  7. [7]

    Schwing, Alexan- der Kirillov, and Rohit Girdhar

    Bowen Cheng, Ishan Misra, Alexander G. Schwing, Alexan- der Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022 , pages 1280–1289. IEEE, 2022. 1, 3

    work page 2022

  8. [8]

    Reproducible scaling laws for contrastive language-image learning

    Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuh- mann, Ludwig Schmidt, and Jenia Jitsev. Reproducible scaling laws for contrastive language-image learning. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2818–2829, 2022. 6, 1

    work page 2023

Show all 76 references
  1. [9]

    MTA- CLIP: language-guided semantic segmentation with mask- text alignment

    Anurag Das, Xinting Hu, Li Jiang, and Bernt Schiele. MTA- CLIP: language-guided semantic segmentation with mask- text alignment. In Computer Vision - ECCV 2024 - 18th Eu- ropean Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part LIV, pages 39–56. Springe...

    2024

  2. [10]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...

    2010 arXiv

  3. [11]

    Plop: Learning without forgetting for contin- ual semantic segmentation

    Arthur Douillard, Yifu Chen, Arnaud Dapogny, and Matthieu Cord. Plop: Learning without forgetting for contin- ual semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 4040–4050, 2021. 3, 4, 7, 8

    2021

  4. [12]

    Mark Everingham, S. M. Ali Eslami, Luc Van Gool, Christo- pher K. I. Williams, John M. Winn, and Andrew Zisser- man. The pascal visual object classes challenge: A retro- spective. International Journal of Computer Vision, 111:98 – 136, 2014. 6

    2014

  5. [13]

    knn-clip: Retrieval enables training-free segmenta- tion on continually expanding large vocabularies

    Zhongrui Gui, Shuyang Sun, Runjia Li, Jianhao Yuan, Zhaochong An, Karsten Roth, Ameya Prabhu, and Philip Torr. knn-clip: Retrieval enables training-free segmenta- tion on continually expanding large vocabularies. CoRR, abs/2404.09447, 2024. 3

    2024 arXiv

  6. [14]

    Zhang, Shaoqing Ren, and Jian Sun

    Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Spa- tial pyramid pooling in deep convolutional networks for vi- sual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37:1904–1916, 2014. 1

    1904

  7. [15]

    Lyu, Dashan Gao, and Nuno Vasconcelos

    Zhiyuan Hu, J. Lyu, Dashan Gao, and Nuno Vasconcelos. Pop: Prompt of prompts for continual learning. ArXiv, abs/2306.08200, 2023. 4

    2023 arXiv

  8. [16]

    Belongie, Bharath Hariharan, and Ser Nam Lim

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge J. Belongie, Bharath Hariharan, and Ser Nam Lim. Vi- sual prompt tuning. ArXiv, abs/2203.12119, 2022. 4

    2022 arXiv

  9. [17]

    ECLIPSE: efficient continual learning in panoptic segmen- tation with visual prompt tuning

    Beomyoung Kim, Joonsang Yu, and Sung Ju Hwang. ECLIPSE: efficient continual learning in panoptic segmen- tation with visual prompt tuning. In IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024 , pages 3346–

    2024

  10. [18]

    Overcoming catastrophic forgetting in neu- ral networks

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barwinska, et al. Overcoming catastrophic forgetting in neu- ral networks. Proceedings of the national academy of sc...

    2017

  11. [19]

    Clearclip: Decom- posing CLIP representations for dense vision-language in- ference

    Mengcheng Lan, Chaofeng Chen, Yiping Ke, Xinjiang Wang, Litong Feng, and Wayne Zhang. Clearclip: Decom- posing CLIP representations for dense vision-language in- ference. In Computer Vision - ECCV 2024 - 18th European Conference, Milan, Italy, September 29-October 4, 2024, Pro...

    2024

  12. [20]

    Continual pro- totype evolution: Learning online from non-stationary data streams

    Matthias De Lange and Tinne Tuytelaars. Continual pro- totype evolution: Learning online from non-stationary data streams. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, Octo- ber 10-17, 2021, pages 8230–8239. IEEE, 2021. 2

    2021

  13. [21]

    Continual learning with extended kronecker- factored approximate curvature

    Janghyeon Lee, Hyeong Gwon Hong, Donggyu Joo, and Junmo Kim. Continual learning with extended kronecker- factored approximate curvature. In 2020 IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020 , pages 8998–

    2020

  14. [22]

    Learn to grow: A continual structure learn- ing framework for overcoming catastrophic forgetting

    Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. Learn to grow: A continual structure learn- ing framework for overcoming catastrophic forgetting. In Proceedings of the 36th International Conference on Ma- chine Learning, ICML 2019, 9-15 June 2019, Long Bea...

    2019

  15. [23]

    A closer look at the explainability of con- trastive language-image pre-training

    Yi Li, Hualiang Wang, Yiqun Duan, Jiheng Zhang, and Xi- aomeng Li. A closer look at the explainability of con- trastive language-image pre-training. Pattern Recognition, 162:111409, 2025. 4

    2025

  16. [24]

    Continual semantic segmentation via structure preserving and projected feature alignment

    Zihan Lin, Zilei Wang, and Yixin Zhang. Continual semantic segmentation via structure preserving and projected feature alignment. In European Conference on Computer Vision , pages 345–361. Springer, 2022. 7, 8

    2022

  17. [25]

    Learning from the web: Language drives weakly- supervised incremental learning for semantic segmentation

    Chang Liu, Giulia Rizzoli, Pietro Zanuttigh, Fu Li, and Yi Niu. Learning from the web: Language drives weakly- supervised incremental learning for semantic segmentation. In Computer Vision - ECCV 2024 - 18th European Confer- ence, Milan, Italy, September 29-October 4, 2024, Pr...

    2024

  18. [26]

    Dynamic extension nets for few-shot se- mantic segmentation

    Lizhao Liu, Junyi Cao, Minqian Liu, Yong Guo, Qi Chen, and Mingkui Tan. Dynamic extension nets for few-shot se- mantic segmentation. Proceedings of the 28th ACM Interna- tional Conference on Multimedia, 2020. 1

    2020

  19. [27]

    A new generative replay approach for incremental class learning of medical image for semantic segmentation

    Mingyang Liu, Li Xiao, Huiqin Jiang, and Qing He. A new generative replay approach for incremental class learning of medical image for semantic segmentation. In Proceedings of the 2022 International Conference on Intelligent Medicine and Health, page 51–56, New York, NY , USA,...

    2022

  20. [28]

    Decoupled weight de- cay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight de- cay regularization. In International Conference on Learning Representations, 2017. 6

    2017

  21. [29]

    Recall: Replay-based continual learning in semantic segmentation

    Andrea Maracani, Umberto Michieli, Marco Toldo, and Pietro Zanuttigh. Recall: Replay-based continual learning in semantic segmentation. In Proceedings of the IEEE/CVF international conference on computer vision , pages 7026– 7035, 2021. 1, 8

    2021

  22. [30]

    Incremental learning techniques for semantic segmentation

    Umberto Michieli and Pietro Zanuttigh. Incremental learning techniques for semantic segmentation. In 2019 IEEE/CVF International Conference on Computer Vision Workshops, ICCV Workshops 2019, Seoul, Korea (South), October 27-28, 2019, pages 3205–3212. IEEE, 2019. 1, 2, 8

    2019

  23. [31]

    Incremental learn- ing techniques for semantic segmentation

    Umberto Michieli and Pietro Zanuttigh. Incremental learn- ing techniques for semantic segmentation. 2019 IEEE/CVF International Conference on Computer Vision Workshop (IC- CVW), pages 3205–3212, 2019. 1

    2019

  24. [32]

    Continual semantic segmentation via repulsion-attraction of sparse and disentan- gled latent representations

    Umberto Michieli and Pietro Zanuttigh. Continual semantic segmentation via repulsion-attraction of sparse and disentan- gled latent representations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 1114–1124, 2021. 1, 3, 7, 8

    2021

  25. [33]

    Mitigating background shift in class- incremental semantic segmentation

    Gilhan Park, WonJun Moon, SuBeen Lee, Tae-Young Kim, and Jae-Pil Heo. Mitigating background shift in class- incremental semantic segmentation. In Computer Vision - ECCV 2024 - 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part L , pages 71–...

    2024

  26. [34]

    Re- lational knowledge distillation

    Wonpyo Park, Dongju Kim, Yan Lu, and Minsu Cho. Re- lational knowledge distillation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3962–3971, 2019. 5

    2019

  27. [35]

    Class similarity weighted knowl- edge distillation for continual semantic segmentation

    Minh Hieu Phan, Son Lam Phung, Long Tran-Thanh, Abdes- selam Bouzerdoum, et al. Class similarity weighted knowl- edge distillation for continual semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 16866–16875, 2022. 7, 8

    2022

  28. [36]

    Class similarity weighted knowledge distillation for continual semantic seg- mentation

    Minh-Hieu Phan, The-Anh Ta, Son Lam Phung, Long Tran- Thanh, and Abdesselam Bouzerdoum. Class similarity weighted knowledge distillation for continual semantic seg- mentation. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16845–16854, 2022. 1

    2022

  29. [37]

    Sats: Self-attention transfer for continual semantic segmentation

    Yiqiao Qiu, Yixing Shen, Zhuohao Sun, Yanchong Zheng, Xiaobin Chang, Weishi Zheng, and Ruixuan Wang. Sats: Self-attention transfer for continual semantic segmentation. ArXiv, abs/2203.07667, 2022. 1

    2022 arXiv

  30. [38]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of th...

    2021

  31. [39]

    Denseclip: Language-guided dense prediction with context- aware prompting

    Yongming Rao, Wenliang Zhao, Guangyi Chen, Yansong Tang, Zheng Zhu, Guan Huang, Jie Zhou, and Jiwen Lu. Denseclip: Language-guided dense prediction with context- aware prompting. In IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, CVPR 2022, New Orleans, LA, U...

    2022

  32. [40]

    Micro: Modeling cross-image semantic relationship dependencies for class-incremental semantic segmentation in remote sens- ing images

    Xuee Rong, Peijin Wang, Wenhui Diao, Yiran Yang, Wenxin Yin, Xuan Zeng, Hongqi Wang, and Xian Sun. Micro: Modeling cross-image semantic relationship dependencies for class-incremental semantic segmentation in remote sens- ing images. IEEE Transactions on Geoscience and Remote ...

    2023

  33. [41]

    Rasp: Relation-aware semantic prior for weakly su- pervised incremental segmentation

    Subhankar Roy, Riccardo V olpi, Gabriela Csurka, and Diane Larlus. Rasp: Relation-aware semantic prior for weakly su- pervised incremental segmentation. In Conference on Life- long Learning Agents, 22-25 August 2023, McGill Univer- sity, Montr ´eal, Qu ´ebec, Canada , pages 24...

    2023

  34. [42]

    Incrementer: Transformer for class-incremental semantic segmentation with knowl- edge distillation focusing on old class

    Chao Shang, Hongliang Li, Fanman Meng, Qingbo Wu, Heqian Qiu, and Lanxiao Wang. Incrementer: Transformer for class-incremental semantic segmentation with knowl- edge distillation focusing on old class. In IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, CVPR 2...

    2023

  35. [43]

    Segmenter: Transformer for semantic segmenta- tion

    Robin Strudel, Ricardo Garcia, Ivan Laptev, and Cordelia Schmid. Segmenter: Transformer for semantic segmenta- tion. In 2021 IEEE/CVF International Conference on Com- puter Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 7242–7252. IEEE, 2021. 1

    2021

  36. [44]

    FOSTER: feature boosting and compression for class- incremental learning

    Fu-Yun Wang, Da-Wei Zhou, Han-Jia Ye, and De-Chuan Zhan. FOSTER: feature boosting and compression for class- incremental learning. In Computer Vision - ECCV 2022 - 17th European Conference, Tel Aviv, Israel, October 23- 27, 2022, Proceedings, Part XXV, pages 398–414. Springer,

    2022

  37. [45]

    Ordisco: Effective and efficient usage of incremental unlabeled data for semi-supervised continual learning

    Liyuan Wang, Kuo Yang, Chongxuan Li, Lanqing Hong, Zhenguo Li, and Jun Zhu. Ordisco: Effective and efficient usage of incremental unlabeled data for semi-supervised continual learning. In IEEE Conference on Computer Vi- sion and Pattern Recognition, CVPR 2021, virtual, June 19...

    2021

  38. [46]

    Dy, and Tomas Pfister

    Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jen- nifer G. Dy, and Tomas Pfister. Learning to prompt for continual learning. 2022 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR) , pages 139– 149, 2021. 4

    2022

  39. [47]

    Reinforced continual learning

    Ju Xu and Zhanxing Zhu. Reinforced continual learning. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Sys- tems 2018, NeurIPS 2018, December 3-8, 2018, Montr ´eal, Canada, pages 907–916, 2018. 2

    2018

  40. [48]

    DER: dynam- ically expandable representation for class incremental learn- ing

    Shipeng Yan, Jiangwei Xie, and Xuming He. DER: dynam- ically expandable representation for class incremental learn- ing. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021 , pages 3014–3023. Computer Vision Foundation / IEEE, 2021. 2

    2021

  41. [49]

    Der: Dynam- ically expandable representation for class incremental learn- ing

    Shipeng Yan, Jiangwei Xie, and Xuming He. Der: Dynam- ically expandable representation for class incremental learn- ing. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3013–3022, 2021. 1

    2021

  42. [50]

    Deep model reassembly

    Xingyi Yang, Zhou Daquan, Songhua Liu, Jingwen Ye, and Xinchao Wang. Deep model reassembly. ArXiv, abs/2210.17409, 2022. 1

    2022 arXiv

  43. [51]

    Adaptive deep models for incremental learning: Considering capacity scalability and sustainability

    Yang Yang, Da-Wei Zhou, De-Chuan Zhan, Hui Xiong, and Yuan Jiang. Adaptive deep models for incremental learning: Considering capacity scalability and sustainability. In Pro- ceedings of the 25th ACM SIGKDD International Confer- ence on Knowledge Discovery & Data Mining, KDD 20...

    2019

  44. [52]

    Cost-effective incremental deep model: Matching model capacity with the least sampling

    Yang Yang, Da-Wei Zhou, De-Chuan Zhan, Hui Xiong, Yuan Jiang, and Jian Yang. Cost-effective incremental deep model: Matching model capacity with the least sampling. IEEE Trans. Knowl. Data Eng., 35(4):3575–3588, 2023. 2

    2023

  45. [53]

    Learning with recoverable forgetting

    Jingwen Ye, Yifang Fu, Jie Song, Xingyi Yang, Songhua Liu, Xin Jin, Mingli Song, and Xinchao Wang. Learning with recoverable forgetting. ArXiv, abs/2207.08224, 2022. 1

    2022 arXiv

  46. [54]

    Lifelong learning with dynamically expandable net- works

    Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable net- works. In 6th International Conference on Learning Rep- resentations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings . OpenRe- ...

    2018

  47. [55]

    Foundation model drives weakly incremental learning for semantic segmentation

    Chaohui Yu, Qiang Zhou, Jingliang Li, Jianlong Yuan, Zhibin Wang, and Fan Wang. Foundation model drives weakly incremental learning for semantic segmentation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17- 24, 2023...

    2023

  48. [56]

    Tikp: Text-to-image knowledge preservation for continual seman- tic segmentation

    Zhidong Yu, Wei Yang, Xike Xie, and Zhenbo Shi. Tikp: Text-to-image knowledge preservation for continual seman- tic segmentation. Proceedings of the AAAI Conference on Artificial Intelligence, 38(15):16596–16604, 2024. 1

    2024

  49. [57]

    A survey on continual seman- tic segmentation: Theory, challenge, method and applica- tion

    Bo Yuan and Danpei Zhao. A survey on continual seman- tic segmentation: Theory, challenge, method and applica- tion. IEEE Trans. Pattern Anal. Mach. Intell., 46(12):10891– 10910, 2024. 1

    2024

  50. [58]

    Frozen CLIP: A strong backbone for weakly supervised semantic segmentation

    Bingfeng Zhang, Siyue Yu, Yunchao Wei, Yao Zhao, and Jimin Xiao. Frozen CLIP: A strong backbone for weakly supervised semantic segmentation. In IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024 , pages 3796–

    2024

  51. [59]

    Representation compensation networks for continual semantic segmentation

    Chang-Bin Zhang, Jia-Wen Xiao, Xialei Liu, Ying-Cong Chen, and Ming-Ming Cheng. Representation compensation networks for continual semantic segmentation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7053–7064, 2022. 3, 7, 8

    2022

  52. [60]

    Memory-efficient class-incremental learning for image clas- sification

    Hanbin Zhao, Hui Wang, Yongjian Fu, Fei Wu, and Xi Li. Memory-efficient class-incremental learning for image clas- sification. IEEE Trans. Neural Networks Learn. Syst. , 33 (10):5966–5977, 2022. 2

    2022

  53. [61]

    RBC: rectifying the biased context in continual semantic segmen- tation

    Hanbin Zhao, Fengyu Yang, Xinghe Fu, and Xi Li. RBC: rectifying the biased context in continual semantic segmen- tation. In Computer Vision - ECCV 2022 - 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceed- ings, Part XXXIV, pages 55–72. Springer, 2022. 7, 8

    2022

  54. [62]

    From pose to part: Weakly-supervised pose evolution for human part segmentation

    Yifan Zhao, Jia Li, Yu Zhang, and Yonghong Tian. From pose to part: Weakly-supervised pose evolution for human part segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45:3107–3120, 2022. 1

    2022

  55. [63]

    Scene parsing through ade20k dataset

    Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5122–5130, 2017. 6

    2017

  56. [64]

    Extract free dense labels from CLIP

    Chong Zhou, Chen Change Loy, and Bo Dai. Extract free dense labels from CLIP. In Computer Vision - ECCV 2022 - 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXVIII , pages 696–712. Springer,

    2022

  57. [65]

    A model or 603 exemplars: Towards memory-efficient class-incremental learning

    Da-Wei Zhou, Qi-Wei Wang, Han-Jia Ye, and De-Chuan Zhan. A model or 603 exemplars: Towards memory-efficient class-incremental learning. In The Eleventh International Conference on Learning Representations, ICLR 2023, Ki- gali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. 2

    2023

  58. [66]

    Learning to prompt for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. Int. J. Comput. Vis., 130(9):2337–2348, 2022. 2

    2022

  59. [67]

    Zegclip: Towards adapting CLIP for zero-shot semantic segmentation

    Ziqin Zhou, Yinjie Lei, Bowen Zhang, Lingqiao Liu, and Yifan Liu. Zegclip: Towards adapting CLIP for zero-shot semantic segmentation. In IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, CVPR 2023, Vancou- ver, BC, Canada, June 17-24, 2023 , pages 11175–11185. ...

    2023

  60. [68]

    Prototype augmentation and self-supervision for incremental learning

    Fei Zhu, Xu-Yao Zhang, Chuang Wang, Fei Yin, and Cheng- Lin Liu. Prototype augmentation and self-supervision for incremental learning. In IEEE Conference on Computer Vi- sion and Pattern Recognition, CVPR 2021, virtual, June 19- 25, 2021, pages 5871–5880. Computer Vision Found...

    2021

  61. [70]

    Visual Encoder Since the original version of CLIP [8, 38] was trained on classification tasks at the image level, it cannot be directly applied to segmentation tasks

    Model Details 1.1. Visual Encoder Since the original version of CLIP [8, 38] was trained on classification tasks at the image level, it cannot be directly applied to segmentation tasks. To address this, we syn- thesized insights from existing methods and implemented the follow...

  62. [71]

    Following MaskCLIP [64], we removed the average pooling in the last layer of the CLIP visual encoder ViT, which allows us to obtain dense features

  63. [72]

    Additionally, we replaced the attention mechanism in the final layer with v-v attention

    Following ClearCLIP [19], we directly removed the feedforward neural network and residual connections from the last layer of ViT. Additionally, we replaced the attention mechanism in the final layer with v-v attention

  64. [73]

    This feature was then used as input to the decoder

    Inspired by the concept of multi-scale feature extrac- tion [14], we first extracted features from different layers of the CLIP visual encoder (specifically, the 4th, 6th, 8th, and 12th layers), concatenated them along the feature dimen- sion, and then used convolution operati...

  65. [74]

    Analysis of Computational Cost In the domain of Continual Learning (CL), model efficiency is as crucial as performance. To provide a clear perspective on the computational overhead of our proposed Language- inspired Bootstrapped Disentanglement (LBD) method, we conduct a compa...

  66. [75]

    Exploration of PEFT The advent of large-scale pre-trained models has spurred the development of Parameter-Efficient Fine-Tuning (PEFT) methods, which aim to adapt these models to downstream tasks by updating only a small fraction of their parame- ters. To assess the feasibilit...

  67. [76]

    Limitations Our method relies on explicit class names, and when only images and numeric labels are available in the dataset, we are unable to leverage textual information. Moreover, due to the limitations of CLIP’s pretraining data, CLIP fails to capture the semantic relations...

  68. [9007]

    Computer Vision Foundation / IEEE, 2020. 2

    2020

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.