Pith. sign in

REVIEW 5 major objections 6 minor 87 references

Continuous Knowledge-Preserving Decomposition with Adaptive Layer Selection for Few-Shot Class-Incremental Learning

T0 review · 5 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims covariance-guided decomposition of pretrained weights lets few-shot class-incremental learning beat existing methods with zero inference overhead.

desk verdict A conceptually appealing but currently overclaimed FSCIL method: the adaptive layer selection is the genuinely useful part, but the theoretical motivation does not hold up and the evidence tables do not support consistent SOTA. read the letter →

arxiv 2501.05017 v3 pith:XH7ON7ZU submitted 2025-01-09 cs.CV

Xiaojie Li , Jianlong Wu , Yue Yu , Liqiang Nie , Min Zhang This is my paper · ORCID
classification cs.CV
keywords few-shotclass-incrementallearningcatastrophicforgettingknowledge-preservingdecompositionadaptivelayerselectionlow-rankadaptationinputcovariancevisiontransformerstability-plasticitybalance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a pretrained vision transformer already contains enough unused capacity to keep learning new classes without forgetting, provided each linear layer is split into a frozen knowledge-sensitive part and a learnable redundant part. It claims the correct split can be found by multiplying the weight matrix $W$ by the input covariance $\Sigma_{\mathrm{in}}$ and taking a singular value decomposition: the top singular directions carry prior knowledge, the bottom $r$ form a safe low-rank adapter. A second mechanism ranks layers by the fraction of energy in that redundant subspace (the Adapter Sensitivity Ratio) and adapts only the safest layers. If right, the method achieves top accuracy on several benchmarks while adding zero parameters and zero extra FLOPs at inference, because each session's adapter is merged back into the original weights.

What carries the argument

The load-bearing object is the covariance-weighted singular value decomposition of $W\Sigma_{\mathrm{in}}$ with reconstruction through $W=USV^{\top}\Sigma_{\mathrm{in}}^{-1}$. Its singular values rank each input-to-output direction by how strongly it shapes the output covariance, which the paper identifies with task-discriminative knowledge. The bottom $r$ singular components are factorized into a low-rank adapter $BA$; the top $R-r$ define $W_{\mathrm{frozen}}$ as the residual $W-BA$, guaranteeing zero initial drift. The Adapter Sensitivity Ratio, $\mathrm{ASR}(\ell)=\sum_{i=R-r+1}^{R}s_i/\sum_{i=1}^{R}s_i$, then selects the $K$ layers whose redundant subspace carries the smallest share of energy, because changing those layers is safest.

What would settle it

Compute the same CKPD pipeline on CIFAR-100 but replace $\Sigma_{\mathrm{in}}$ with a random matrix that has the same singular values and is rotationally uncorrelated with the data; if the resulting incremental accuracy matches the true-covariance version, then the covariance guidance is not doing the work the paper claims. A sharper check: after the frozen subspace is built, zero out or randomly perturb $W_{\mathrm{frozen}}$ and measure base-class accuracy; if it drops as much as perturbing $W_{\mathrm{learnable}}$, the top singular directions are not the knowledge-bearing ones.

Watch

Extended reading notes

Core claim

The central discovery is that covariance-guided decomposition of pretrained weights turns the stability-plasticity trade-off into a subspace-allocation problem. For a layer with weight $W$ and input covariance $\Sigma_{\mathrm{in}}$, the output covariance satisfies $\Sigma_{\mathrm{out}}=W\Sigma_{\mathrm{in}}W^{\top}$, so the directions of $W\Sigma_{\mathrm{in}}$ that most amplify input structure are the ones that matter for previously learned classes. The paper defines the frozen subspace from the top $R-r$ singular components and the learnable adapter from the bottom $r$, reconstructing $W$ exactly as $W_{\mathrm{frozen}}+BA$ so that no prediction changes before training. The decomposition is recomputed each session with updated covariance statistics, and the layer selection is redone with the Adapter Sensitivity Ratio, the share of a layer's total singular-value energy in its redundant subspace. The claim is that this yields both higher adaptability and better retention than freezing, full fine-tuning, or adding external prompt and adapter modules.

Load-bearing premise

The whole approach rests on the assumption that the directions of the weight matrix that most amplify the input-data covariance are exactly the directions needed to remember old classes, so cutting off the smallest directions leaves knowledge intact.

Editorial extensions

If this is right

  • If the claim holds, FSCIL no longer requires a choice between freezing the backbone and accumulating per-session modules: the same parameter count and inference cost suffice.
  • Because adapters are merged after each session, deployed models can keep growing in knowledge without growing in size, which matters for on-device continual learning.
  • The continuous recalibration of the decomposition predicts that static decompositions will drift out of alignment, and the paper reports that the continuous version outperforms the one-shot version on CIFAR-100 and CUB-200.
  • The ASR-based selection implies that layer choice can be automated from data statistics instead of manual heuristics, and the paper's selection maps largely avoid middle transformer blocks.
  • Applying the same low-rank repurposing inside an existing continual-learning method improves its accuracy, suggesting the mechanism transfers beyond few-shot settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same machinery should transfer to other transformer-based continual settings, since it only requires linear layers, an input covariance estimate, and a mergeable low-rank update; language models and multimodal models are natural candidates.
  • An implicit consequence is that the amount of safe adaptation capacity is bounded by how much truly redundant spectral energy the pretrained weights contain; on heavily compressed or highly optimized models, the bottom-$r$ subspace may be too small to carry new classes, and accuracy gains would shrink.
  • One testable extension is to replace the single random sample per class in the covariance buffer with a small ensemble of samples: the paper reports low seed variance, but measuring whether covariance quality degrades as class count grows would indicate when the one-sample buffer stops being sufficient.
  • The ASR ranking could itself be used as a no-validation heuristic for choosing adapter rank per layer, since the metric already reports how much energy would be touched by adaptation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes CKPD-FSCIL, a few-shot class-incremental learning method that decomposes selected linear layers of a ViT backbone into a frozen 'knowledge-sensitive' subspace and a learnable low-rank 'redundant' subspace, then merges the learned adapters back into the weights after each session. Layer selection is driven by an Adapter Sensitivity Ratio (ASR) computed from the same decomposition. The method is evaluated on CUB-200, CIFAR-100, miniImageNet, ImageNet-1K, and a standard CIL benchmark, reporting state-of-the-art average accuracies and low performance drops while adding zero parameters and zero FLOPs at inference.

Significance. If the mechanism worked as claimed, the paper would make a useful contribution: it targets two under-explored aspects of FSCIL (intra-layer redundancy and layer-wise sensitivity), provides a continuous recalibration scheme, and achieves zero inference overhead by weight merging. The manuscript has concrete strengths: the code is released, the experiments cover several backbones and benchmarks, the appendix-style ablations include adapter dropout and random-seed robustness, and the efficiency analysis separately measures offline and online training cost. However, the central theoretical motivation for the decomposition is mathematically flawed, and the reported tables contain unexplained duplicate rows. The contribution is therefore not currently established.

major comments (5)
  1. [Sec. 3.2.1, Eq. (1)-(4)] The SVD of WΣ_in does not diagonalize the output covariance that the motivation is built on. From Eq. (1), if WΣ_in = U S V^T then W = U S V^T Σ_in^{-1}, so Σ_out = WΣ_in W^T = U S (V^T Σ_in^{-1} V) S^T U^T, which is not diagonal in general because V^T Σ_in^{-1} V is not diagonal. The matrix whose SVD genuinely diagonalizes Σ_out is WΣ_in^{1/2}, not WΣ_in. Thus the singular vectors and singular values of WΣ_in do not, by the paper's own definition, isolate the directions of W that are most important for the model's discriminative output structure.
  2. [Sec. 3.2.1/3.2.2, Eq. (3)] The 'input covariance' used in the method is a raw second moment E[xx^T] estimated from one sample per class, not a centered covariance and not a between-class scatter. The theoretical motivation appeals to classical discriminant analysis, whose relevant object is inter-class versus intra-class scatter; Eq. (3) cannot capture intra-class covariance with one sample per class. The claim that the resulting spectrum ranks knowledge-sensitive directions is therefore unsupported by the stated LDA-style motivation.
  3. [Sec. 3.2.2, Eq. (5)-(10)] A small singular value s_i of WΣ_in does not imply that the corresponding component has a small effect in the original input space. After the reconstruction in Eq. (5), the i-th component is s_i u_i (Σ_in^{-1} v_i)^T, whose operator norm is s_i times the norm of Σ_in^{-1} v_i; the inverse covariance can amplify a small s_i considerably. Consequently, the bottom-r subspace is not guaranteed to be safe to adapt, and the top subspace is not guaranteed to be knowledge-critical. The exact-residual construction of W_frozen in Eq. (10) only guarantees that the initial output is unchanged; it does not repair the incorrect ranking.
  4. [Sec. 3.3.1, Eq. (13)] The ASR metric is defined from the same singular value decomposition used to create the redundant subspace: a low ASR is, by construction, the condition that the bottom-r singular values have small total energy. The sentence 'A low ASR indicates that the redundant subspace contributes minimally to the layer's overall function' is therefore definitionally aligned with the decomposition rather than an independent measure of forgetting risk. The empirical comparison against manual and uniform layer selection in Fig. 4 provides useful evidence, but it does not validate ASR as a measure of safety outside the paper's own decomposition choice.
  5. [Tables 2, 3, 4] The main comparison tables contain multiple CKPD-FSCIL rows with the same backbone but different base session accuracies and no identifying configuration: Table 2 has two ViT-B-CLIP rows, Table 3 has three ViT-B-CLIP rows, and Table 4 has two ViT-B-CLIP rows. Without labels explaining differences in training protocol, hyperparameters, or settings, the claim that CKPD-FSCIL 'consistently outperforms' state-of-the-art approaches is ambiguous and the results are not reproducible from the paper as written.
minor comments (6)
  1. [Sec. 2.2] The sentence 'While effective, these methods face several critical:' is incomplete and should be reworded.
  2. [Sec. 2.2] There is a duplicated word in 'KANet [22] and and CA-CLIP [20]' that should be corrected.
  3. [Sec. 3.2.2, Eq. (3)] The text says features are centered by Layer Normalization, but Eq. (3) is a raw second moment without subtracting a mean; the notation and the prose should be made consistent.
  4. [Fig. 5] The figure axis labels use 'ffn0, ffn1' while the text and Fig. 2 refer to 'FFN1, FFN2'; the naming should be unified.
  5. [Table 12] Table 12 states that CUB-200 incremental sessions are trained for 2000 iterations, while Sec. 4.1 specifies 1000 iterations for CUB-200; the inconsistency should be resolved.
  6. [Sec. 3.3.2] The paragraph 'At the start of each incremental session, ALS computes...' contains a repeated sentence; one of the two occurrences should be removed.

Circularity Check

1 steps flagged · score 3.0 of 10

ASR safety interpretation is definitionally tied to the SVD-based redundant subspace, but the central benchmark claims remain empirically grounded.

  1. self definitional [Sec. 3.3.1, Eq. 13 and Sec. 3.2.2, Eq. 6]
    "A low ASR indicates that the redundant subspace contributes minimally to the layer’s overall function. Adapting this layer is considered safe, as changes are confined to a low-impact part of the weight’s function."

    ASR is computed from the singular values produced by the KPD SVD (Eq. 4), and the 'redundant subspace' is defined as the components with the smallest r singular values (Eq. 6). The numerator of Eq. 13 is exactly the energy of that same redundant subspace. Hence 'low ASR implies the redundant subspace contributes minimally' is true by construction, and the further inference that adaptation is 'safe' restates the unproven assumption that small singular components of WΣ_in are low-impact for prior knowledge. The layer-selection metric therefore cannot independently validate that assumption; its empirical support comes only from the end-to-end comparison against manual and uniform selection, which tests the full method rather than isolating the metric's meaning.

full rationale

The paper's central claim is empirical: CKPD-FSCIL outperforms state-of-the-art methods on multiple FSCIL benchmarks, with ablations comparing KPD against LoRA, PiSSA/SVD, ASVD, and full fine-tuning, and comparing ASR-based layer selection against manual and uniform strategies. These results are self-contained and do not reduce to a fitted law. The principal circularity concern is confined to the interpretation of the ASR: the metric measures the relative singular-value energy of the subspace that the method itself labels 'redundant', so calling a low ASR 'safe' is definitionally aligned with the decomposition rather than independently established. This does not invalidate the empirical comparisons, but it means the theoretical motivation for the layer-selection signal is partly a restatement of the method's own construction. Self-citations (Mamba-FSCIL for the projector/SSM setup, CorDA for covariance-guided decomposition) are used as component choices or related work, not as an external uniqueness theorem or as the sole justification for the central claim; accordingly they are not load-bearing in a circular sense.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No physical or architectural entities are introduced; KPD, CALS, and ASR are algorithmic constructs within the existing architecture. The central claim depends on the four domain assumptions listed above, which are not proven.

free parameters (3)
  • adapter rank r = 128 (miniImageNet, CUB-200), 256 (CIFAR-100)
    Manual hyperparameter per dataset (Sec. 4.1); controls capacity of the redundant subspace and influences the ASR denominator, so the layer selection outcome depends on it.
  • number of selected layers K = 6 (CIFAR-100, CUB-200), 9 (miniImageNet)
    Manual hyperparameter per dataset (Sec. 4.1); determines how many layers are adapted each session. Robustness is shown in Fig. 6 but the choice is not derived.
  • covariance regularization lambda = not specified (dynamic doubling)
    Added to Sigma_in to force invertibility (Sec. 3.2.2). The resulting inverse is sensitive to lambda; values are not reported.
assumptions (4)
  • domain assumption SVD of W Sigma_in separates knowledge-sensitive from redundant directions, with singular value magnitude as importance.
    Invoked in Sec. 3.2.1 and 3.2.2 (Eq. 4-6). The paper motivates it via Sigma_out = W Sigma_in W^T, but that identity does not imply the spectrum of W Sigma_in is the discriminative spectrum; the partition is a heuristic.
  • domain assumption A single random sample per class yields a reliable input covariance.
    Used in Eq. 2-3. Empirically tested in Sec. 4.4.3, but no guarantee for arbitrary classes; influences the entire decomposition.
  • domain assumption LayerNorm centers features so that the second moment matrix approximates a covariance.
    Stated in Sec. 3.2.2 before Eq. 3. LayerNorm centers across feature dimension per token, not across tokens, so E[x x^T] is not necessarily a centered covariance.
  • domain assumption The jitter-regularized inverse of Sigma_in is numerically benign.
    Used in Eq. 5 and 8 to map the decomposition back to the weight space. The inverse amplifies small eigenvalues, and the regularization threshold is not reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Continuous Knowledge-Preserving Decomposition with Adaptive Layer Selection for Few-Shot Class-Incremental Learning." pith.science (2026). https://pith.science/paper/XH7ON7ZU

@misc{pith2026250105017,
  author       = {Pith},
  title        = {Pith review of: Continuous Knowledge-Preserving Decomposition with Adaptive Layer Selection for Few-Shot Class-Incremental Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XH7ON7ZU}},
  note         = {Machine review of arXiv:2501.05017}
}
read the original abstract

Few-Shot Class-Incremental Learning (FSCIL) faces a critical challenge: balancing the retention of prior knowledge with the acquisition of new classes. Existing methods either freeze the backbone to prevent catastrophic forgetting, sacrificing plasticity, or add new modules, incurring high costs. These approaches treat pretrained models as black boxes, overlooking two key opportunities to exploit their internal capacity: reusing redundant representational space within layers and selectively adapting layers based on their sensitivity to forgetting. We propose CKPD-FSCIL, a unified framework that unlocks the underutilized capacity of pretrained weights, achieving a superior stability-plasticity balance with zero inference overhead. Our design integrates two continuously adapting mechanisms: At the weight level, a Continuous Knowledge-Preserving Decomposition mechanism uses feature covariance to split each weight matrix into a frozen subspace that safeguards prior knowledge and a learnable, redundant subspace for new tasks. At the layer level, a Continuous Adaptive Layer Selection mechanism leverages an Adapter Sensitivity Ratio to automatically select layers with the highest redundant capacity and lowest forgetting risk for adaptation. By targeting only safe, high-potential subspaces and layers, CKPD-FSCIL enables efficient adaptation. After each session, the learned adapters are merged back into the original weights, ensuring zero additional parameters or FLOPs during inference. Extensive experiments on multiple FSCIL benchmarks demonstrate that our method consistently outperforms state-of-the-art approaches in both adaptability and knowledge retention. The code is available at https://github.com/xiaojieli0903/CKPD-FSCIL.

Figures

Figures reproduced from arXiv: 2501.05017 by the authors.

Figure 1
Figure 1. Comparison of Parameter Adaptation Paradigms in FSCIL.(a) Full Fine-tuning: Updates all backbone weights (𝑊1, . . . , 𝑊𝑁 ) in every session, offering high plasticity but suffering from overfitting and catastrophic forgetting. (b) Full Freezing: Keeps all backbone weights fixed after the base session, preserving prior knowledge but limiting the capacity for adaptation. (c) Prompt/Adapter Tuning: Adds session-specific… view at source ↗
Figure 2
Figure 2. Overview of the CKPD-FSCIL framework: Our method integrates two core mechanisms for continual adaptation. (a) Continuous Knowledge-Preserving Decomposition performs weight-level adaptation on a selected layer 𝑊 (𝑙,𝑡) at layer 𝑙 during session 𝑡: (1) Input Covariance Calculation: Compute the input covariance Σ (𝑙,𝑡) in from a small covariance sample set 𝐷cov. (2) Covariance-Guided Decomposition: Apply SVD to 𝑊 (𝑙,𝑡)Σ… view at source ↗
Figure 4
Figure 4. Comparison of adaptive vs. manual layer selection on CIFAR-100. (a) Average accuracy on all previously seen novel classes (measuring knowledge retention). (b) Accuracy on the newly introduced classes in the current session (measuring plasticity). space. By dynamically updating the covariance statistics and subspace split, CKPD is expected to better preserve prior knowl￾edge, maintain fine-grained discrimination, and… view at source ↗
Figures from the paper (2 more)
Figure 3
Figure 3. Figure 3: Class-wise accuracy across incremental sessions for three CIFAR-100 novel classes introduced in session 1: (a) “plain,” (b) “plate,” and (c) “poppy”. The x-axis denotes the session index, and the y-axis shows the classification accuracy (%) for each class at the corres…
Figure 5
Figure 5. Figure 5: Layer selection frequency across sessions on three datasets. The x-axis lists all linear layers in the ViT backbone, including QKV projections (qkv), attention outputs (proj), and feed-forward layers (ffn0, ffn1), indexed by block (e.g., 0.proj refers to the attention …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

87 extracted references · 61 canonical work pages

  1. [1]

    Few-shot class-incremental learning,

    X. Tao, X. Hong, X. Chang, S. Dong, X. Wei, and Y. Gong, “Few-shot class-incremental learning,” in CVPR, 2020

    2020

  2. [2]

    Jarvis-1: Open-world multi-task agents with memory- augmented multimodal language models,

    Z. Wang, S. Cai, A. Liu, Y. Jin, J. Hou, B. Zhang, H. Lin, Z. He, Z. Zheng, Y. Yang et al. , “Jarvis-1: Open-world multi-task agents with memory- augmented multimodal language models,” TPAMI

  3. [3]

    Optimus-1: Hybrid multimodal memory empowered agents excel in long-horizon tasks,

    Z. Li, Y. Xie, R. Shao, G. Chen, D. Jiang, and L. Nie, “Optimus-1: Hybrid multimodal memory empowered agents excel in long-horizon tasks,” in NeurIPS, 2024

    2024

  4. [4]

    Lifelong learning of large language model based agents: A roadmap,

    J. Zheng, C. Shi, X. Cai, Q. Li, D. Zhang, C. Li, D. Yu, and Q. Ma, “Lifelong learning of large language model based agents: A roadmap,” arXiv:2501.07278, 2025

    arXiv 2025

  5. [5]

    Vision-language navigation with continual learning,

    Z. Li, Y. Lv, Z. Tu, D. Shang, and H. Qiao, “Vision-language navigation with continual learning,” arXiv:2409.02561, 2024

    arXiv 2024

  6. [6]

    McCloskey and N

    M. McCloskey and N. J. Cohen, Catastrophic interference in connectionist networks: The sequential learning problem , ser. Psychology of Learning and Motivation. Elsevier, 1989, vol. 24

    1989

  7. [7]

    An empirical investigation of catastrophic forgetting in gradient-based neural networks,

    I. J. Goodfellow, M. Mirza, D. Xiao, A. Courville, and Y. Bengio, “An empirical investigation of catastrophic forgetting in gradient-based neural networks,” arXiv:1312.6211, 2013

    arXiv 2013

  8. [8]

    Prototypical networks for few-shot learning,

    J. Snell, K. Swersky, and R. Zemel, “Prototypical networks for few-shot learning,” in NeurIPS, 2017

    2017

Show all 87 references
  1. [9]

    Learning to compare: Relation network for few-shot learning,

    F. Sung, Y. Yang, L. Zhang, T. Xiang, P. H. S. Torr, and T. M. Hospedales, “Learning to compare: Relation network for few-shot learning,” inCVPR, 2018

    2018

  2. [10]

    The stability-plasticity dilemma: Investigating the continuum from catastrophic forgetting to age- limited learning effects,

    M. Mermillod, A. Bugaiska, and P. Bonin, “The stability-plasticity dilemma: Investigating the continuum from catastrophic forgetting to age- limited learning effects,” 2013

    2013

  3. [11]

    icarl: Incremental classifier and representation learning,

    S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert, “icarl: Incremental classifier and representation learning,” in CVPR, 2017

    2017

  4. [12]

    Learning a unified classifier incrementally via rebalancing,

    S. Hou, X. Pan, C. C. Loy, Z. Wang, and D. Lin, “Learning a unified classifier incrementally via rebalancing,” in CVPR, 2019

    2019

  5. [13]

    Overcoming catastrophic forgetting in neural networks,

    J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska et al., “Overcoming catastrophic forgetting in neural networks,” National Academy of Sciences, 2017

    2017

  6. [14]

    Continual learning with deep generative replay,

    H. Shin, J. K. Lee, J. Kim, and J. Kim, “Continual learning with deep generative replay,” 2017

    2017

  7. [15]

    Few-shot incremental learning with continually evolved classifiers,

    C. Zhang, N. Song, G. Lin, Y. Zheng, P. Pan, and Y. Xu, “Few-shot incremental learning with continually evolved classifiers,” inCVPR, 2021

    2021

  8. [16]

    Neural collapse inspired feature-classifier alignment for few-shot class-incremental learn- ing,

    Y. Yang, H. Yuan, X. Li, Z. Lin, P. Torr, and D. Tao, “Neural collapse inspired feature-classifier alignment for few-shot class-incremental learn- ing,” in ICLR, 2023

    2023

  9. [17]

    Mamba- fscil: Dynamic adaptation with selective state space model for few-shot class-incremental learning,

    X. Li, Y. Yang, J. Wu, B. Ghanem, L. Nie, and M. Zhang, “Mamba- fscil: Dynamic adaptation with selective state space model for few-shot class-incremental learning,” arXiv:2407.06136, 2024

    2024 arXiv

  10. [18]

    Pre-trained vision and language transformers are few-shot incremental learners,

    K.-H. Park, K. Song, and G.-M. Park, “Pre-trained vision and language transformers are few-shot incremental learners,” inCVPR, 2024

    2024

  11. [19]

    Brain-inspired fast-and slow-update prompt tuning for few-shot class- incremental learning,

    H. Ran, X. Gao, L. Li, W. Li, S. Tian, G. Wang, H. Shi, and X. Ning, “Brain-inspired fast-and slow-update prompt tuning for few-shot class- incremental learning,” TNNLS, 2024

    2024

  12. [20]

    Calibrating higher- order statistics for few-shot class-incremental learning with pre-trained vision transformers,

    D. Goswami, B. Twardowski, and J. Van De Weijer, “Calibrating higher- order statistics for few-shot class-incremental learning with pre-trained vision transformers,” in CVPR, 2024

    2024

  13. [21]

    Multimodal parameter-efficient few-shot class incremental learning,

    M. D’ Alessandro, A. Alonso, E. Calabr ´es, and M. Galar, “Multimodal parameter-efficient few-shot class incremental learning,” inICCV, 2023

    2023

  14. [22]

    Knowledge adaptation network for few-shot class-incremental learning,

    Y. Wang, Y. Wang, G. Zhao, and X. Qian, “Knowledge adaptation network for few-shot class-incremental learning,”arXiv:2409.11770, 2024

    2024 arXiv

  15. [23]

    Few-shot class incremental learning with attention-aware self-adaptive prompt,

    C. Liu, Z. Wang, T. Xiong, R. Chen, Y. Wu, J. Guo, and H. Huang, “Few-shot class incremental learning with attention-aware self-adaptive prompt,” arXiv:2403.09857, 2024

    2024 arXiv

  16. [24]

    Pl-fscil: Har- nessing the power of prompts for few-shot class-incremental learning,

    S. Tian, L. Li, W. Li, H. Ran, L. Li, and X. Ning, “Pl-fscil: Har- nessing the power of prompts for few-shot class-incremental learning,” arXiv:2401.14807, 2024

    2024 arXiv

  17. [25]

    Learning without forgetting,

    Z. Li and D. Hoiem, “Learning without forgetting,” TPAMI, 2017

    2017

  18. [26]

    Few-shot class- incremental learning: A survey,

    J. Zhang, L. Liu, O. Silven, M. Pietik ¨ainen, and D. Hu, “Few-shot class- incremental learning: A survey,”arXiv:2308.06764, 2023

    2023 arXiv

  19. [27]

    A survey on few-shot class-incremental learning,

    S. Tian, L. Li, W. Li, H. Ran, X. Ning, and P. Tiwari, “A survey on few-shot class-incremental learning,” Neural Networks, 2024

    2024

  20. [28]

    Matching networks for one shot learning,

    O. Vinyals, C. Blundell, T. Lillicrap, and D. Wierstra, “Matching networks for one shot learning,” in NeurIPS, 2016

    2016

  21. [29]

    Optimization as a model for few-shot learning,

    S. Ravi and H. Larochelle, “Optimization as a model for few-shot learning,” in ICLR, 2017

    2017

  22. [30]

    Corda: Context-oriented decomposition adaptation of large language models,

    Y. Yang, X. Li, Z. Zhou, S. L. Song, J. Wu, L. Nie, and B. Ghanem, “Corda: Context-oriented decomposition adaptation of large language models,” arXiv:2406.05223, 2024

    2024 arXiv

  23. [31]

    Few-shot class-incremental learning via entropy-regularized data-free replay,

    H. Liu, L. Gu, Z. Chi, Y. Wang, Y. Yu, J. Chen, and J. Tang, “Few-shot class-incremental learning via entropy-regularized data-free replay,” in ECCV, 2022

    2022

  24. [32]

    Semantics- driven generative replay for few-shot class incremental learning,

    A. Agarwal, B. Banerjee, F. Cuzzolin, and S. Chaudhuri, “Semantics- driven generative replay for few-shot class incremental learning,” inACM MM, 2022

    2022

  25. [33]

    Few-shot class- incremental learning from an open-set perspective,

    C. Peng, K. Zhao, T. Wang, M. Li, and B. C. Lovell, “Few-shot class- incremental learning from an open-set perspective,” in ECCV, 2022

    2022

  26. [34]

    Xtarnet: Learning to extract task-adaptive representation for incremental few-shot learning,

    S. W. Yoon, D.-Y. Kim, J. Seo, and J. Moon, “Xtarnet: Learning to extract task-adaptive representation for incremental few-shot learning,” inICML, 2020

    2020

  27. [35]

    Metafscil: A meta- learning approach for few-shot class incremental learning,

    Z. Chi, L. Gu, H. Liu, Y. Wang, Y. Yu, and J. Tang, “Metafscil: A meta- learning approach for few-shot class incremental learning,” inCVPR, 2022

    2022

  28. [36]

    Few-shot class-incremental learning by sampling multi-phase tasks,

    D.-W. Zhou, H.-J. Ye, L. Ma, D. Xie, S. Pu, and D.-C. Zhan, “Few-shot class-incremental learning by sampling multi-phase tasks,”TPAMI, 2022

    2022

  29. [37]

    Synthesized feature based few-shot class- incremental learning on a mixture of subspaces,

    A. Cheraghian, S. Rahman, S. Ramasinghe, P. Fang, C. Simon, L. Pe- tersson, and M. Harandi, “Synthesized feature based few-shot class- incremental learning on a mixture of subspaces,” in ICCV, 2021

    2021

  30. [38]

    Forward compatible few-shot class-incremental learning,

    D.-W. Zhou, F.-Y. Wang, H.-J. Ye, L. Ma, S. Pu, and D.-C. Zhan, “Forward compatible few-shot class-incremental learning,” in CVPR, 2022

    2022

  31. [39]

    Topology-preserving class-incremental learning,

    X. Tao, X. Chang, X. Hong, X. Wei, and Y. Gong, “Topology-preserving class-incremental learning,” in ECCV, 2020

    2020

  32. [40]

    Energy-based latent aligner for incremental learning,

    K. Joseph, S. Khan, F. S. Khan, R. M. Anwer, and V. N. Balasubramanian, “Energy-based latent aligner for incremental learning,” in CVPR, 2022

    2022

  33. [41]

    Geometer: Graph few-shot class-incremental learning via prototype representation,

    B. Lu, X. Gan, L. Yang, W. Zhang, L. Fu, and X. Wang, “Geometer: Graph few-shot class-incremental learning via prototype representation,” inACM MM, 2022

    2022

  34. [42]

    Incremental few-shot learning via vector quantization in deep embedded space,

    K. Chen and C.-G. Lee, “Incremental few-shot learning via vector quantization in deep embedded space,” in ICLR, 2021

    2021

  35. [43]

    Subspace regularizers for few-shot class incremental learning,

    A. F. Aky¨ urek, E. Aky¨ urek, D. Wijaya, and J. Andreas, “Subspace regularizers for few-shot class incremental learning,” inICLR, 2022

    2022

  36. [44]

    Learning with fantasy: Semantic-aware virtual contrastive constraint for few-shot class- incremental learning,

    Z. Song, Y. Zhao, Y. Shi, P. Peng, L. Yuan, and Y. Tian, “Learning with fantasy: Semantic-aware virtual contrastive constraint for few-shot class- incremental learning,” in CVPR, 2023

    2023

  37. [45]

    Orco: Towards better gener- alization via orthogonality and contrast for few-shot class-incremental learning,

    N. Ahmed, A. Kukleva, and B. Schiele, “Orco: Towards better gener- alization via orthogonality and contrast for few-shot class-incremental learning,” in CVPR, 2024

    2024

  38. [46]

    Compositional few-shot class- incremental learning,

    Y. Zou, S. Zhang, Y. Li, R. Li et al. , “Compositional few-shot class- incremental learning,” in ICML, 2024

    2024

  39. [47]

    Delve into base-novel confu- sion: redundancy exploration for few-shot class-incremental learning,

    H. Zhou, Y. Zou, R. Li, Y. Li, and K. Xiao, “Delve into base-novel confu- sion: redundancy exploration for few-shot class-incremental learning,” in IJCAI, 2024

    2024

  40. [48]

    Closer: Towards better representation learning for few-shot class-incremental learning,

    J. Oh, S. Baik, and K. M. Lee, “Closer: Towards better representation learning for few-shot class-incremental learning,” in ECCV, 2024, pp. 18–35

    2024

  41. [49]

    Constrained few-shot class-incremental learning,

    M. Hersche, G. Karunaratne, G. Cherubini, L. Benini, A. Sebastian, and A. Rahimi, “Constrained few-shot class-incremental learning,” in CVPR, 2022

    2022

  42. [50]

    Neural collapse terminus: A unified solution for class incremental learning and its variants,

    Y. Yang, H. Yuan, X. Li, J. Wu, L. Zhang, Z. Lin, P. Torr, D. Tao, and B. Ghanem, “Neural collapse terminus: A unified solution for class incremental learning and its variants,” arXiv:2308.01746, 2023

    2023 arXiv

  43. [51]

    Self-promoted prototype refinement for few-shot class-incremental learning,

    K. Zhu, Y. Cao, W. Zhai, J. Cheng, and Z.-J. Zha, “Self-promoted prototype refinement for few-shot class-incremental learning,” inCVPR, 2021

    2021

  44. [52]

    Few- shot class-incremental learning via training-free prototype calibration,

    Q.-W. Wang, D.-W. Zhou, Y.-K. Zhang, D.-C. Zhan, and H.-J. Ye, “Few- shot class-incremental learning via training-free prototype calibration,” in NeurIPS. MIT Press, 2024

    2024

  45. [53]

    Few-shot class-incremental learning via relation knowledge distillation,

    S. Dong, X. Hong, X. Tao, X. Chang, X. Wei, and Y. Gong, “Few-shot class-incremental learning via relation knowledge distillation,” in AAAI, 2021

    2021

  46. [54]

    Semantic-aware knowledge distillation for few-shot class- incremental learning,

    A. Cheraghian, S. Rahman, P. Fang, S. K. Roy, L. Petersson, and M. Harandi, “Semantic-aware knowledge distillation for few-shot class- incremental learning,” in CVPR, 2021

    2021

  47. [55]

    Few-shot class-incremental learning via class-aware bilateral distillation,

    L. Zhao, J. Lu, Y. Xu, Z. Cheng, D. Guo, Y. Niu, and X. Fang, “Few-shot class-incremental learning via class-aware bilateral distillation,” inCVPR, 2023

    2023

  48. [56]

    The power of scale for parameter- efficient prompt tuning,

    B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter- efficient prompt tuning,” in EMNLP, 2021

    2021

  49. [57]

    Prefix-tuning: Optimizing continuous prompts for generation,

    X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” in IJCNLP, 2021

    2021

  50. [58]

    Visual prompt tuning,

    M. Jia, L. Tang, B.-C. Chen, C. Cardie, S. Belongie, B. Hariharan, and S.-N. Lim, “Visual prompt tuning,” in ECCV, 2022

    2022

  51. [59]

    Dualprompt: Complementary prompting for rehearsal-free continual learning,

    Z. Wang, Z. Zhang, S. Ebrahimi, R. Sun, H. Zhang, C.-Y. Lee, X. Ren, G. Su, V. Perot, J. Dyet al., “Dualprompt: Complementary prompting for rehearsal-free continual learning,” in ECCV, 2022

    2022

  52. [60]

    Parameter-efficient transfer learning for nlp,

    N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for nlp,” in ICML, 2019. 15

    2019

  53. [61]

    Towards a unified view of parameter-efficient transfer learning,

    J. He, C. Zhou, X. Ma, T. Berg-Kirkpatrick, and G. Neubig, “Towards a unified view of parameter-efficient transfer learning,” inICLR, 2022

    2022

  54. [62]

    Conditional adapters: Parameter-efficient transfer learning with fast inference,

    T. Lei, J. Bai, S. Brahma, J. Ainslie, K. Lee, Y. Zhou, N. Du, V. Y. Zhao, Y. Wu, B. Li et al. , “Conditional adapters: Parameter-efficient transfer learning with fast inference,” in NeurIPS, 2023

    2023

  55. [63]

    LoRA: Low-rank adaptation of large language models,

    E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” in ICLR, 2022

    2022

  56. [64]

    Pissa: Principal singular values and singular vectors adaptation of large language models,

    F. Meng, Z. Wang, and M. Zhang, “Pissa: Principal singular values and singular vectors adaptation of large language models,”arXiv:2404.02948, 2024

    2024 arXiv

  57. [65]

    Asvd: Activation-aware singular value decomposition for compressing large language models,

    Z. Yuan, Y. Shang, Y. Song, Q. Wu, Y. Yan, and G. Sun, “Asvd: Activation-aware singular value decomposition for compressing large language models,” arXiv:2312.05821, 2023

    2023 arXiv

  58. [66]

    Dora: Weight-decomposed low-rank adaptation,

    S.-Y. Liu, C.-Y. Wang, H. Yin, P. Molchanov, Y.-C. F. Wang, K.-T. Cheng, and M.-H. Chen, “Dora: Weight-decomposed low-rank adaptation,” in ICML, 2024

    2024

  59. [67]

    The use of multiple measurements in taxonomic problems,

    R. A. Fisher, “The use of multiple measurements in taxonomic problems,” Annals of eugenics, 1936

    1936

  60. [68]

    G. J. McLachlan, Discriminant analysis and statistical pattern recogni- tion. John Wiley & Sons, 2005

    2005

  61. [69]

    A law of data separation in deep learning,

    H. He and W. J. Su, “A law of data separation in deep learning,” National Academy of Sciences, 2023

    2023

  62. [70]

    Imagenet large scale visual recognition challenge,

    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein et al., “Imagenet large scale visual recognition challenge,” IJCV, 2015

    2015

  63. [71]

    The caltech- ucsd birds-200-2011 dataset,

    C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie, “The caltech- ucsd birds-200-2011 dataset,” California Institute of Technology, 2011

    2011

  64. [72]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky, “Learning multiple layers of features from tiny images,” Citeseer, 2009

    2009

  65. [73]

    mixup: Beyond empirical risk management,

    H. Zhang, M. Cisse, Y. Dauphin, and D. Lopez-Paz, “mixup: Beyond empirical risk management,” in ICLR, 2018

    2018

  66. [74]

    Improved regularization of convolutional neural networks with cutout,

    T. DeVries, “Improved regularization of convolutional neural networks with cutout,” arXiv:1708.04552, 2017

    2017 arXiv

  67. [75]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in ICLR, 2020

    2020

  68. [76]

    Mamba: Linear-time sequence modeling with selective state spaces,

    A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv:2312.00752, 2023

    2023 arXiv

  69. [77]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in ICML, 2021

    2021

  70. [78]

    Improved continually evolved classifiers for few-shot class-incremental learning,

    Y. Wang, G. Zhao, and X. Qian, “Improved continually evolved classifiers for few-shot class-incremental learning,”TCSVT, 2023

    2023

  71. [79]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in ICCV, 2021

    2021

  72. [80]

    Few-shot class incremental learning leveraging self- supervised features,

    T. Ahmad, A. R. Dhamija, S. Cruz, R. Rabinowitz, C. Li, M. Jafarzadeh, and T. E. Boult, “Few-shot class incremental learning leveraging self- supervised features,” in CVPR, 2022

    2022

  73. [81]

    Dynamic support network for few-shot class incremental learning,

    B. Yang, M. Lin, Y. Zhang, B. Liu, X. Liang, R. Ji, and Q. Ye, “Dynamic support network for few-shot class incremental learning,”TPAMI, 2022

    2022

  74. [82]

    Margin-based few-shot class- incremental learning with class-level overfitting mitigation,

    Y. Zou, S. Zhang, Y. Li, and R. Li, “Margin-based few-shot class- incremental learning with class-level overfitting mitigation,” in NeurIPS, 2022

    2022

  75. [83]

    Imagenet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” inCVPR, 2009

    2009

  76. [84]

    Coda-prompt: Continual decom- posed attention-based prompting for rehearsal-free continual learning,

    J. S. Smith, L. Karlinsky, V. Gutta, P. Cascante-Bonilla, D. Kim, A. Ar- belle, R. Panda, R. Feris, and Z. Kira, “Coda-prompt: Continual decom- posed attention-based prompting for rehearsal-free continual learning,” in CVPR, 2023

    2023

  77. [85]

    Expandable subspace ensemble for pre-trained model-based class-incremental learning,

    D.-W. Zhou, H.-L. Sun, H.-J. Ye, and D.-C. Zhan, “Expandable subspace ensemble for pre-trained model-based class-incremental learning,” in CVPR, 2024

    2024

  78. [86]

    Large scale incremental learning,

    Y. Wu, Y. Chen, L. Wang, Y. Ye, Z. Liu, Y. Guo, and Y. Fu, “Large scale incremental learning,” in CVPR, 2019

    2019

  79. [87]

    Generalized and incremental few-shot learning by explicit learning and calibration without forgetting,

    A. Kukleva, H. Kuehne, and B. Schiele, “Generalized and incremental few-shot learning by explicit learning and calibration without forgetting,” in ICCV, 2021

    2021

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.