Pith. sign in

REVIEW 3 major objections 5 minor 68 references

On Distilling the Displacement Knowledge for Few-Shot Class-Incremental Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that matching displacement vectors between sample pairs, instead of similarity scores, preserves structural knowledge better during few-shot class-incremental learning, and that splitting distillation by class type…

desk verdict Genuinely new distillation loss with real empirical gains, but the KR metric inconsistency, missing code/error bars, and the unexplained session-2 drop need to be fixed before the headline numbers are fully trustworthy. read the letter →

arxiv 2412.11017 v2 pith:WMW3INXO submitted 2024-12-15 cs.LG cs.CV

Pengfei Fang , Yongchun Qin , Hui Xue This is my paper · ORCID
classification cs.LGcs.CV
keywords few-shotclass-incrementallearningknowledgedistillationdisplacementrelationalcatastrophicforgettingstructuralinformationdualnetwork
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Few-shot class-incremental learning (FSCIL) asks a model to keep recognizing previously learned classes while learning new classes from just a handful of examples, and knowledge distillation is the usual tool to avoid forgetting. Classic distillation matches the teacher's and student's outputs for each sample, while structured distillation matches similarity scores between sample pairs; this paper argues that both discard information that matters in the few-shot regime. The central claim is that matching the displacement vector between two samples—the pointwise difference of their features, treated as a probability distribution—preserves distance and direction information that a scalar similarity collapses, and that this is especially valuable for the few-shot novel classes whose features are poorly learned. Building on that, the paper proposes a dual network that distills base-class knowledge from output scores and novel-class knowledge from displacements, with a learned sample selector deciding how much to trust each branch at inference. The authors report that this design holds more knowledge across sessions than prior methods on CIFAR-100, miniImageNet, and CUB-200, and that the displacement loss tolerates outlier samples better than similarity-based relational distillation.

What carries the argument

The load-bearing object is the displacement vector $z_i - z_j$ between two samples' feature vectors, which the method normalizes into a probability distribution and matches between teacher and student with KL divergence. Unlike an inner-product similarity, which collapses a pair to a scalar, the displacement lives in the original $d$-dimensional feature space and keeps both the magnitude and the direction of the relation between samples. The second load-bearing mechanism is the dual-branch design: per-sample logit matching (IKD) is applied to the base classes because their features are already discriminative, while DKD is applied to the novel classes because their few-shot features are not, and an instance-aware sample selector trained with triplet and binary cross-entropy losses over momentum-updated base/novel prototypes weights the two branches at test time.

What would settle it

Run the CIFAR-100 FSCIL protocol with the teacher's novel-class features deliberately corrupted (for example, permute the features of a random 10% of novel samples before computing displacement targets): if knowledge retention does not drop, the robustness claim would be contradicted; alternatively, train the same DDNet with DKD applied only from session 3 onward and compare with DKD from session 2, and if the early-session accuracy drop at session 2 is not recovered, the advantage attributed to DKD during the first incremental step fails to appear.

Watch

Extended reading notes

Core claim

Structured knowledge distillation for FSCIL has been done with relational knowledge distillation (RKD), which measures a sample pair by their similarity (inner product or distance) and matches the teacher's and student's similarity matrices. This paper's discovery is that the right structural quantity to distill is the displacement between samples: for any pair of feature vectors $z_i, z_j$, the displacement $z_i - z_j$ is a $d$-dimensional structure that retains the full geometric relation, including direction and magnitude, whereas similarity reduces the relation to a single scalar. DKD turns each displacement into a probability distribution and minimizes the KL divergence between the teacher's and the student's displacement distributions, which multiplies the number of matched teacher-student pairs by $N-1$ and removes the coupling that makes an RKD gradient fully contaminated by any single outlier. Observing a performance gap between base and novel class features, the paper assigns per-sample logit matching (IKD) to the base classes and DKD to the novel classes within one network, DDNet, whose instance-aware sample selector fuses the two streams during inference. On three benchmarks the method raises knowledge retention to 69.39%, 72.91%, and 76.84%, and under an outlier-attack experiment the average accuracy drop falls from 0.986% to 0.526%.

Load-bearing premise

The teacher model trained on a few samples per novel class must produce displacement patterns that are reliable distillation targets; if the few-shot teacher's displacement vectors are noisy or uninformative, the DKD loss teaches the student to reproduce that noise, and the paper offers no bound or analysis on that teacher noise.

Editorial extensions

If this is right

  • Replacing similarity-based relational distillation with displacement distillation should improve any incremental-learning method that currently matches sample similarities; the paper shows gains when DKD replaces the standard loss inside LwF, iCaRL, and WA on general class-incremental learning.
  • The displacement formulation densifies supervision by giving each sample $N-1$ matched pairs instead of one, which should be especially useful when training data are scarce, without needing synthetic samples.
  • Splitting the distillation strategy by class domain—logit matching for well-trained base classes, displacement matching for few-shot novel classes—should extend to other base/novel splits beyond the three benchmarks tested.
  • Outside incremental learning, displacement matching should compete with relational KD as a general model-compression tool; on CIFAR-10 it beats RKD variants in four of five teacher-student configurations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the robustness argument generalizes beyond FSCIL: under a single outlier, an $N(N-1)$-term displacement loss has only $1/N$ of its terms contaminated, while a similarity matrix couples every term, so displacement-based losses should be preferred in any noisy or adversarial distillation setting.
  • Editorial inference: the transient ~10% accuracy drop at session 2, which the paper itself reports when DKD first activates, suggests the few-shot teacher's displacement distribution is noisy early on; a natural extension is to anneal the DKD weight or gate it on a teacher-confidence estimate during the first incremental sessions.
  • Editorial inference: the sample selector adds triplet loss, momentum prototypes, and binary cross-entropy yet improves knowledge retention by only about half a percentage point in ablations; a simpler fixed or distance-based branch weight may capture most of the benefit, which would be worth testing.
  • Editorial inference: applying DKD to normalized features (removing magnitude) would isolate whether the gains come from distance information or from direction information; if retention collapses under normalization, the distance component is doing the work.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Displacement Knowledge Distillation (DKD), a relational distillation objective that matches distributions of pairwise displacement vectors between teacher and student features, and combines it with individual logit distillation in a Dual Distillation Network (DDNet) for few-shot class-incremental learning. The authors argue that displacement preserves more structural information than similarity-based distillation, that it is more robust to outliers, and that a dual-branch design with a learned sample selector improves knowledge retention. The method is evaluated on CIFAR-100, miniImageNet, and CUB-200, with additional experiments on general class-incremental learning and standard knowledge distillation.

Significance. If the empirical claims hold, DKD is a simple and potentially general distillation objective: it is well-defined, it has a clean gradient interpretation, and the paper demonstrates consistent gains over IKD and RKD in FSCIL, CIL, and model-compression settings. The dual-branch design with an instance-aware selector is a reasonable response to the base/novel performance gap, and the robustness analysis is a useful addition. The main weaknesses are reporting inconsistencies in the central ablation table and an unresolved confound between the DKD objective and teacher quality in the few-shot setting; these need to be addressed before the state-of-the-art claim can be accepted.

major comments (3)
  1. [Section V-B / Table II] The KR column in Table II is not computed according to the definition given in Section V-B. KR is defined as Acc_tau / Acc_0 x 100%, but the first row of Table II reports KR = 62.34% while 51.61 / 76.97 = 67.05%; the reported value is instead the mean of the nine per-session accuracies (561.04 / 9 = 62.34). This discrepancy affects every ablation row and the text's quantitative claims about KR gains, so all ablation KR values need to be recomputed from the final-session accuracy and the conclusions rechecked.
  2. [Sections IV-B, IV-C and V-D] The theoretical analysis in Eqs. (23)-(24) counts how many gradient terms an outlier pollutes, but it does not model how errors in the teacher's displacement targets propagate to the student through the KL loss. The 0%-outlier row of Table III shows the current branch dropping from 70.11 at session 1 to 60.50 at session 2, which is exactly the first session where DKD operates on the pre-order dataset; the paper attributes this to a 'semantic gap,' but it is equally consistent with a noisy few-shot teacher. Without a control that matches teacher quality or ablates DKD at session 2, the final KR improvement cannot be cleanly attributed to DKD's structural target rather than to the dual-branch/selector architecture.
  3. [Section V-D / Table III] The outlier-robustness conclusion is based only on current-branch accuracy, while BiDist and DDNet differ in their objective functions and branch design. This makes the claimed reduction in average accuracy drop from 0.986% to 0.526% potentially confounded. The authors should report the outlier-induced accuracy drop for the full DDNet model and for a baseline that differs only in the distillation loss, so that the robustness gain can be attributed to DKD.
minor comments (5)
  1. [Eq. (6)] The subscripts and superscripts in Eq. (6) are inconsistent (z^tau_{f i} vs. z^{tau-1}_{f i}); define z^tau_{f,i} and z^{tau-1}_{f,i} explicitly as logits of the i-th and j-th samples.
  2. [Fig. 1 caption] The caption of Fig. 1 contains a duplicated older caption passage ('Figures a and b show two different structures...') that should be removed.
  3. [Section V-B] The sentence 'respectively leading FCIL (M=2), BiDist (M=1), and BiDist (M=5) by 0.94%, 2.32%, 2.64%' does not match the numbers in Table I; for example, the average KR difference between Ours (M=1) and FCIL is 0.94% if compared with the average KR, but FCIL has no M parameter in the table, and the other quoted differences also need to be verified against the recomputed averages.
  4. [Section IV-C] The phrase 'considering the opposite direction as the same pair' is confusing because Eq. (20) sums over ordered pairs and yields N(N-1) terms; please clarify whether the count refers to ordered or unordered pairs.
  5. [Section V-C] The statement that RKD and DKD increase Acc_n by averages of 1.33% and 4.81% is not directly verifiable from Table II, which reports only the session-8 Acc_n, not per-session averages; provide the per-session Acc_n curves or state explicitly how the averages were computed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found: DKD's reported gains are out-of-sample test measurements; the definition-derived properties of displacement are used as empirically validated hypotheses, and the paper contains no self-citations and no fit-renamed-as-prediction step.

full rationale

Walking the derivation chain, I find no circular step that merits a nonzero score. The central claim is that DKD (matching KL divergence over displacement vectors z_i − z_j, Eqs. 5–6 and 20) retains structural information better than similarity-based distillation. The theoretical remarks in Sec. IV — that the per-pair DKD measure lives in R^d while the inner-product RKD measure is scalar, that DKD yields N(N−1) pairs versus N, and that DKD's gradient terms are pair-local per Eq. (23) — are mathematical properties that follow from the definitions of the three losses in Sec. IV-A, but they are presented as hypotheses whose payoff is tested out-of-sample, not as conclusions that presuppose the intended result. The headline KR values (69.39%, 72.91%, 76.84% on CIFAR-100, miniImageNet, CUB-200; Table I) are test-set measurements after incremental training; the contribution of DKD over RKD is isolated in a matched ablation (Table II, e.g., KR 62.85% with DKD vs 62.46% with RKD in comparable rows) and in the external CIFAR-10 distillation benchmark (Table IV, DKD 87.98% vs RKD-A 87.68% average). No parameter is fitted to the reported test outcome and then renamed a prediction; the hyperparameters (w1, w2, β1, β2, α, γ) are standard model selection, and the reported accuracies are not reconstructions of the training objective. The reference list contains no works by the present authors (Fang, Qin, Xue), so no self-citation chain is load-bearing. The paper even reports adverse evidence against its own method — a ~10-point accuracy drop at session 2 precisely when DKD begins, attributed to a base-novel semantic gap (Sec. V-D), and a limited KR gain from the sample selector (Sec. V-C) — which indicates the empirical claims are not force-fitted to a favorable conclusion. Two non-circularity concerns remain as correctness risks, and I weigh them in the verdict without treating them as circular: the robustness derivation's count of outlier-polluted gradient terms (Sec. IV-C states 1/N, whereas the paper's own Eq. (23) contains 2(N−1) polluted terms out of 2N(N−1), i.e., 2/N, an arithmetic slip), and the reliability of few-shot teacher displacement targets is assumed without bias/variance bounds. Neither equates a prediction with an input, so the honest finding is no circularity.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard ML assumptions, a benchmark protocol, and several tuned hyperparameters. No new physical or model entities are postulated; the 'difference space D^d' of Section IV-B is a mathematical re-description, not an entity requiring independent evidence. The most fragile assumption is the reliability of the few-shot teacher's displacement signal, which the paper's own robustness data partially contradicts.

free parameters (6)
  • w1 (IKD loss weight) = 50 (CIFAR-100, miniImageNet); 30 (CUB-200)
    Tuned hyperparameter balancing base-class logit distillation; no sensitivity analysis is provided.
  • w2 (DKD loss weight) = 50 (CIFAR-100, miniImageNet); 50 (CUB-200)
    Tuned hyperparameter balancing displacement distillation; the paper sets w1 and w2 asymmetrically for CUB-200.
  • beta1 (sample selector triplet loss weight) = 0.2
    Tuned weight for the triplet loss in the sample selector.
  • beta2 (sample selector binary CE loss weight) = 0.8
    Tuned weight for the binary cross-entropy loss in the sample selector.
  • alpha (prototype momentum weight) = 0.9
    Momentum used for updating the base and novel prototypes in the sample selector.
  • gamma (triplet loss margin) = 1
    Margin used in the triplet loss for the sample selector.
assumptions (5)
  • standard math Softmax-based knowledge distillation via KL divergence is an effective transfer learning objective (Hinton et al. [5]).
    Section IV-A Eq (16)-(18) builds IKD, RKD, and DKD on this assumption.
  • domain assumption The FSCIL protocol of TOPIC [4] with 60 base classes and 5-way 5-shot novel sessions is a valid and meaningful benchmark.
    Section V-A adopts the split from [4], and all comparisons assume this setting is representative.
  • domain assumption The teacher network from session tau-1 produces displacement distributions that are reliable targets for distillation of novel classes.
    Section III-C Eq (5)-(6) and Section V-D implicitly rely on this; the paper itself reports a 10% accuracy drop at session 2, suggesting the teacher signal is fragile.
  • domain assumption Freezing all but the last residual block (following BiDist [7]) preserves base knowledge while allowing novel adaptation.
    Section V-A: 'Building upon insights gleaned from [7], [30], the last residual block of the ResNet remains trainable during class-incremental learning'.
  • domain assumption Logits of different sessions lie in a common space so that displacement vectors are comparable across teacher and student.
    Section IV-B assumes z_s and z_t are in the same R^d space; no alignment mechanism is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On Distilling the Displacement Knowledge for Few-Shot Class-Incremental Learning." pith.science (2026). https://pith.science/paper/WMW3INXO

@misc{pith2026241211017,
  author       = {Pith},
  title        = {Pith review of: On Distilling the Displacement Knowledge for Few-Shot Class-Incremental Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WMW3INXO}},
  note         = {Machine review of arXiv:2412.11017}
}
read the original abstract

Few-shot Class-Incremental Learning (FSCIL) addresses the challenges of evolving data distributions and the difficulty of data acquisition in real-world scenarios. To counteract the catastrophic forgetting typically encountered in FSCIL, knowledge distillation is employed as a way to maintain the knowledge from learned data distribution. Recognizing the limitations of generating discriminative feature representations in a few-shot context, our approach incorporates structural information between samples into knowledge distillation. This structural information serves as a remedy for the low quality of features. Diverging from traditional structured distillation methods that compute sample similarity, we introduce the Displacement Knowledge Distillation (DKD) method. DKD utilizes displacement rather than similarity between samples, incorporating both distance and angular information to significantly enhance the information density retained through knowledge distillation. Observing performance disparities in feature distribution between base and novel classes, we propose the Dual Distillation Network (DDNet). This network applies traditional knowledge distillation to base classes and DKD to novel classes, challenging the conventional integration of novel classes with base classes. Additionally, we implement an instance-aware sample selector during inference to dynamically adjust dual branch weights, thereby leveraging the complementary strengths of each approach. Extensive testing on three benchmarks demonstrates that DDNet achieves state-of-the-art results. Moreover, through rigorous experimentation and comparison, we establish the robustness and general applicability of our proposed DKD method.

Figures

Figures reproduced from arXiv: 2412.11017 by the authors.

Figure 1
Figure 1. (a) and (b) respectively show two different structures with [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (a) Performance differences between base and novel classes on [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The framework of the DDNet and the illustration of DKD. We employs IKD to preserve the base knowledge and the proposed DKD method to protect [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: The illustration of differences between (a) IKD, (b) RKD, and (c) DKD. IKD directly computes the KL-divergence of teacher and student’s output [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: The illustration of the gradient of DKD. The red part represents the [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: The Knowledge Retention rate of different methods on (a) CIFAR-100, (b) [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: The t-SNE visualization of ablation study on CIFAR-100. Classes 1-5 are base classes, and classes 6-9 are novel classes. In (a), only IKD is applied [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 9
Figure 9. Figure 9: Studies about the effect of DKD on CIL. For the three methods, [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

68 extracted references · 58 canonical work pages

  1. [1]

    iCaRL: Incremental classifier and representation learning,

    S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert, “iCaRL: Incremental classifier and representation learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2001–2010, 2017

    work page 2001

  2. [3]

    Learning a unified classifier incrementally via rebalancing,

    S. Hou, X. Pan, C. C. Loy, Z. Wang, and D. Lin, “Learning a unified classifier incrementally via rebalancing,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 831–839, 2019

    work page 2019

  3. [4]

    Few-shot class-incremental learning,

    X. Tao, X. Hong, X. Chang, S. Dong, X. Wei, and Y . Gong, “Few-shot class-incremental learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 12180–12189, 2020

    work page 2020

  4. [5]

    Distilling the knowledge in a neural network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531 , 2015

    arXiv 2015

  5. [6]

    Relational knowledge distilla- tion,

    W. Park, D. Kim, Y . Lu, and M. Cho, “Relational knowledge distilla- tion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3967–3976, 2019

    work page 2019

  6. [7]

    Few- shot class-incremental learning via class-aware bilateral distillation,

    L. Zhao, J. Lu, Y . Xu, Z. Cheng, D. Guo, Y . Niu, and X. Fang, “Few- shot class-incremental learning via class-aware bilateral distillation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11838–11847, 2023

    work page 2023

  7. [8]

    Few- shot class-incremental learning via relation knowledge distillation,

    S. Dong, X. Hong, X. Tao, X. Chang, X. Wei, and Y . Gong, “Few- shot class-incremental learning via relation knowledge distillation,” in Proceedings of the AAAI Conference on Artificial Intelligence, pp. 1255– 1263, 2021

    work page 2021

  8. [9]

    Optimization as a model for few-shot learning,

    S. Ravi and H. Larochelle, “Optimization as a model for few-shot learning,” in International Conference on Learning Representations , pp. 1–11, 2016

    work page 2016

Show all 68 references
  1. [10]

    Matching networks for one shot learning,

    O. Vinyals, C. Blundell, T. Lillicrap, D. Wierstra, et al. , “Matching networks for one shot learning,” in Proceedings of the Advances in Neural Information Processing Systems , vol. 29, pp. 3630–3638, 2016

    2016

  2. [11]

    Model-agnostic meta-learning for fast adaptation of deep networks,

    C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in Proceedings of the International Conference on Machine Learning , pp. 1126–1135, 2017

    2017

  3. [12]

    Prototypical networks for few- shot learning,

    J. Snell, K. Swersky, and R. Zemel, “Prototypical networks for few- shot learning,” in Proceedings of the Advances in Neural Information Processing Systems, vol. 30, pp. 4077–4087, 2017

    2017

  4. [13]

    Meta-learning with latent embedding optimization,

    A. A. Rusu, D. Rao, J. Sygnowski, O. Vinyals, R. Pascanu, S. Osindero, and R. Hadsell, “Meta-learning with latent embedding optimization,” arXiv preprint arXiv:1807.05960 , 2018

    2018 arXiv

  5. [14]

    Learning to compare: Relation network for few-shot learning,

    F. Sung, Y . Yang, L. Zhang, T. Xiang, P. H. Torr, and T. M. Hospedales, “Learning to compare: Relation network for few-shot learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1199–1208, 2018

    2018

  6. [15]

    Dynamic few-shot visual learning without forgetting,

    S. Gidaris and N. Komodakis, “Dynamic few-shot visual learning without forgetting,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 4367–4375, 2018

    2018

  7. [16]

    Cross attention network for few-shot classification,

    R. Hou, H. Chang, B. Ma, S. Shan, and X. Chen, “Cross attention network for few-shot classification,” in Proceedings of the Advances in Neural Information Processing Systems , vol. 32, pp. 4005–4016, 2019

    2019

  8. [17]

    Re- thinking few-shot image classification: A good embedding is all you need?,

    Y . Tian, Y . Wang, D. Krishnan, J. B. Tenenbaum, and P. Isola, “Re- thinking few-shot image classification: A good embedding is all you need?,” in Proceedings of the European Conference on Computer Vision, pp. 266–282, 2020

    2020

  9. [18]

    Few-shot learning via embedding adaptation with set-to-set functions,

    H.-J. Ye, H. Hu, D.-C. Zhan, and F. Sha, “Few-shot learning via embedding adaptation with set-to-set functions,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 8808–8817, 2020

    2020

  10. [19]

    Learning adaptive classifiers synthesis for generalized few-shot learning,

    H.-J. Ye, H. Hu, and D.-C. Zhan, “Learning adaptive classifiers synthesis for generalized few-shot learning,” International Journal of Computer Vision, vol. 129, no. 6, pp. 1930–1953, 2021

    1930

  11. [20]

    Hybrid graph neural networks for few-shot learning,

    T. Yu, S. He, Y .-Z. Song, and T. Xiang, “Hybrid graph neural networks for few-shot learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, pp. 3179–3187, 2022

    2022

  12. [21]

    Few-shot learning with noisy labels,

    K. J. Liang, S. B. Rangrej, V . Petrovic, and T. Hassner, “Few-shot learning with noisy labels,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 9089–9098, 2022

    2022

  13. [22]

    A closer look at few-shot classification,

    W.-Y . Chen, Y .-C. Liu, Z. Kira, Y .-C. F. Wang, and J.-B. Huang, “A closer look at few-shot classification,” in International Conference on Learning Representations, pp. 1–16, 2018

    2018

  14. [23]

    Self-promoted prototype refinement for few-shot class-incremental learning,

    K. Zhu, Y . Cao, W. Zhai, J. Cheng, and Z.-J. Zha, “Self-promoted prototype refinement for few-shot class-incremental learning,” in Pro- JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 12 ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,...

    2021

  15. [24]

    S3c: Self-supervised stochastic classifiers for few-shot class-incremental learning,

    J. Kalla and S. Biswas, “S3c: Self-supervised stochastic classifiers for few-shot class-incremental learning,” in Proceedings of the European Conference on Computer Vision , pp. 432–448, 2022

    2022

  16. [25]

    Mo- boo: Memory-boosted vision transformer for class-incremental learn- ing,

    B. Ni, X. Nie, C. Zhang, S. Xu, X. Zhang, G. Meng, and S. Xiang, “Mo- boo: Memory-boosted vision transformer for class-incremental learn- ing,” IEEE Transactions on Circuits and Systems for Video Technology , 2024

    2024

  17. [26]

    Few-shot class incremental learning leveraging self- supervised features,

    T. Ahmad, A. R. Dhamija, S. Cruz, R. Rabinowitz, C. Li, M. Jafarzadeh, and T. E. Boult, “Few-shot class incremental learning leveraging self- supervised features,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 3900–3910, 2022

    2022

  18. [27]

    Uncertainty-aware distillation for semi-supervised few-shot class-incremental learning.,

    Y . Cui, W. Deng, H. Chen, and L. Liu, “Uncertainty-aware distillation for semi-supervised few-shot class-incremental learning.,” IEEE Trans- actions on Neural Networks and Learning Systems , pp. 1–14, 2023

    2023

  19. [28]

    Lcsl: Long- tailed classification via self-labeling,

    D.-Q. Vu, T. T. Phung, J.-C. Wang, and S. T. Mai, “Lcsl: Long- tailed classification via self-labeling,” IEEE Transactions on Circuits and Systems for Video Technology , 2024

    2024

  20. [29]

    Analogical learning-based few-shot class-incremental learning,

    J. Li, S. Dong, Y . Gong, Y . He, and X. Wei, “Analogical learning-based few-shot class-incremental learning,” IEEE Transactions on Circuits and Systems for Video Technology, 2024

    2024

  21. [30]

    Memorizing complementation network for few-shot class-incremental learning,

    Z. Ji, Z. Hou, X. Liu, Y . Pang, and X. Li, “Memorizing complementation network for few-shot class-incremental learning,” IEEE Transactions on Image Processing, vol. 32, pp. 937–948, 2023

    2023

  22. [31]

    Reformulating classification as image-class matching for class incremental learning,

    Y . Hu, Z. Liang, X. Liu, Q. Hou, and M.-M. Cheng, “Reformulating classification as image-class matching for class incremental learning,” IEEE Transactions on Circuits and Systems for Video Technology, 2024

    2024

  23. [32]

    Multimodal parameter-efficient few-shot class incremental learning,

    M. D’Alessandro, A. Alonso, E. Calabr ´es, and M. Galar, “Multimodal parameter-efficient few-shot class incremental learning,” in Proceed- ings of the IEEE/CVF International Conference on Computer Vision , pp. 3393–3403, 2023

    2023

  24. [33]

    Representation robustness and feature expansion for exemplar-free class-incremental learning,

    Y . Luo, H. Ge, Y . Liu, and C. Wu, “Representation robustness and feature expansion for exemplar-free class-incremental learning,” IEEE Transactions on Circuits and Systems for Video Technology , 2023

    2023

  25. [34]

    Warping the space: Weight space rotation for class-incremental few-shot learning,

    D.-Y . Kim, D.-J. Han, J. Seo, and J. Moon, “Warping the space: Weight space rotation for class-incremental few-shot learning,” in International Conference on Learning Representations , pp. 1–19, 2022

    2022

  26. [35]

    Class incremental learning with less forgetting direction and equilibrium point,

    H. Wen, H. Qiu, L. Wang, H. Cheng, and H. Li, “Class incremental learning with less forgetting direction and equilibrium point,” IEEE Transactions on Circuits and Systems for Video Technology , 2024

    2024

  27. [36]

    Few-shot incre- mental learning with continually evolved classifiers,

    C. Zhang, N. Song, G. Lin, Y . Zheng, P. Pan, and Y . Xu, “Few-shot incre- mental learning with continually evolved classifiers,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12455–12464, 2021

    2021

  28. [37]

    Improved continually evolved clas- sifiers for few-shot class-incremental learning,

    Y . Wang, G. Zhao, and X. Qian, “Improved continually evolved clas- sifiers for few-shot class-incremental learning,” IEEE Transactions on Circuits and Systems for Video Technology , vol. 34, no. 2, 2023

    2023

  29. [38]

    For- ward compatible few-shot class-incremental learning,

    D.-W. Zhou, F.-Y . Wang, H.-J. Ye, L. Ma, S. Pu, and D.-C. Zhan, “For- ward compatible few-shot class-incremental learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9046–9056, 2022

    2022

  30. [39]

    Mics: Midpoint interpolation to learn compact and separated representations for few- shot class-incremental learning,

    S. Kim, Y . Jeong, J. S. Park, and S. W. Yoon, “Mics: Midpoint interpolation to learn compact and separated representations for few- shot class-incremental learning,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pp. 2236–2245, 2024

    2024

  31. [40]

    mixup: Beyond empirical risk minimization,

    H. Zhang, M. Cisse, Y . N. Dauphin, and D. Lopez-Paz, “mixup: Beyond empirical risk minimization,” arXiv preprint arXiv:1710.09412 , 2017

    2017 arXiv

  32. [41]

    Knowledge distillation: A survey,

    J. Gou, B. Yu, S. J. Maybank, and D. Tao, “Knowledge distillation: A survey,” International Journal of Computer Vision , vol. 129, no. 6, pp. 1789–1819, 2021

    2021

  33. [42]

    Do deep nets really need to be deep?,

    J. Ba and R. Caruana, “Do deep nets really need to be deep?,” in Proceedings of the Advances in Neural Information Processing Systems , vol. 27, pp. 2654–2662, 2014

    2014

  34. [43]

    Paraphrasing complex network: Network compression via factor transfer,

    J. Kim, S. Park, and N. Kwak, “Paraphrasing complex network: Network compression via factor transfer,” in Proceedings of the Advances in Neural Information Processing Systems , vol. 31, pp. 2765–2774, 2018

    2018

  35. [44]

    Improved knowledge distillation via teacher assis- tant,

    S. I. Mirzadeh, M. Farajtabar, A. Li, N. Levine, A. Matsukawa, and H. Ghasemzadeh, “Improved knowledge distillation via teacher assis- tant,” in Proceedings of the AAAI Conference on Artificial Intelligence , pp. 5191–5198, 2020

    2020

  36. [45]

    Like what you like: Knowledge distill via neuron selectivity transfer,

    Z. Huang and N. Wang, “Like what you like: Knowledge distill via neuron selectivity transfer,” arXiv preprint arXiv:1707.01219 , 2017

    2017 arXiv

  37. [46]

    Paying more attention to attention: improving the performance of convolutional neural networks via atten- tion transfer,

    N. Komodakis and S. Zagoruyko, “Paying more attention to attention: improving the performance of convolutional neural networks via atten- tion transfer,” in International Conference on Learning Representations, pp. 1–13, 2017

    2017

  38. [47]

    Varia- tional information distillation for knowledge transfer,

    S. Ahn, S. X. Hu, A. Damianou, N. D. Lawrence, and Z. Dai, “Varia- tional information distillation for knowledge transfer,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9163–9171, 2019

    2019

  39. [48]

    Knowledge transfer via distillation of activation boundaries formed by hidden neurons,

    B. Heo, M. Lee, S. Yun, and J. Y . Choi, “Knowledge transfer via distillation of activation boundaries formed by hidden neurons,” in Proceedings of the AAAI Conference on Artificial Intelligence, pp. 3779– 3787, 2019

    2019

  40. [49]

    A gift from knowledge distillation: Fast optimization, network minimization and transfer learning,

    J. Yim, D. Joo, J. Bae, and J. Kim, “A gift from knowledge distillation: Fast optimization, network minimization and transfer learning,” in Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4133–4141, 2017

    2017

  41. [50]

    Graph-based knowledge distillation by multi- head attention network,

    S. Lee and B. C. Song, “Graph-based knowledge distillation by multi- head attention network,” arXiv preprint arXiv:1907.02226 , 2019

    1907 arXiv

  42. [51]

    Knowledge distillation via instance relationship graph,

    Y . Liu, J. Cao, B. Li, C. Yuan, W. Hu, Y . Li, and Y . Duan, “Knowledge distillation via instance relationship graph,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 7096–7104, 2019

    2019

  43. [52]

    Similarity-preserving knowledge distillation,

    F. Tung and G. Mori, “Similarity-preserving knowledge distillation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1365–1374, 2019

    2019

  44. [53]

    Incremental few-shot learning via vector quantization in deep embedded space,

    K. Chen and C.-G. Lee, “Incremental few-shot learning via vector quantization in deep embedded space,” in International Conference on Learning Representations, pp. 1–16, 2020

    2020

  45. [54]

    End-to-end incremental learning,

    F. M. Castro, M. J. Mar ´ın-Jim´enez, N. Guil, C. Schmid, and K. Alahari, “End-to-end incremental learning,” in Proceedings of the European Conference on Computer Vision , pp. 233–248, 2018

    2018

  46. [55]

    Metafscil: A meta-learning approach for few-shot class incremental learning,

    Z. Chi, L. Gu, H. Liu, Y . Wang, Y . Yu, and J. Tang, “Metafscil: A meta-learning approach for few-shot class incremental learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14166–14175, 2022

    2022

  47. [56]

    Overcoming catastrophic forgetting in incremental few-shot learning by finding flat minima,

    G. Shi, J. Chen, W. Zhang, L.-M. Zhan, and X.-M. Wu, “Overcoming catastrophic forgetting in incremental few-shot learning by finding flat minima,” in Proceedings of the Advances in Neural Information Processing Systems, vol. 34, pp. 6747–6761, 2021

    2021

  48. [57]

    Few-shot continual infomax learning,

    Z. Gu, C. Xu, J. Yang, and Z. Cui, “Few-shot continual infomax learning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , pp. 19224–19233, 2023

    2023

  49. [58]

    Neural col- lapse inspired feature-classifier alignment for few-shot class-incremental learning,

    Y . Yang, H. Yuan, X. Li, Z. Lin, P. Torr, and D. Tao, “Neural col- lapse inspired feature-classifier alignment for few-shot class-incremental learning,” in International Conference on Learning Representations , pp. 1–13, 2022

    2022

  50. [59]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky, G. Hinton, et al., “Learning multiple layers of features from tiny images,” Toronto, ON, Canada, 2009

    2009

  51. [60]

    The caltech-ucsd birds-200-2011 dataset,

    C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie, “The caltech-ucsd birds-200-2011 dataset,” California Institute of Technology, 2011

    2011

  52. [61]

    Ima- genet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Ima- genet: A large-scale hierarchical image database,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 248–255, 2009

    2009

  53. [62]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 770–778, 2016

    2016

  54. [63]

    Learning without forgetting,

    Z. Li and D. Hoiem, “Learning without forgetting,” IEEE transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 12, pp. 2935– 2947, 2017

    2017

  55. [64]

    Maintaining dis- crimination and fairness in class incremental learning,

    B. Zhao, X. Xiao, G. Gan, B. Zhang, and S.-T. Xia, “Maintaining dis- crimination and fairness in class incremental learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13208–13217, 2020

    2020

  56. [65]

    Online continual learning in image classification: An empirical survey,

    Z. Mai, R. Li, J. Jeong, D. Quispe, H. Kim, and S. Sanner, “Online continual learning in image classification: An empirical survey,” Neuro- computing, vol. 469, pp. 28–51, 2022

    2022

  57. [66]

    Densely connected convolutional networks,

    G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 4700– 4708, 2017

    2017

  58. [67]

    Identity mappings in deep residual networks,

    K. He, X. Zhang, S. Ren, and J. Sun, “Identity mappings in deep residual networks,” arXiv preprint arXiv:1603.05027 , 2016

    2016 arXiv

  59. [68]

    Aggregated residual transformations for deep neural networks,

    S. Xie, R. Girshick, P. Doll ´ar, Z. Tu, and K. He, “Aggregated residual transformations for deep neural networks,” arXiv preprint arXiv:1611.05431, 2016

    2016 arXiv

  60. [69]

    Wide residual networks,

    S. Zagoruyko and N. Komodakis, “Wide residual networks,” in British Machine Vision Conference, pp. 1–15, 2016. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 13 Pengfei Fang is an Associate Professor at the School of Computer Science and Engineering, Southeast Unive...

    2016

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.