REVIEW 3 major objections 6 minor 31 references
Controllable Forgetting Mechanism for Few-Shot Class-Incremental Learning
T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read This paper claims that a single distance threshold can fix the base-class forgetting rate in advance and simultaneously lift novel-class accuracy in one-shot class-incremental learning.
desk verdict A simple and clearly described inference-time threshold for one-shot FSCIL, but the 'a priori controllable forgetting' headline is calibration, not prediction, and the evaluation needs an independent holdout split and error bars. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the boolean Novel Class Detection rule $DR_{\mathrm{ncd}}(f^q; \alpha, B_p) = \mathbf{1}\{ \min_{c \in B_p} \mathrm{dist}(f^q, \mathrm{proto}_c) > \alpha \}$, combined with the calibration procedure that selects $\alpha$ on the base test split to match a target forgetting rate. The rule carries the argument because it decouples base-class predictions from the novel prototypes: the set of base queries that will be misclassified depends only on $\alpha$ and the frozen backbone, so the achieved base-class accuracy, and therefore FOR, is known before any novel sample arrives. That is what turns unconstrained catastrophic forgetting into a pre-settable quality-of-service parameter.
What would settle it
Pick $\alpha$ on the base test split to achieve a 2% forgetting rate, then measure the forgetting rate on a separate set of base-class images with a mild domain shift (different lighting, resolution, or background) and on novel-class queries; if the measured forgetting rate exceeds 2% by more than run-to-run noise, the claim of a-priori controllable forgetting fails.
Extended reading notes
Core claim
The central claim is that vanilla nearest-prototype inference is the wrong default for one-shot class-incremental learning, because the single novel support sample is a noisy estimate of the novel-class centroid. The proposed Novel Class Detection (NCD) rule gates inference on the stable base prototypes alone: if the minimum distance from the query feature to any base prototype exceeds a threshold $\alpha$, the query is assigned to the nearest novel prototype; otherwise it is assigned to the nearest base prototype. Since the gate does not depend on novel samples, the forgetting rate $\mathrm{FOR} = \mathrm{BCR} - \mathrm{ACC}(M^{\alpha}_{\mathrm{IT,ncd}}; X_{\mathrm{test}}^{(0)})$ is computable a priori, and $\alpha$ can be calibrated on the base test split to hit a preset budget such as 2% or 5%. Across 25 random episodes, NCD inference improves novel-class recognition over vanilla inference, for example by 33.9 percentage points for ResNet18-FACT on CUB200 with one novel class at 5% forgetting, and by 18.7 points for DINOv2s-PN on CORe50 at 2% forgetting; the paper also reports that the advantage shrinks or disappears once three or more shots per novel class are available.
Load-bearing premise
The promise that forgetting stays within the chosen budget depends on the assumption that the images used to pick the distance threshold have the same distance-to-base-prototype statistics as the images the device will actually see after deployment.
Editorial extensions
If this is right
- A device can be deployed with a chosen forgetting budget, such as a 2% drop in base-class accuracy, and the threshold is set once on base test data without storing old samples or retraining the backbone.
- The NCD rule is plug-and-play on top of existing prototype-based FSCIL training methods and works across CNN and transformer backbones, so it can be added to already-deployed models.
- Because the rule sends anything beyond the threshold to a novel class, it doubles as an out-of-distribution signal for base classes, which could trigger a prompt for user annotation.
- The benefit is concentrated in ultra-low-shot settings: once three or more shots per novel class are available, vanilla inference is equal or better, so a practical system would switch between modes.
- Gains are largest when only one novel class is added (pure base-versus-novel separation); with five novel classes the rule helps less because it does not improve discrimination among novel classes themselves.
Reading between the lines
- If the distance statistics of deployment images drift from those of the base test split used to pick $\alpha$, the achieved forgetting rate can exceed the preset budget; an on-device recalibration from a small stream of unlabeled base-class queries would test and correct this.
- The $N_1=1$ results effectively measure how well base training separates base clusters from generic novel features, so NCD could be reused as a diagnostic to compare feature-space geometry across base-training losses.
- A natural extension for open-world use is a second reject threshold so that queries that are far from both base and novel prototypes are not force-assigned to a novel class.
- An adaptive schedule that shrinks $\alpha$ as more shots accumulate could in principle combine the ultra-low-shot gains of NCD with the better multi-shot behavior of vanilla inference, but the paper does not propose one.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a Novel Class Detection (NCD) decision rule for one-shot class-incremental learning. At inference, a query is routed to the novel class branch if its distance to the nearest base-class prototype exceeds a threshold alpha; otherwise it is assigned to the closest base class. The authors argue that alpha can be chosen a priori to achieve a prescribed base-class forgetting rate (FOR) without using novel-class samples, and they introduce NCR@2FOR and NCR@5FOR metrics to report novel-class accuracy at 2% and 5% forgetting. Experiments on CUB200, CIFAR100, and CORe50 with MobileNetV2, ResNet18, and DINOv2 backbones are compared against vanilla nearest-prototype inference, with relative NCR gains up to 33.9% reported in Table I.
Significance. The manuscript addresses a practically important scenario: ultra-low-shot class-incremental learning on resource-constrained devices. The NCD rule is simple, plug-and-play, and agnostic to the base-training procedure, which are genuine strengths. If the controllable-forgetting claim were properly validated on data not used to select alpha, the method would provide a useful inference-time lever for trading base and novel accuracy. The paper also contributes new evaluation metrics and evaluates a broad set of backbones and base-training methods. However, the central guarantee currently rests on calibration on the same split used for evaluation, and the headline comparisons are not at matched forgetting rates; these issues must be resolved before the claims can be endorsed.
major comments (3)
- [Sec. II-E and Sec. III-B/C] The load-bearing 'a priori controllable forgetting' claim is not supported because alpha is selected on the base test split X_test(0), and the same split is then used to report FOR and NCR. In Sec. III-B the authors state 'We find alpha values corresponding to those two levels of forgetting' but do not describe any held-out validation split; consequently the reported FOR values are calibration outcomes, not predictions. To establish the guarantee, alpha should be chosen on a validation subset of base data (or on a separate split) and FOR and NCR should be reported on an untouched test split, together with the distribution of achieved FOR values across this split.
- [Table I and Sec. III-C] The comparison against vanilla inference is not at matched forgetting rates. Vanilla nearest-prototype inference (Eq. 5) has negligible and uncontrolled FOR because the backbone is frozen, whereas NCR@2FOR and NCR@5FOR deliberately allow 2% or 5% base-class forgetting. The reported gains may therefore reflect the additional forgetting budget rather than a genuine advantage of the NCD rule. The paper should either report FOR for the vanilla baseline, compare NCR at the same FOR for both methods (e.g., by forcing the same trade-off in vanilla), or present full NCR-vs-FOR trade-off curves for both approaches.
- [Table I and Fig. 3] No error bars or confidence intervals are reported for the 25-episode means, and several rows show negative gains (e.g., MobileNetv2-SAVC N1=1 NCR@2FOR -2.1, DINOv2s-PN N1=5 NCR@2FOR -6.1 and NCR@5FOR -0.7). The text's claim of 'consistent improvements' is therefore contradicted by the table. The authors should report standard errors or per-episode distributions and temper the consistency claim, or restrict it to the rows where the gains are statistically significant.
minor comments (6)
- [Abstract] The abstract states 'up to a 30% improvement' while Table I reports relative gains up to +33.9 and absolute percentage-point gains up to +31.0; please clarify whether the reported gains are absolute or relative and specify the exact setting.
- [Sec. III-B] The notation V-NCR is used in Table I but is not defined in the text; define it explicitly as the NCR of the vanilla inference rule of Eq. (5).
- [Sec. II-E] The phrase 'a-priori' and 'apriori' should be made consistent; also clarify that 'a priori' means before seeing novel support samples, not before seeing the base test set used for calibration.
- [Table I caption] The caption contains a grammatical error: 'OUR STRATEGIES ... CONSISTENTLY OUTPERFORMS' should be 'OUTPERFORM'.
- [Sec. III-C] There is a typo in 'RssNet18-PN' which should be 'ResNet18-PN'.
- [Sec. III-B] The metrics NCR@2FOR and NCR@5FOR are described only in words; please give formal definitions, including how alpha is chosen when the discrete test set does not allow an exact target FOR.
Circularity Check
No significant circularity: alpha is a transparent calibration knob and NCR is measured on disjoint novel queries.
full rationale
The derivation chain is self-contained. The NCD decision rule (Eq. 6) is an explicitly proposed threshold heuristic, not a quantity derived from the evaluation metrics. Controllable forgetting follows directly from the rule's design: DR_ncd depends only on base prototypes, so the base accuracy ACC(M^alpha_IT,ncd; X_test(0)) is computable without novel samples, and alpha is chosen to realize a stated FOR budget (Sec II-E, Sec III-B). This is transparent calibration, not a hidden prediction; the reported NCR@2FOR and NCR@5FOR are measured on novel query splits disjoint from X_test(0), so the novel-class improvements are not fitted by the alpha selection. The self-citations (e.g., [19]-[21]) are background references and carry no load-bearing justification. The limitations are external-validity and reporting issues rather than circularity: alpha is calibrated on X_test(0) and the 'xFOR' in the reported metric is the same predefined target used for calibration, so the FOR side is a construction on that split and does not by itself establish robustness to distribution shift; and the caption's 'consistently outperforms' is contradicted by some negative NCR@2FOR entries (e.g., MobileNetv2-SAVC -2.1, DINOv2s-PN -6.1). These are correctness/evidence concerns, not circular derivation.
Assumptions & free parameters
free parameters (1)
- alpha (NCD distance threshold) =
Not reported; varies per backbone, dataset, and FOR target
assumptions (3)
- domain assumption The backbone is frozen during incremental training (Sec II-C), so base feature prototypes are stable and no base-class samples need to be retained beyond prototypes.
- domain assumption Base-test distances to nearest base prototype are representative of deployment distances (Sec II-E).
- domain assumption Novel classes can be treated as a single incremental session because support sets can be merged (Sec II-A).
Cite this review
Pith. "Pith review of Controllable Forgetting Mechanism for Few-Shot Class-Incremental Learning." pith.science (2026). https://pith.science/paper/JVPP4NKW
@misc{pith2026250115998,
author = {Pith},
title = {Pith review of: Controllable Forgetting Mechanism for Few-Shot Class-Incremental Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/JVPP4NKW}},
note = {Machine review of arXiv:2501.15998}
}
read the original abstract
Class-incremental learning in the context of limited personal labeled samples (few-shot) is critical for numerous real-world applications, such as smart home devices. A key challenge in these scenarios is balancing the trade-off between adapting to new, personalized classes and maintaining the performance of the model on the original, base classes. Fine-tuning the model on novel classes often leads to the phenomenon of catastrophic forgetting, where the accuracy of base classes declines unpredictably and significantly. In this paper, we propose a simple yet effective mechanism to address this challenge by controlling the trade-off between novel and base class accuracy. We specifically target the ultra-low-shot scenario, where only a single example is available per novel class. Our approach introduces a Novel Class Detection (NCD) rule, which adjusts the degree of forgetting a priori while simultaneously enhancing performance on novel classes. We demonstrate the versatility of our solution by applying it to state-of-the-art Few-Shot Class-Incremental Learning (FSCIL) methods, showing consistent improvements across different settings. To better quantify the trade-off between novel and base class performance, we introduce new metrics: NCR@2FOR and NCR@5FOR. Our approach achieves up to a 30% improvement in novel class accuracy on the CIFAR100 dataset (1-shot, 1 novel class) while maintaining a controlled base class forgetting rate of 2%.
Figures
Reference graph
Works this paper leans on
-
[1]
Deep learning in food category recognition,
Y . Zhang, L. Deng, H. Zhu, W. Wang, Z. Ren, Q. Zhou, S. Lu, S. Sun, Z. Zhu, J. M. Gorriz et al., “Deep learning in food category recognition,” Information Fusion, vol. 98, p. 101859, 2023
work page 2023
-
[2]
Large scale visual food recognition,
W. Min, Z. Wang, Y . Liu, M. Luo, L. Kang, X. Wei, X. Wei, and S. Jiang, “Large scale visual food recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 8, pp. 9932–9949, 2023
work page 2023
-
[3]
Online class-incremental learning for real-world food image classification,
S. Raghavan, J. He, and F. Zhu, “Online class-incremental learning for real-world food image classification,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2024, pp. 8195– 8204
work page 2024
-
[4]
A compact joint distillation network for visual food recognition,
Z. Heng, K.-H. Yap, and A. C. Kot, “A compact joint distillation network for visual food recognition,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2021, pp. 4105–4109
work page 2021
-
[5]
Long-Tailed Continual Learning For Visual Food Recognition
J. He, L. Lin, J. Ma, H. A. Eicher-Miller, and F. Zhu, “Long- tailed continual learning for visual food recognition,” arXiv preprint arXiv:2307.00183, 2023
work page Pith review arXiv 2023
-
[6]
Incremental Learning on Food Instance Segmentation
H.-T. Nguyen, Y . Cao, C.-W. Ngo, and W.-K. Chan, “Incremental learn- ing on food instance segmentation,” arXiv preprint arXiv:2306.15910 , 2023
work page Pith review arXiv 2023
-
[7]
Class-incremental exemplar compression for class-incremental learning,
Z. Luo, Y . Liu, B. Schiele, and Q. Sun, “Class-incremental exemplar compression for class-incremental learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 11 371–11 380
work page 2023
-
[8]
Few-shot class-incremental learning,
X. Tao, X. Hong, X. Chang, S. Dong, X. Wei, and Y . Gong, “Few-shot class-incremental learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 12 183–12 192
2020
Show all 31 references
-
[9]
A survey on few- shot class-incremental learning,
S. Tian, L. Li, W. Li, H. Ran, X. Ning, and P. Tiwari, “A survey on few- shot class-incremental learning,” Neural Networks , vol. 169, pp. 307– 324, 2024
2024
-
[10]
Few-shot in- cremental learning with continually evolved classifiers,
C. Zhang, N. Song, G. Lin, Y . Zheng, P. Pan, and Y . Xu, “Few-shot in- cremental learning with continually evolved classifiers,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 12 455–12 464
2021
-
[11]
Few-shot continual learning for audio classification,
Y . Wang, N. J. Bryan, M. Cartwright, J. P. Bello, and J. Salamon, “Few-shot continual learning for audio classification,” in ICASSP 2021- 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 321–325
2021
-
[12]
Semantic-aware knowledge distillation for few-shot class- incremental learning,
A. Cheraghian, S. Rahman, P. Fang, S. K. Roy, L. Petersson, and M. Harandi, “Semantic-aware knowledge distillation for few-shot class- incremental learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2021, pp. 2534–2543
2021
-
[13]
Learning with fantasy: Semantic-aware virtual contrastive constraint for few-shot class-incremental learning,
Z. Song, Y . Zhao, Y . Shi, P. Peng, L. Yuan, and Y . Tian, “Learning with fantasy: Semantic-aware virtual contrastive constraint for few-shot class-incremental learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2023, pp. 24 183–24 192
2023
-
[14]
For- ward compatible few-shot class-incremental learning,
D.-W. Zhou, F.-Y . Wang, H.-J. Ye, L. Ma, S. Pu, and D.-C. Zhan, “For- ward compatible few-shot class-incremental learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 9046–9056
2022
-
[15]
Overcoming catastrophic forgetting in incremental few-shot learning by finding flat minima,
G. Shi, J. Chen, W. Zhang, L.-M. Zhan, and X.-M. Wu, “Overcoming catastrophic forgetting in incremental few-shot learning by finding flat minima,” Advances in neural information processing systems , vol. 34, pp. 6747–6761, 2021
2021
-
[16]
Prototypical networks for few-shot learning,
J. Snell, K. Swersky, and R. Zemel, “Prototypical networks for few-shot learning,” Advances in neural information processing systems , vol. 30, 2017
2017
-
[17]
Orco: Towards better gener- alization via orthogonality and contrast for few-shot class-incremental learning,
N. Ahmed, A. Kukleva, and B. Schiele, “Orco: Towards better gener- alization via orthogonality and contrast for few-shot class-incremental learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 28 762–28 771
2024
-
[18]
Few- shot class-incremental learning via relation knowledge distillation,
S. Dong, X. Hong, X. Tao, X. Chang, X. Wei, and Y . Gong, “Few- shot class-incremental learning via relation knowledge distillation,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 35, no. 2, 2021, pp. 1255–1263
2021
-
[19]
Swiss dino: Efficient and versatile vision framework for on-device personal object search,
K. Paramonov, J.-X. Zhong, U. Michieli, J. Moon, and M. Ozay, “Swiss dino: Efficient and versatile vision framework for on-device personal object search,” IROS, 2024
2024
-
[20]
Object-conditioned bag of instances for few-shot personalized instance recognition,
U. Michieli, J. Moon, D. Kim, and M. Ozay, “Object-conditioned bag of instances for few-shot personalized instance recognition,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2024, pp. 7885–7889
2024
-
[21]
Online continual learning for robust indoor object recognition,
U. Michieli and M. Ozay, “Online continual learning for robust indoor object recognition,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2023, pp. 3849–3856
2023
-
[22]
Generalizable two-branch framework for image class-incremental learning,
C. Wu, X. Chang, and R. Wang, “Generalizable two-branch framework for image class-incremental learning,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 4265–4269
2024
-
[23]
Mgsvf: Multi- grained slow versus fast framework for few-shot class-incremental learn- ing,
H. Zhao, Y . Fu, M. Kang, Q. Tian, F. Wu, and X. Li, “Mgsvf: Multi- grained slow versus fast framework for few-shot class-incremental learn- ing,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 46, no. 3, pp. 1576–1588, 2021
2021
-
[24]
Distance-based image classification: Generalizing to new classes at near-zero cost,
T. Mensink, J. Verbeek, F. Perronnin, and G. Csurka, “Distance-based image classification: Generalizing to new classes at near-zero cost,” IEEE transactions on pattern analysis and machine intelligence , vol. 35, no. 11, pp. 2624–2637, 2013
2013
-
[25]
Mobilenetv2: Inverted residuals and linear bottlenecks,
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 4510–4520
2018
-
[26]
Deep residual learning for image recognition,
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778
2016
-
[27]
Dinov2: Learning robust visual features without supervision,
M. Oquab, T. Darcet, T. Moutakanni, H. V . V o, M. Szafraniec, V . Khali- dov, P. Fernandez, D. HAZIZA, F. Massa, A. El-Nouby et al., “Dinov2: Learning robust visual features without supervision,” Transactions on Machine Learning Research
-
[28]
ImageNet Large Scale Visual Recognition Challenge,
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei, “ImageNet Large Scale Visual Recognition Challenge,” International Journal of Computer Vision (IJCV) , vol. 115, no. 3, pp. 211–252, 2015
2015
-
[29]
The caltech-ucsd birds-200-2011 dataset,
C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie, “The caltech-ucsd birds-200-2011 dataset,” 2011
2011
-
[30]
Learning multiple layers of features from tiny images,
A. Krizhevsky, G. Hinton et al. , “Learning multiple layers of features from tiny images,” 2009
2009
-
[31]
Core50: a new dataset and benchmark for continuous object recognition,
V . Lomonaco and D. Maltoni, “Core50: a new dataset and benchmark for continuous object recognition,” in Conference on robot learning . PMLR, 2017, pp. 17–26
2017
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.