{"id":"1f143d57-4b85-41be-845f-b3210f9e9800","arxiv_id":"2501.15998","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A distance-threshold rule over base-class prototypes, calibrated to a chosen forgetting rate, improves new-class accuracy in one-shot class-incremental learning while trading a controlled amount of base-class accuracy.","lead":"The paper adds a test-time rule to one-shot class-incremental learning: classify a query as a new class only if it is far from every old-class prototype, using a threshold set to cap old-class accuracy loss. This dial between new-class gain and base-class forgetting could matter for on-device personalization, and the paper introduces metrics for reporting that trade-off.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central 'a priori controllable forgetting' claim rests on calibrating alpha on the same base test split later used to report FOR; an independent held-out evaluation is needed to distinguish calibration from prediction.","rationale":"The reader's weakest assumption correctly identifies that alpha is selected on the same base test split used to report FOR. I agree this is the most load-bearing weakness: the method's advertised 'a priori' guarantee is a calibrated property of one particular test split, not a verified transferable property. The paper's conclusion that forgetting is 'always within the forgetting budget, regardless of the encountered novel samples' is logically true only relative to that fixed base distribution; the paper provides no evidence for robustness to base-query distribution shift or even for calibration on one split and evaluation on another. The negative NCR@2FOR rows in Table I are a separate empirical inconsistency, but they reinforce the need for a careful evaluation protocol. Since the core mechanism is simple and plausible, and the authors themselves acknowledge the method targets the ultra-low-shot regime, the appropriate verdict remains CONDITIONAL rather than REJECT. I therefore keep the reader's verdict unchanged, with the concrete test above as the condition for acceptance.","tokens_in":635,"tokens_out":2697,"duration_ms":73216,"concrete_test":"Before any alpha selection, randomly split X_test(0) into two halves, A and B (or use 5-fold cross-validation). For each of the 25 evaluation episodes, calibrate alpha on split A to meet FOR = 2% and FOR = 5%, then report mean and standard deviation of FOR on the held-out split B, together with NCR on X_query(1). If the held-out FOR exceeds the target by more than one percentage point (e.g., >3% for the 2% budget) or the NCR gains become insignificant, the a priori controllable-forgetting claim and the headline NCR numbers are overstated. Also report per-episode min/max FOR and NCR to quantify instability across the 25 sampled novel-class sets.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim is that NCD provides 'controllable forgetting' a priori: setting alpha on the base test split yields a chosen FOR before deployment, while simultaneously improving NCR. The load-bearing step is Section II-E, where alpha is selected on X_test(0) to hit a target FOR, and the same X_test(0) is then used to report FOR in Table I. This is calibration, not prediction: the reported FOR is the value used to choose alpha, so the 'a priori' guarantee is essentially an i.i.d. interpolation on the calibration split. In deployment, the distribution of min-distances to base prototypes can shift due to lighting, camera, device, or user environment; for that shifted distribution the same alpha may produce a FOR well above the advertised 2% or 5% budget. The paper does not evaluate this shift, nor does it provide confidence intervals over the 25 episodes. The selection bias is further visible in Table I: several NCR@2FOR entries are negative (e.g., MobileNetv2-SAVC -2.1, ResNet18-PN on CUB200 -2.2, DINOv2-PN -6.1), so the text's claim of 'consistent improvements' is not supported by the table. The mechanism is plausible, but the central guarantee and the reported NCR@xFOR numbers can only be endorsed after an honest held-out evaluation of both FOR and NCR.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Novel Class Detection (NCD) decision rule for one-shot class-incremental learning. At inference, a query is routed to the novel class branch if its distance to the nearest base-class prototype exceeds a threshold alpha; otherwise it is assigned to the closest base class. The authors argue that alpha can be chosen a priori to achieve a prescribed base-class forgetting rate (FOR) without using novel-class samples, and they introduce NCR@2FOR and NCR@5FOR metrics to report novel-class accuracy at 2% and 5% forgetting. Experiments on CUB200, CIFAR100, and CORe50 with MobileNetV2, ResNet18, and DINOv2 backbones are compared against vanilla nearest-prototype inference, with relative NCR gains up to 33.9% reported in Table I.","tokens_in":9706,"tokens_out":3983,"duration_ms":41310,"significance":"The manuscript addresses a practically important scenario: ultra-low-shot class-incremental learning on resource-constrained devices. The NCD rule is simple, plug-and-play, and agnostic to the base-training procedure, which are genuine strengths. If the controllable-forgetting claim were properly validated on data not used to select alpha, the method would provide a useful inference-time lever for trading base and novel accuracy. The paper also contributes new evaluation metrics and evaluates a broad set of backbones and base-training methods. However, the central guarantee currently rests on calibration on the same split used for evaluation, and the headline comparisons are not at matched forgetting rates; these issues must be resolved before the claims can be endorsed.","major_comments":[{"comment":"The load-bearing 'a priori controllable forgetting' claim is not supported because alpha is selected on the base test split X_test(0), and the same split is then used to report FOR and NCR. In Sec. III-B the authors state 'We find alpha values corresponding to those two levels of forgetting' but do not describe any held-out validation split; consequently the reported FOR values are calibration outcomes, not predictions. To establish the guarantee, alpha should be chosen on a validation subset of base data (or on a separate split) and FOR and NCR should be reported on an untouched test split, together with the distribution of achieved FOR values across this split.","section":"Sec. II-E and Sec. III-B/C"},{"comment":"The comparison against vanilla inference is not at matched forgetting rates. Vanilla nearest-prototype inference (Eq. 5) has negligible and uncontrolled FOR because the backbone is frozen, whereas NCR@2FOR and NCR@5FOR deliberately allow 2% or 5% base-class forgetting. The reported gains may therefore reflect the additional forgetting budget rather than a genuine advantage of the NCD rule. The paper should either report FOR for the vanilla baseline, compare NCR at the same FOR for both methods (e.g., by forcing the same trade-off in vanilla), or present full NCR-vs-FOR trade-off curves for both approaches.","section":"Table I and Sec. III-C"},{"comment":"No error bars or confidence intervals are reported for the 25-episode means, and several rows show negative gains (e.g., MobileNetv2-SAVC N1=1 NCR@2FOR -2.1, DINOv2s-PN N1=5 NCR@2FOR -6.1 and NCR@5FOR -0.7). The text's claim of 'consistent improvements' is therefore contradicted by the table. The authors should report standard errors or per-episode distributions and temper the consistency claim, or restrict it to the rows where the gains are statistically significant.","section":"Table I and Fig. 3"}],"minor_comments":[{"comment":"The abstract states 'up to a 30% improvement' while Table I reports relative gains up to +33.9 and absolute percentage-point gains up to +31.0; please clarify whether the reported gains are absolute or relative and specify the exact setting.","section":"Abstract"},{"comment":"The notation V-NCR is used in Table I but is not defined in the text; define it explicitly as the NCR of the vanilla inference rule of Eq. (5).","section":"Sec. III-B"},{"comment":"The phrase 'a-priori' and 'apriori' should be made consistent; also clarify that 'a priori' means before seeing novel support samples, not before seeing the base test set used for calibration.","section":"Sec. II-E"},{"comment":"The caption contains a grammatical error: 'OUR STRATEGIES ... CONSISTENTLY OUTPERFORMS' should be 'OUTPERFORM'.","section":"Table I caption"},{"comment":"There is a typo in 'RssNet18-PN' which should be 'ResNet18-PN'.","section":"Sec. III-C"},{"comment":"The metrics NCR@2FOR and NCR@5FOR are described only in words; please give formal definitions, including how alpha is chosen when the discrete test set does not allow an exact target FOR.","section":"Sec. III-B"}],"recommendation":"major_revision","confidential_remarks":"The core idea is simple and potentially useful, but the main 'controllable forgetting' claim currently reduces to calibration on the evaluation split. With a proper held-out alpha-selection procedure, matched-forgetting baselines, and confidence intervals, the paper could become acceptable. Without those changes, the reported NCR@xFOR numbers are not predictive of deployment behavior."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper has a genuinely useful idea. An inference-time distance threshold routes each query to either the base or the novel branch, letting you tune how much base accuracy you trade for novel accuracy in one-shot FSCIL. The writing is clear, the math is elementary, and the experiments cover multiple backbones and datasets. But the headline claim of 'controllable forgetting a priori' is overclaimed: alpha is selected on the base test split to hit a target FOR, and the same split is then used to report FOR in Table I. That is calibration, not prediction. The FOR numbers are the values you tuned for, not independent estimates. For deployment you would need a separate held-out set or an explicit distribution-shift analysis, and the paper provides neither.\n\nWhat is actually new: the NCD rule itself is the standard nearest-centroid distance threshold used in open-set recognition, and the paper does not cite that literature, so the novelty is moderate. But the specific combination with frozen-backbone one-shot FSCIL, plus the NCR@2FOR and NCR@5FOR metrics, is a fair contribution. Reporting NCR at controlled forgetting budgets is more honest than the usual unconstrained NCR, and it gives practitioners a useful knob.\n\nSoft spots, in order of importance: (1) the calibration/valuation leakage is real and affects the central guarantee. (2) No error bars are given over the 25 episodes, and several rows in Table I show negative gains (e.g., DINOv2s-PN with N1=5 is -6.1 at 2FOR), so the caption's claim of 'consistently outperforms' is not supported by the table. (3) The only comparison is vanilla nearest-prototype inference; other inference-time OOD or distance-threshold methods are not considered. (4) The ablation shows NCD hurts when shots are 3 or more, which the paper acknowledges, but it means the contribution is narrowly scoped to the one-shot regime.\n\nThe core mechanism is plausible, the implementation is reproducible from the description, and the metric framing has some value. I would send this to a serious referee, but with the expectation of required revisions: add a cleanly separated alpha-selection split, report confidence intervals, and soften the consistency language. It is worth a look if you work on FSCIL or on-device personalization.","headline":"A simple and clearly described inference-time threshold for one-shot FSCIL, but the 'a priori controllable forgetting' headline is calibration, not prediction, and the evaluation needs an independent holdout split and error bars.","tokens_in":10308,"tokens_out":2374,"would_cite":true,"duration_ms":23956,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a single distance threshold can fix the base-class forgetting rate in advance and simultaneously lift novel-class accuracy in one-shot class-incremental learning.","keywords":["few-shot class-incremental learning","one-shot learning","catastrophic forgetting","controllable forgetting","novel class detection","prototype-based inference","out-of-distribution detection","on-device continual learning"],"falsifier":"Pick $\\alpha$ on the base test split to achieve a 2% forgetting rate, then measure the forgetting rate on a separate set of base-class images with a mild domain shift (different lighting, resolution, or background) and on novel-class queries; if the measured forgetting rate exceeds 2% by more than run-to-run noise, the claim of a-priori controllable forgetting fails.","tokens_in":9241,"feed_emoji":"🎯","tokens_out":11237,"duration_ms":88363,"temperature":0.7,"pith_summary":"The paper targets one-shot class-incremental learning, in which a model trained on base classes must add new classes from a single labeled example each. It claims that catastrophic forgetting can be made controllable and that novel-class accuracy can rise at the same time by changing how predictions are made: a query image is routed to a novel class only when its feature vector is farther than a threshold $\\alpha$ from every base-class prototype. Because this routing decision ignores the novel prototypes, the model's accuracy on base test images can be computed before deployment, so the threshold can be chosen to meet a user-specified forgetting budget. On CUB200, CIFAR100, and CORe50, across MobileNetV2, ResNet18, and DINOv2 backbones with ProtoNet, SAVC, OrCo, and FACT base training, the rule reports consistent novel-class accuracy gains in the one-shot regime, with the largest gains when only one novel class is added.","feed_headline":"One threshold sets forgetting rate, lifts novel-class accuracy","feed_subtitle":"In one-shot class-incremental learning, send a query to a new class only when it is far from every old-class prototype.","key_machinery":"The load-bearing object is the boolean Novel Class Detection rule $DR_{\\mathrm{ncd}}(f^q; \\alpha, B_p) = \\mathbf{1}\\{ \\min_{c \\in B_p} \\mathrm{dist}(f^q, \\mathrm{proto}_c) > \\alpha \\}$, combined with the calibration procedure that selects $\\alpha$ on the base test split to match a target forgetting rate. The rule carries the argument because it decouples base-class predictions from the novel prototypes: the set of base queries that will be misclassified depends only on $\\alpha$ and the frozen backbone, so the achieved base-class accuracy, and therefore FOR, is known before any novel sample arrives. That is what turns unconstrained catastrophic forgetting into a pre-settable quality-of-service parameter.","core_discovery":"The central claim is that vanilla nearest-prototype inference is the wrong default for one-shot class-incremental learning, because the single novel support sample is a noisy estimate of the novel-class centroid. The proposed Novel Class Detection (NCD) rule gates inference on the stable base prototypes alone: if the minimum distance from the query feature to any base prototype exceeds a threshold $\\alpha$, the query is assigned to the nearest novel prototype; otherwise it is assigned to the nearest base prototype. Since the gate does not depend on novel samples, the forgetting rate $\\mathrm{FOR} = \\mathrm{BCR} - \\mathrm{ACC}(M^{\\alpha}_{\\mathrm{IT,ncd}}; X_{\\mathrm{test}}^{(0)})$ is computable a priori, and $\\alpha$ can be calibrated on the base test split to hit a preset budget such as 2% or 5%. Across 25 random episodes, NCD inference improves novel-class recognition over vanilla inference, for example by 33.9 percentage points for ResNet18-FACT on CUB200 with one novel class at 5% forgetting, and by 18.7 points for DINOv2s-PN on CORe50 at 2% forgetting; the paper also reports that the advantage shrinks or disappears once three or more shots per novel class are available.","pith_inferences":["If the distance statistics of deployment images drift from those of the base test split used to pick $\\alpha$, the achieved forgetting rate can exceed the preset budget; an on-device recalibration from a small stream of unlabeled base-class queries would test and correct this.","The $N_1=1$ results effectively measure how well base training separates base clusters from generic novel features, so NCD could be reused as a diagnostic to compare feature-space geometry across base-training losses.","A natural extension for open-world use is a second reject threshold so that queries that are far from both base and novel prototypes are not force-assigned to a novel class.","An adaptive schedule that shrinks $\\alpha$ as more shots accumulate could in principle combine the ultra-low-shot gains of NCD with the better multi-shot behavior of vanilla inference, but the paper does not propose one."],"forward_implications":["A device can be deployed with a chosen forgetting budget, such as a 2% drop in base-class accuracy, and the threshold is set once on base test data without storing old samples or retraining the backbone.","The NCD rule is plug-and-play on top of existing prototype-based FSCIL training methods and works across CNN and transformer backbones, so it can be added to already-deployed models.","Because the rule sends anything beyond the threshold to a novel class, it doubles as an out-of-distribution signal for base classes, which could trigger a prompt for user annotation.","The benefit is concentrated in ultra-low-shot settings: once three or more shots per novel class are available, vanilla inference is equal or better, so a practical system would switch between modes.","Gains are largest when only one novel class is added (pure base-versus-novel separation); with five novel classes the rule helps less because it does not improve discrimination among novel classes themselves."],"supporting_citations":[{"why":"Defines prototypical training and the prototype centroids that both vanilla inference and NCD use.","marker":"[16]"},{"why":"Provides the distance-based nearest-centroid classification rule (Eq. 5) that NCD replaces.","marker":"[24]"},{"why":"Sets up the FSCIL problem and the catastrophic-forgetting formulation the paper addresses.","marker":"[8]"},{"why":"SAVC is one of the state-of-the-art base-training methods NCD is applied to and compared against.","marker":"[13]"},{"why":"FACT is the base-training method behind the largest reported NCR gain.","marker":"[14]"},{"why":"OrCo supplies another base-training baseline whose orthogonality constraint shapes the feature space NCD exploits.","marker":"[17]"},{"why":"DINOv2 supplies the transformer backbone used for the CORe50 experiments.","marker":"[27]"}],"fun_headline_variants":["One alpha sets forgetting budget while boosting new classes","Distance gate to old classes lifts one-shot new-class accuracy","Novel Class Detection: control forgetting without hurting new classes","Threshold on base prototypes sets forgetting rate a priori","For one-shot learning, gate with base prototypes to trade-off"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The promise that forgetting stays within the chosen budget depends on the assumption that the images used to pick the distance threshold have the same distance-to-base-prototype statistics as the images the device will actually see after deployment.","fun_headline_variants_meta":{"raw":{"variants":["One alpha sets forgetting budget while boosting new classes","Distance gate to old classes lifts one-shot new-class accuracy","Novel Class Detection: control forgetting without hurting new classes","Threshold on base prototypes sets forgetting rate a priori","For one-shot learning, gate with base prototypes to trade-off"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000427,"raw_usage":{"total_tokens":2246,"prompt_tokens":1065,"completion_tokens":1181,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":681,"completion_tokens_details":{"reasoning_tokens":1103}},"tokens_in":681,"tokens_out":1181,"duration_ms":8722,"temperature":1.0,"reasoning_tokens":1103,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T13:49:41.109423+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Pick $\\alpha$ on the base test split to achieve a 2% forgetting rate, then measure the forgetting rate on a separate set of base-class images with a mild domain shift (different lighting, resolution, or background) and on novel-class queries; if the measured forgetting rate exceeds 2% by more than run-to-run noise, the claim of a-priori controllable forgetting fails.","supporting_citations":[{"cited_title":"Distance-based image classification: Generalizing to new classes at near-zero cost,","cited_arxiv_id":null,"evidence_quote":"Provides the distance-based nearest-centroid classification rule (Eq. 5) that NCD replaces."},{"cited_title":"Learning with fantasy: Semantic-aware virtual contrastive constraint for few-shot class-incremental learning,","cited_arxiv_id":null,"evidence_quote":"SAVC is one of the state-of-the-art base-training methods NCD is applied to and compared against."},{"cited_title":"For- ward compatible few-shot class-incremental learning,","cited_arxiv_id":null,"evidence_quote":"FACT is the base-training method behind the largest reported NCR gain."},{"cited_title":"Orco: Towards better gener- alization via orthogonality and contrast for few-shot class-incremental learning,","cited_arxiv_id":null,"evidence_quote":"OrCo supplies another base-training baseline whose orthogonality constraint shapes the feature space NCD exploits."},{"cited_title":"Dinov2: Learning robust visual features without supervision,","cited_arxiv_id":null,"evidence_quote":"DINOv2 supplies the transformer backbone used for the CORe50 experiments."}],"review_version":1}