{"id":"296ca2a5-6cbf-4669-af3b-ae3ff5060d8d","arxiv_id":"2504.18437","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"NCPTM-CIL applies a dynamic equiangular-tight-frame classifier and pull-and-push loss to pre-trained model-based class-incremental learning and reports state-of-the-art average accuracy on four benchmarks.","lead":"This paper proposes a class-incremental learning method that fine-tunes a pre-trained vision transformer once, then aligns class means to a dynamically expanding neural-collapse geometry during later tasks. The authors report accuracy gains over prior methods on four image benchmarks, but the effect is shown without code, error bars, or a consistent inference formula.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (5) compares raw test features f(x_i) directly with ETF weights, while Sec. 4.3 trains a per-task alignment layer only on class means; the paper never shows the two spaces match, so the reported gains may not be produced by the described mechanism.","rationale":"The reader's weakest assumption pinpoints exactly the load-bearing gap: training supervises aligned class means but inference compares raw features with ETF weights. My independent reading of Sec. 4.3 and Eq. (5) reaches the same conclusion, and the per-task reinitialization of the alignment layer makes the gap more consequential rather than less. I also noted the secondary issues named by the reader: the PAP gradient in Eq. (8) is not the exact derivative of Eq. (7), and the causal claim in Sec. 3.2 rests on correlation of NC2 with accuracy. Those are real but less decisive; if the inference mismatch is a typo and the alignment layer is applied at test time, the method could still be sound. Since the reader already issued a CONDITIONAL verdict and my concern is the same assumption, I recommend no change to that verdict. A single benchmark comparison of raw-feature inference versus aligned-feature inference would settle whether the central claim is supported.","tokens_in":16576,"tokens_out":10785,"duration_ms":109095,"concrete_test":"Run the described pipeline on the 10-task CIFAR-100 split and compare two inference rules: (a) Eq. (5) exactly as written, using raw f(x_i) dot w_hat_k; (b) the same rule after applying the per-task alignment layer to the test feature, A_t(f(x_i)) dot w_hat_k. If the two accuracies differ by more than a small tolerance (e.g., 0.5%), the stated inference does not implement the trained alignment, and the reported gains cannot be attributed to the described alignment mechanism. If they are nearly identical, report the learned A_t weights and the raw-versus-aligned feature statistics to show why the missing layer does not matter.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.3 trains the alignment layer by pulling aligned class means c_hat_k = A_t(h_k) toward the fixed ETF weights w_hat_k of Eq. (6), and it explicitly initializes a fresh A_t for every incremental task. Yet Eq. (5) computes test scores as s_y = f(x_i)^T w_hat_k / (||f(x_i)|| ||w_hat_k||), using the raw backbone feature f(x_i) and the ETF weights, with no application of A_t. The manuscript never states that raw features and ETF weights live in the same aligned space, and neither the PAP loss (Eq. 7) nor the dynamic ETF construction (Eq. 6) guarantees that they do. If inference really follows Eq. (5), the per-task alignment layers cannot influence test classification at all; the ablation moves in Tables 3 and 5 attributed to Init-Align and PAP loss, and the central NC-alignment story in Secs. 3.2 and 4.2, would be unexplained. Either Eq. (5) is a typo and actual inference applies A_t to test features, in which case the paper as written is not reproducible, or Eq. (5) is correct and the alignment mechanism is only a training-time device that cannot deliver the claimed mechanism at test time. Both readings leave the central claim unsupported as stated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes NCPTM-CIL, a class-incremental learning method for pre-trained models that uses neural collapse (NC) geometry as a guiding principle. The method fine-tunes a ViT on the base task, then freezes it and introduces a dynamic ETF classifier whose weight matrix is expanded as new classes arrive. An alignment layer maps stored class-mean features to the ETF, and a pull-and-push (PAP) loss is designed to enforce the ETF geometry. The paper claims state-of-the-art average accuracy on CIFAR-100, CUB-200, VTAB, and OmniBenchmark, and reports that NCPTM-CIL approaches joint-training upper bounds. It also argues, based on NC2 metric trajectories and correlations, that catastrophic forgetting in PTM-CIL essentially stems from attenuation of inter-class linear separability.","tokens_in":16788,"tokens_out":6805,"duration_ms":61800,"significance":"If the results hold, the paper makes a useful contribution: it brings the neural-collapse perspective to PTM-based CIL, proposes a conceptually simple dynamic ETF classifier with an alignment layer and PAP loss, and reports consistent gains over existing methods, including a large 6.73% improvement on VTAB. The ablation study and NC-metric analysis are a good start toward explaining why the method works. However, the central mechanism as described is not reproducible from the manuscript: the inference equation omits the alignment layer, the per-task re-initialization of that layer is not reconciled with the dynamic ETF, and the causality claims rest on correlations that are partly by construction. These issues currently prevent acceptance.","major_comments":[{"comment":"The inference rule in Eq. (5) computes cosine similarity between the raw test feature f(x_i) and the ETF weight vectors w_hat_k, with no application of the alignment layer A_t. However, Sec. 4.3 trains A_t only on class-mean features h_k to produce aligned class means c_hat_k = A_t(h_k) that match the ETF weights. The paper nowhere states that raw features and ETF weights reside in the same space, nor does it specify which alignment layer (if any) is applied at test time. Consequently, as written, the alignment layer cannot affect the predictions, and the reported accuracy gains plus the NC-alignment mechanism in Secs. 3.2 and 4.2 are unexplained. Please provide the corrected inference equation (e.g., applying A_t to f(x_i) or mapping both to a common space) and specify how alignment layers from different tasks are combined or selected at test time.","section":"Sec. 4.1, Eq. (5) vs Sec. 4.3"},{"comment":"The text states that a new set of alignment layer parameters is initialized for each task. With this design, the paper does not explain how previously learned classes remain aligned with the dynamic ETF after their alignment layer is discarded. The class-mean pool is updated, but no procedure is given for recomputing old class prototypes under the new alignment mapping. Without this detail, the method's ability to \"preserve the orthogonality and separability of previously learned classes\" is not demonstrated, and the experimental results are not reproducible.","section":"Sec. 4.3"},{"comment":"The push term in Eq. (7) uses 1/(k-1) where k is the class index, but the simplex ETF target for the cosine between distinct class vectors is -1/(K_t - 1), with K_t the current total number of classes. Since k ranges over classes, the target varies per class, which is inconsistent with the stated global optimum. Additionally, Eq. (8) is not the gradient of Eq. (7) as written: for unnormalized vectors, the derivative of (w^T c - 1)^2 with respect to c is 2(w^T c - 1)w, not (1 - cos angle)w. If the loss or feature normalization differs from what is written, the derivation must be provided. This affects the theoretical claim that PAP loss drives the class-mean matrix to a simplex ETF.","section":"Sec. 4.2, Eqs. (7)-(8)"},{"comment":"The conclusion that catastrophic forgetting \"essentially stems from the attenuation of inter-class linear separability capacity\" is supported only by correlational evidence (Fig. 1b, Fig. 5, Tab. 5). In Tab. 5, the NC2 values come from a method whose PAP loss explicitly optimizes the NC2 objective, so observing lower NC2 for NCPTM-CIL is partly by construction and is not independent evidence that NC causes the accuracy gain. A control experiment, such as aligning to a random orthonormal frame instead of the ETF, or reporting NC2 on the actual inference-time features after correcting Eq. (5), would provide a more direct test of the causal mechanism.","section":"Sec. 3.2 and Sec. 5.4"},{"comment":"All results in Tables 1-5 are reported as single numbers without standard deviations or number of seeds, and no code is provided. Given the method's free choices (Phase II epochs, PAP loss weight, alignment-layer architecture, fine-tuning ranks), the smaller reported gains (e.g., the 1.25% on CIFAR-100 in Fig. 4) could be within run-to-run variation. Reporting mean±std over at least three seeds and releasing code would substantiate the state-of-the-art claim.","section":"Sec. 5"}],"minor_comments":[{"comment":"The text says \"five benchmark datasets\" but only four are listed and evaluated (CIFAR-100, CUB-200, VTAB, OmniBenchmark).","section":"Sec. 5"},{"comment":"The 6.73%, 1.25%, and 2.5% improvements are quoted in the abstract and Sec. 5.2, but they correspond to Fig. 4 (ViT-B/16-IN1K, final-stage accuracy), whereas Table 1 reports average accuracy with ViT-B/16-IN21K. Please clarify which backbone and which metric each number refers to.","section":"Abstract and Sec. 5.2"},{"comment":"The training details state \"we use the SGD optimizer to train ALIGNMENT LAYER\" and later mention reducing epochs when overfitting is observed; specifying the alignment-layer architecture, learning rate, and the exact stopping rule would improve reproducibility.","section":"Sec. 5.1"},{"comment":"The column header \"N C1 CIFAR-100\" is misformatted and should read \"NC1\" with the dataset name separated clearly.","section":"Table 4"}],"recommendation":"major_revision","confidential_remarks":"The central concern is the mismatch between the described mechanism and the reported inference rule; if the authors can clarify that the alignment layer is applied at test time and specify how it is shared or per-task, the method may be salvageable. The paper would also be considerably strengthened by multiple seeds and code release. The present version, however, does not support its central explanatory claim as written."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is a reasonable attempt to bring neural collapse to PTM-based CIL: it introduces a dynamic ETF classifier, a per-task alignment layer, and a pull-and-push loss, and reports consistent gains on four benchmarks, getting within ~1% of joint learning. The NC2 diagnostic (lower NC2 correlates with higher accuracy across methods) and the NC1-based fine-tuning selection are nice empirical contributions.\n\nThat said, the central mechanism as written does not hold together. Sec. 4.3 trains the alignment layer on class means (with a fresh layer each task) so that aligned means match the ETF weights. But Eq. (5) at inference compares the raw feature f(x_i) directly against the ETF weights with no alignment layer applied. The paper never says raw features live in the same space as the aligned means, and the PAP loss only touches class means, not the backbone. So if the text is accurate, the alignment layers have no influence on test predictions. If the alignment layer is meant to be applied at test time, the fresh-per-task initialization raises an obvious task-identification problem, since CIL test samples arrive without task labels. Either way, the reported gains are not explained by the described procedure. I suspect this is a serious omission in the write-up rather than a broken method, but it is not a minor typo: the entire causal story about ETF alignment depends on it.\n\nOther issues are smaller. Eq. (7) uses k-1 in the push term where it likely means K_t-1. There are no error bars anywhere, no code, and the claim that forgetting 'essentially stems from' separability loss is correlation-only. The PAP loss is also close in form to the dot-regression loss in Yang et al. 2022, so the novelty is more application than principle.\n\nWho is this for? CIL researchers who care about geometry-based methods. If the inference pipeline is fixed and code is released, the results could be real and useful. As submitted, it shouldn't be accepted without a careful rewrite of Sec. 4 and the experiments. I'd still send it to peer review—the idea and the empirical trend deserve referee time—but the reviewers should be asked to check the consistency between training and inference, and the authors should provide code and standard deviations.\n\nRecommendation: engage with it, but treat the reported numbers as provisional until the pipeline is clarified.","headline":"Plausible NC-based CIL method with strong reported gains, but the inference equation omits the alignment layer the whole mechanism depends on—needs a careful rewrite before the results can be trusted.","tokens_in":17384,"tokens_out":4286,"would_cite":false,"duration_ms":41850,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Catastrophic forgetting in pre-trained-model class-incremental learning is a loss of inter-class separability, and maintaining a growing equiangular feature frame keeps old and new classes separable.","keywords":["class-incremental learning","neural collapse","pre-trained models","simplex equiangular tight frame","catastrophic forgetting","ETF alignment","continual learning"],"falsifier":"Run a trained NCPTM-CIL model and measure classification accuracy and NC2 score on held-out test features in two inference variants: raw features compared with ETF weights, and raw features first passed through the learned alignment layer then compared. If applying the alignment layer at test time changes accuracy substantially, or if raw test features have high NC2 while class means used in training are well aligned, then the learned geometry is not actually carried by the raw representation and the reported procedure does not implement its own claim.","tokens_in":16297,"feed_emoji":"🧠","tokens_out":5391,"duration_ms":48270,"temperature":0.7,"pith_summary":"This paper argues that catastrophic forgetting in class-incremental learning with pre-trained models is, at root, a geometric failure: as new classes arrive, the features of old classes lose their linear separability relative to one another. To test that diagnosis, the authors monitor the neural-collapse metric NC2 during incremental training and find it rises as accuracy falls, across several existing methods. They then design NCPTM-CIL, which keeps features close to a growing simplex equiangular tight frame by combining a dynamic ETF classifier, an alignment layer fitted to stored class means, and a pull-and-push loss that attracts each class mean to its own prototype and repels it from others. On CIFAR-100, CUB-200, VTAB, and OmniBenchmark, they report state-of-the-art average accuracy, and on CIFAR-100 and OmniBenchmark they come within 1.5% and 0.52% of joint-training accuracy respectively. If the diagnosis is right, neural-collapse geometry is a practical design principle for continual learning rather than just a terminal-phase curiosity.","feed_headline":"Forgetting in incremental learning is loss of class separability","feed_subtitle":"Aligning features to an equiangular frame keeps old classes separable and beats prior methods on four benchmarks.","key_machinery":"The load-bearing object is the simplex equiangular tight frame (ETF), a set of class weight vectors in which every pair has the same angle and the vectors are balanced around zero; in neural collapse, class means converge to such a frame and to the classifier weights, a property called self-duality. NCPTM-CIL makes this geometry dynamic: the dynamic ETF classifier builds the frame for the current number of classes and expands it when new classes arrive; the alignment layer, an MLP over stored class-mean features, maps drifting class means onto the frame; and the pull-and-push loss enforces unit cosine similarity to the correct prototype and negative cosine similarity to all others, whose global optimum is exactly the ETF condition. Together these components translate the claim that forgetting is loss of separability into a procedure that re-establishes the separability after every task.","core_discovery":"Central claim: catastrophic forgetting in PTM-based CIL essentially stems from the attenuation of inter-class linear separability capacity. The paper supports this with NC2 trajectories showing separability degrades across tasks, and with a correlation between methods' accuracy and NC2. Its proposed NCPTM-CIL maintains a dynamic ETF classifier: as classes grow, the classifier adds a new vertex to a simplex equiangular tight frame, an alignment layer maps stored class-mean features to that frame, and a pull-push loss aligns each class mean with its own weight vector while pushing it away from the others. With ViT-B/16-IN1K initialization, it reports gains over the runner-up of 6.73% on VTAB, 1.25% on CIFAR-100, and 2.5% on OmniBenchmark, and near-joint-learning final accuracy on CIFAR-100, CUB-200, and OmniBenchmark. The intended upshot: keeping the feature space in equiangular form throughout incremental training is what prevents old classes from being forgotten.","pith_inferences":["Beyond the paper: if the alignment layer were applied at inference time, the method would likely become more consistent with its own mechanism; the paper is ambiguous about whether raw features are already in ETF space. This is a testable change, not a claim the paper makes.","Beyond the paper: the class-mean-only memory footprint suggests NCPTM-CIL could be adapted to privacy-sensitive or memory-constrained continual learning, where raw exemplars are not stored; the paper does not discuss this.","Beyond the paper: the observed monotone relation between base-stage NC1 and final accuracy could be turned into a predictive probe for picking among fine-tuning strategies across other backbones and datasets, independent of the NCPTM-CIL pipeline.","Beyond the paper: dynamic ETF expansion may also be applicable to open-set or few-shot incremental settings where the number of classes is unknown and class prototypes are sparse; the paper does not test these."],"forward_implications":["If forgetting is loss of inter-class separability, then NC metrics, especially NC2, can serve as a monitoring signal for when a continual learner is about to forget, and as a cheap model-selection criterion using only the base-stage NC1 value.","Maintaining an ETF geometry that grows with the class count removes the need to know the final label space in advance, which fixes a practical limitation of fixed simplex-ETF classifiers.","NCPTM-CIL's near-joint accuracy on CIFAR-100, CUB-200, and OmniBenchmark suggests that rehearsal-free incremental learning can approach the joint-training upper bound when the feature geometry is stabilized.","The reported gains on out-of-distribution benchmarks such as VTAB and OmniBenchmark imply the benefit is largest when the pre-trained features are far from the target domain, since that is where drift is most severe."],"supporting_citations":[{"why":"Defines neural collapse and the NC1/NC2/NC3 metrics; supplies the ETF geometry the paper uses to diagnose forgetting.","marker":"[38]"},{"why":"Shows that under unconstrained features the global optimum of cross-entropy is a simplex equiangular tight frame; basis for the dynamic ETF classifier.","marker":"[75]"},{"why":"Provides the alignment-layer design the paper borrows for ETF alignment.","marker":"[17]"},{"why":"AdaptFormer fine-tuning method used in the base learning phase.","marker":"[5]"},{"why":"VPT Deep fine-tuning method used in the base learning phase.","marker":"[24]"},{"why":"Aper, the class-mean baseline and main comparison; also the source of training protocol details.","marker":"[70]"},{"why":"EASE, the strongest prior PTM-CIL baseline on several datasets, used as comparison.","marker":"[72]"},{"why":"L2P, prompt-based CIL baseline whose NC2 trajectory supports the separability-loss diagnosis.","marker":"[56]"}],"fun_headline_variants":["Neural collapse geometry prevents forgetting in incremental learning","Aligning features to equiangular frame boosts incremental learning","Catastrophic forgetting is just lost class separability","Neural collapse alignment keeps old classes from being forgotten","Equiangular frame alignment preserves class separability in incremental learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that aligning stored class-mean features to the ETF geometry also brings raw test features into that geometry, although the reported inference compares raw features directly with the ETF weight vectors rather than passing them through the alignment layer.","fun_headline_variants_meta":{"raw":{"variants":["Neural collapse geometry prevents forgetting in incremental learning","Aligning features to equiangular frame boosts incremental learning","Catastrophic forgetting is just lost class separability","Neural collapse alignment keeps old classes from being forgotten","Equiangular frame alignment preserves class separability in incremental learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000962,"raw_usage":{"total_tokens":4136,"prompt_tokens":1025,"completion_tokens":3111,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":641,"completion_tokens_details":{"reasoning_tokens":3033}},"tokens_in":641,"tokens_out":3111,"duration_ms":20740,"temperature":1.0,"reasoning_tokens":3033,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:16:35.106617+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a trained NCPTM-CIL model and measure classification accuracy and NC2 score on held-out test features in two inference variants: raw features compared with ETF weights, and raw features first passed through the learned alignment layer then compared. If applying the alignment layer at test time changes accuracy substantially, or if raw test features have high NC2 while class means used in training are well aligned, then the learned geometry is not actually carried by the raw representation and the reported procedure does not implement its own claim.","supporting_citations":[{"cited_title":"Prevalence of neural collapse during the terminal phase of deep learning training","cited_arxiv_id":null,"evidence_quote":"Defines neural collapse and the NC1/NC2/NC3 metrics; supplies the ETF geometry the paper uses to diagnose forgetting."},{"cited_title":"A geometric analysis of neu- ral collapse with unconstrained features","cited_arxiv_id":null,"evidence_quote":"Shows that under unconstrained features the global optimum of cross-entropy is a simplex equiangular tight frame; basis for the dynamic ETF classifier."},{"cited_title":"Con- strained few-shot class-incremental learning","cited_arxiv_id":null,"evidence_quote":"Provides the alignment-layer design the paper borrows for ETF alignment."},{"cited_title":"Adaptformer: Adapting vision transformers for scalable visual recogni- tion","cited_arxiv_id":null,"evidence_quote":"AdaptFormer fine-tuning method used in the base learning phase."},{"cited_title":"Vi- sual prompt tuning","cited_arxiv_id":null,"evidence_quote":"VPT Deep fine-tuning method used in the base learning phase."},{"cited_title":"Revisiting class-incremental learning with pre- trained models: Generalizability and adaptivity are all you need","cited_arxiv_id":null,"evidence_quote":"Aper, the class-mean baseline and main comparison; also the source of training protocol details."},{"cited_title":"Expandable subspace ensemble for pre-trained model- based class-incremental learning","cited_arxiv_id":null,"evidence_quote":"EASE, the strongest prior PTM-CIL baseline on several datasets, used as comparison."}],"review_version":1}