{"id":"77903f0f-1f44-41fd-ab31-5cedbddb6bfa","arxiv_id":"2508.21424","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"ICPL generates pseudo-labels by clustering embeddings with KMeans, keeps only confident ones, and uses them to train class-incremental models without human labels, losing about 10 points versus supervised CIL but beating class-iNCD baselines by more than 5.","lead":"This paper proposes ICPL, a way to keep training a classifier on new classes without human labels: it clusters image embeddings with KMeans, keeps only high-confidence pseudo-labels, and feeds them to standard class-incremental methods. The method stays within about 10 accuracy points of fully supervised incremental learning on CIFAR100 and ImageNet100, and beats existing class-discovery methods on long incremental streams.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The method's central 'unsupervised' claim depends on exact a-priori knowledge of the number of classes per incremental task (n); Algorithm 1 has no fallback when n is misspecified, so the headline comparison is conditional on an assumption not stated in the abstract.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern I see. The method is explicitly and necessarily built on KMeans with exactly n clusters, and no mechanism is proposed for estimating or relaxing n. The paper states this limitation in Section 3, but the abstract's broad wording ('enables incremental learning from unlabeled datasets') makes the assumption easy to miss. Consequently, the empirical comparisons in Tables 1 and 2 all operate in the favorable known-n regime. This does not make the results wrong, but it does mean the central claim is narrower than the title and abstract suggest. Other issues raised by the reader - missing error bars, per-method augmentation choices, and the lack of comparison to prior unsupervised IL methods - are real but secondary to this one, because they affect the strength of the evidence rather than the defined scope of the method. My concern reinforces the CONDITIONAL verdict rather than changing it: the paper should clearly scope the claim to known class counts and, ideally, test sensitivity to misspecified n or add an estimator for n.","tokens_in":17780,"tokens_out":15649,"duration_ms":183044,"concrete_test":"Run the CIFAR100 Base50 Inc10 configuration (W A + ICPL, Table 1) with KMeans cluster counts set to k_true, k_true+2, k_true-2, k_true/2, and 2*k_true, keeping alpha=0.85 and all other settings fixed. Report final and average accuracy for each. If accuracy at k_true+2 or k_true/2 drops by more than a few points, or if the pseudo-label generation collapses, then the reported results are contingent on exact class-count knowledge. As a secondary check, add a cluster-count estimator (e.g., silhouette score over candidate k values) and see whether the same final accuracy is achievable without assuming n.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The weakest load-bearing point is the a-priori class-count assumption stated in Section 3: 'n is assumed to be known in our unsupervised setting.' For every task k >= 2, Algorithm 1 requires the number of clusters to be exactly n. The distance matrix in line 2 is (|D_k| x n), the pseudo-labels in line 2 are argmin over those n cluster centers, and the confidence scores in lines 4-5 are normalized over the same n centers. If the true number of classes in an incremental batch is not supplied, the procedure is undefined: there are no pseudo-labels, no confidence threshold, and no training set. The paper is transparent about this assumption, but the abstract claims the method 'enables incremental learning from unlabeled datasets' without the class-count caveat. This is not an internal inconsistency, but it is load-bearing: the headline comparison to supervised CIL and to class-iNCD/GCD methods is established only in the favorable regime where every task's class count is given. Replacing human annotations is the claimed contribution, yet exact class-count knowledge is itself a form of supervision that many real-world incremental streams do not provide.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ICPL, an unsupervised class-incremental learning pipeline that replaces human annotations for incremental tasks with KMeans-based pseudo-labels. For each new task, embeddings are extracted with the current model, KMeans is applied with the (assumed known) number of classes, pseudo-labels are assigned by nearest cluster, and a confidence threshold derived from softmax-transformed cluster distances selects a subset of samples. These pseudo-labels are plugged into four standard CIL methods (Replay, iCaRL, WA, FOSTER). The paper also introduces a 'static encoding' evaluation protocol that uses a fixed Hungarian assignment between pseudo-classes and ground-truth classes. Experiments on CIFAR100, ImageNet100, and fine-grained datasets report that ICPL is within about 10 points of supervised CIL (sometimes 1.33 points) and outperforms adapted class-iNCD/GCD baselines by more than 5% final accuracy, while reducing training cost.","tokens_in":18121,"tokens_out":7360,"duration_ms":70980,"significance":"If the results are reproducible, ICPL would be a useful, simple baseline for unsupervised CIL, and the static-encoding protocol is a genuine improvement over raw cluster accuracy for measuring cross-task confusion. The paper is transparent about the known-class-count assumption in Section 3 and provides extensive ablations over the confidence threshold, pseudo-label recomputation step, clustering algorithm, data augmentation, and per-method modifications. However, the strong comparative claims (e.g., >5% over class-iNCD, 1.33% gap to supervised CIL) rest on single runs and on hyperparameters selected using the same benchmarks, so the significance is conditional on the concerns below being addressed.","major_comments":[{"comment":"The method is only defined when the exact number of classes in every new task is supplied: Section 3 states 'n is assumed to be known in our unsupervised setting', and Algorithm 1 requires the number of clusters to be exactly n. The abstract and contribution list say the method 'replaces human annotations' and 'enables incremental learning from unlabeled datasets' without this caveat. Exact class count is a form of supervision that is not available in many real streams. Please either relax the assumption (e.g., estimate the number of clusters or evaluate sensitivity to misspecified k) or state the assumption prominently in the title/abstract.","section":"Section 3, Algorithm 1"},{"comment":"There is a numerical inconsistency for the same configuration. Table 9 reports, for WA+ICPL with Base0 Inc20 and alpha=0.85, final/mean accuracy 37.78/53.52 and 73% skipped samples, while Table 1 and Table 7 (tau=10, same setting) report 36.78/53.22. This is not a formatting difference: the default-alpha setting in the main results disagrees with the ablation table. Please correct the inconsistency and rerun/verify the affected numbers.","section":"Table 9 vs Tables 1 and 7"},{"comment":"The choice to add MixUp and class-weight balancing only for Replay and iCaRL was made after observing performance on the same CIFAR100 configurations, as stated in Section 4.3 ('we experimentally found that these methods can improve performance') and shown in Tables 8 and 10. These choices change final accuracy by several points, so the reported comparison among ICPL variants is partly a result of model selection on the evaluation benchmarks. At minimum, disclose this selection procedure; better, validate the choice on an independent split or with a pre-registered protocol.","section":"Section 4.3, Appendix B"},{"comment":"All quantitative claims are based on a single run with no error bars or multiple seeds. The headline margins (e.g., >5% over class-iNCD, 1.33% gap to supervised) are comparable to or smaller than the seed-to-seed variability typically observed for CIFAR100 CIL. Please report at least 3 seeds with mean and standard deviation for the main tables, and provide a significance statement for the central comparative claims.","section":"Tables 1, 2, 4, 7"}],"minor_comments":[{"comment":"Notation is overloaded: Section 3 defines n as the number of training instances, while Base-m Inc-n and Section 4.2 use n as the number of classes per task, and Algorithm 1 does not list the number of clusters as an input. Please use separate symbols (e.g., N_inst and C) consistently.","section":"Section 3, Algorithm 1"},{"comment":"The arithmetic in the GFLOPs calculation is inconsistent. From the stated values, Eq. (2) gives 0.14*5000 + 0.16 = 700.16 GFLOPs, not 688.66; Eq. (4) does not evaluate to 360207 GFLOPs; and the later '11707 GFLOPs' figure is not derived. Please recompute and correct the equations and the reported percentage reduction.","section":"Appendix C, Equations (2)-(4)"},{"comment":"The claim that 'the default threshold of 0.85 provides the best overall performance' is not supported for several individual configurations: in Base0 Inc5, alpha=0.65 and 0.75 give higher final accuracy; in Base50 Inc10, alpha=0.95 gives higher final accuracy. State the aggregation criterion (e.g., mean average accuracy over settings) and report per-configuration variance.","section":"Table 9"},{"comment":"The sentence 'the differences are small... suggesting this issue may be optional' is contradicted by the following paragraph arguing that the cluster-accuracy protocol is dangerous. Rephrase to avoid ambiguity about whether the authors believe the evaluation protocol change is necessary.","section":"Section 6.2"},{"comment":"Table 1 uses 'Base0'/'B50' while Figure 4 uses 'Base10' in the ImageNet100 panel; also the method name 'W A' is written with a space. Please make labels consistent.","section":"Table 1 / Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable empirical baseline contribution, but the numerical inconsistency in Table 9, the single-run reporting, and the benchmark-driven per-method hyperparameter choices are load-bearing for the central claims. These issues are fixable with additional experiments and a revised abstract that states the class-count assumption. I see no grounds for rejection, but the manuscript needs a careful revision before it can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: the gem here is the static encoding evaluation protocol (Section 4.1), not ICPL itself. The method is KMeans on embeddings, confidence via softmax over inverse distance, threshold at 0.85, plug into Replay/iCaRL/WA/FOSTER. Nothing conceptually new, and the authors admit it. But the evaluation protocol fixes a real problem: cluster accuracy can hit 100% while confusing old and new classes. The static Hungarian encoding is a sensible, simple fix, and the paper shows it makes a small but real difference. That part deserves credit.\n\nWhat else works: the experiments are broad. Four CIL methods, CIFAR100, ImageNet100, fine-grained sets, ablations on clustering algorithm, threshold, step size, augmentation. The comparison to class-iNCD methods shows ICPL is much stronger in long-horizon incremental settings, which is credible. The paper also includes complexity numbers, though they cut both ways.\n\nThe soft spots, in order of severity. First, the 'unsupervised' label is doing more work than the method. For every incremental task, the number of classes n is assumed known; Algorithm 1 needs it for KMeans. The paper states this in Section 3, so it is not hidden, but the abstract claims 'enabling incremental learning from unlabeled datasets' without the caveat. Knowing n is a form of supervision. The comparison to supervised CIL is therefore conditional on that knowledge. Second, everything rests on one seed with no error bars, and the key hyperparameters (alpha, tau, MixUp for some methods) are selected on the same benchmarks. That is a meaningful risk, even if the ablations show the method is reasonably stable. Third, there is an internal inconsistency: Table 9 gives alpha=0.85, Base0 Inc20 as 37.78/53.52, while Tables 1 and 7 give 36.78/53.22 for the same configuration. Probably a typo, but it needs fixing. Fourth, the computational claim is softer than it looks: TFLOPs drop 24% with tau=10, but wall-clock training time drops by 18 seconds out of 2060. The KMeans overhead eats the savings.\n\nOverall: this is a solid, honest empirical paper that would be a useful baseline for unsupervised/class-iNCD. It deserves serious peer review, but only after the authors release code, report multiple seeds, and rewrite the abstract to state the class-count assumption. I would not desk-reject it.","headline":"A straightforward pseudo-label pipeline with a genuinely useful evaluation protocol, undermined by an over-broad 'unsupervised' claim and one-seed results.","tokens_in":18552,"tokens_out":2425,"would_cite":true,"duration_ms":25067,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Replacing human labels with confidence-filtered KMeans pseudo-labels lets standard class-incremental learning methods train on unlabeled datasets, losing on average about 10 points of final accuracy and beating adapted novel-category-discov","keywords":["unsupervised class-incremental learning","pseudo-labels","KMeans clustering","confidence selection","catastrophic forgetting","novel category discovery","CIFAR100","ImageNet100"],"falsifier":"Run the WA+ICPL pipeline on CIFAR100 Base50 Inc10 with the per-task cluster count set to n−1, n, n+1 (and to n/2) and compare final accuracy; if the method tolerates wrong cluster counts, its stated dependence on known n is not the load-bearing constraint. Alternatively, on a held-in incremental task, compare the ground-truth accuracy of pseudo-labels above the α=0.85 threshold against those below it; if high-confidence pseudo-labels are not systematically more accurate, the confidence selection is not doing the work claimed.","tokens_in":17707,"feed_emoji":"🏷️","tokens_out":6571,"duration_ms":55121,"temperature":0.7,"pith_summary":"Standard class-incremental learning assumes every new batch of classes arrives with human labels. This paper argues that the labels can be replaced by pseudo-labels generated from KMeans clustering of the model's own feature embeddings, with a confidence threshold discarding the least reliable clusters, and that this cost saving costs surprisingly little accuracy. On CIFAR100 and ImageNet100, plugging these pseudo-labels into four established CIL methods (Replay, iCaRL, WA, FOSTER) yields final accuracies on average about 10 points below the same methods with true labels, and in one configuration the drop is only 1.33 points. Against adapted class-iNCD and generalized category discovery baselines designed for the same unlabeled setting, ICPL is more than 5 points better in final accuracy on CIFAR100, especially when many incremental tasks are run. The paper also introduces a static encoding evaluation protocol that penalizes confusion between old and new classes, which cluster-accuracy metrics miss.","feed_headline":"Label-free incremental learning lands within ~10 points of supervised","feed_subtitle":"Confidence-filtered KMeans labels let standard class-incremental methods train on unlabeled data and beat novel-category baselines by 5+ poi","key_machinery":"KMeans clustering over the feature embeddings of the unlabeled new-class data, which assigns each image a pseudo-label from the cluster index; a confidence score for each pseudo-label computed as a softmax over the inverse of the KMeans distance matrix (so samples far from all cluster centers get low confidence), with a threshold α=0.85 selecting a subset for training; and a static encoding evaluation protocol in which the Hungarian assignment maps pseudo-classes to ground-truth classes once per task and is frozen, so later tasks cannot rewrite earlier class identities. The pseudo-labels are fed into standard CIL methods (Replay, iCaRL, WA, FOSTER) in place of human labels; periodic recomput","core_discovery":"ICPL's central claim is that the bottleneck for real-world incremental learning is not the learning algorithm but the annotation pipeline: given an unlabeled batch of a known number of new classes, KMeans on the embeddings of the previous model produces clusters that can stand in for ground-truth labels, and a softmax over inverse KMeans distances gives a confidence score that lets the method keep only the most reliable pseudo-labels (threshold α=0.85, recomputed every 10 epochs). Combining these pseudo-labels with off-the-shelf CIL methods—substituting ŷ for y—keeps the model within roughly 10 points of supervised accuracy on average, outperforms class-iNCD methods adapted to long task sequ","pith_inferences":["ICPL could serve as a probe decomposing a CIL method's failure into forgetting versus labeling noise: run the method supervised and unsupervised on the same stream; the gap isolates the effect of imperfect pseudo-labels, and the residual drop after correcting labels isolates forgetting.","Because the known-n assumption is the fragile part, an immediate extension is estimating k per task; if k is over- or under-specified, the pseudo-label quality and thus the 10-point margin will likely degrade, so methods to estimate class counts could make it fully unsupervised.","The confidence selection's utility implies that embeddings from the previous task's model must separate new classes well; applying stronger representation learning (e.g., contrastive objectives on the unlabeled data before clustering) could push the unsupervised numbers above the 5-point margin or even close the gap to supervised.","The static encoding protocol is reusable beyond this paper; if adopted as a standard metric for unsupervised CIL, it would expose cross-task confusion in other methods that currently report cluster accuracy, likely changing published conclusions."],"forward_implications":["Any existing CIL method can be made label-free by swapping human labels for ICPL pseudo-labels, so the annotation cost of continual learning in new domains drops by the entire labeling budget of the incremental tasks.","The average final-accuracy gap to supervised CIL of about 10 points, shrinking to 1.33 points in the CIFAR100 Base50 Inc10 configuration with FOSTER, defines what label noise costs in practice; methods that improve embedding separability should narrow it further.","ICPL's more-than-5-point final-accuracy advantage over adapted FRoST, GM, DCCL, and CMS on CIFAR100 indicates that long task sequences are a regime class-iNCD methods currently do not handle well, while CIL-plus-pseudo-labels does.","Because low-confidence samples are dropped (about 30% at α=0.85), ICPL training uses roughly 24% fewer TFLOPs than supervised training, making unlabeled continual learning cheaper than labeled continual learning.","The static-encoding protocol, not cluster accuracy, is the right way to measure success in this setting; cluster accuracy masks cross-task confusion that a deployed model would exhibit."],"supporting_citations":[{"why":"Largest KMeans (Lloyd's algorithm) as the clustering step that generates pseudo-labels from feature embeddings; the core mechanism of ICPL.","marker":"[27]"},{"why":"iCaRL supplies one of the four CIL methods into which ICPL is plugged, contributing distillation loss and herding-based exemplar selection that ICPL inherits.","marker":"[35]"},{"why":"WA supplies the weight-rectification CIL method that ICPL is combined with for the main comparisons against class-iNCD baselines.","marker":"[55]"},{"why":"FOSTER supplies the model-centric CIL method showing the smallest gap to supervised accuracy (1.33 points on CIFAR100 Base50 Inc10).","marker":"[43]"},{"why":"Source of the Replay method and the Base-m/Inc-n incremental protocol and rehearsal setup used in all experiments.","marker":"[56]"},{"why":"Hungarian assignment is the matching step that defines the static encoding between pseudo-clusters and ground-truth classes in the evaluation protocol.","marker":"[23]"},{"why":"Together with [40], it motivates the softmax-on-inverse-distance confidence score used to filter pseudo-labels.","marker":"[18]"},{"why":"Provides the confidence-measure inspiration for selecting which pseudo-labels to trust before training.","marker":"[40]"},{"why":"FRoST is the class-iNCD baseline adapted and compared against; ICPL claims a >5-point final-accuracy improvement over it.","marker":"[38]"},{"why":"GM is the other class-iNCD baseline; ICPL outperforms it in all reported incremental settings.","marker":"[53]"}],"fun_headline_variants":["Pseudo-labels close gap to supervised incremental learning","Unsupervised incremental learning: KMeans labels beat iNCD by 5%","Confidence-filtered pseudo-labels for label-free class-incremental learning","ICPL: unsupervised incremental learning within 10% of supervised","Pseudo-label confidence trick boosts incremental learning without labels"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The method only works if the number of new classes in each incremental task is known in advance, since KMeans is given that number as its cluster count; if the true class count is unknown, the pseudo-label generation cannot be applied as described.","fun_headline_variants_meta":{"raw":{"variants":["Pseudo-labels close gap to supervised incremental learning","Unsupervised incremental learning: KMeans labels beat iNCD by 5%","Confidence-filtered pseudo-labels for label-free class-incremental learning","ICPL: unsupervised incremental learning within 10% of supervised","Pseudo-label confidence trick boosts incremental learning without labels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000178,"raw_usage":{"total_tokens":1142,"prompt_tokens":760,"completion_tokens":382,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":504,"completion_tokens_details":{"reasoning_tokens":305}},"tokens_in":504,"tokens_out":382,"duration_ms":3629,"temperature":1.0,"reasoning_tokens":305,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T14:19:19.001861+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the WA+ICPL pipeline on CIFAR100 Base50 Inc10 with the per-task cluster count set to n−1, n, n+1 (and to n/2) and compare final accuracy; if the method tolerates wrong cluster counts, its stated dependence on known n is not the load-bearing constraint. Alternatively, on a held-in incremental task, compare the ground-truth accuracy of pseudo-labels above the α=0.85 threshold against those below it; if high-confidence pseudo-labels are not systematically more accurate, the confidence selection is not doing the work claimed.","supporting_citations":[{"cited_title":"Least squares quantization in pcm","cited_arxiv_id":null,"evidence_quote":"Largest KMeans (Lloyd's algorithm) as the clustering step that generates pseudo-labels from feature embeddings; the core mechanism of ICPL."},{"cited_title":"icarl: Incremental classifier and representation learning","cited_arxiv_id":null,"evidence_quote":"iCaRL supplies one of the four CIL methods into which ICPL is plugged, contributing distillation loss and herding-based exemplar selection that ICPL inherits."},{"cited_title":"Maintaining discrimination and fairness in class incremental learning","cited_arxiv_id":null,"evidence_quote":"WA supplies the weight-rectification CIL method that ICPL is combined with for the main comparisons against class-iNCD baselines."},{"cited_title":"Foster: Feature boosting and compression for class- incremental learning","cited_arxiv_id":null,"evidence_quote":"FOSTER supplies the model-centric CIL method showing the smallest gap to supervised accuracy (1.33 points on CIFAR100 Base50 Inc10)."},{"cited_title":"When to Accept Automated Predictions and When to Defer to Human Judgment?","cited_arxiv_id":"2407.07821","evidence_quote":"Provides the confidence-measure inspiration for selecting which pseudo-labels to trust before training."},{"cited_title":"Class-incremental novel class discovery","cited_arxiv_id":null,"evidence_quote":"FRoST is the class-iNCD baseline adapted and compared against; ICPL claims a >5-point final-accuracy improvement over it."},{"cited_title":"Grow and merge: A unified framework for continu- ous categories discovery","cited_arxiv_id":null,"evidence_quote":"GM is the other class-iNCD baseline; ICPL outperforms it in all reported incremental settings."}],"review_version":1}