{"id":"5d54df44-18ae-4b98-9f13-82d781681bed","arxiv_id":"1908.02160","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"An iterative self-learning framework with multiple class prototypes corrects noisy labels and trains a ConvNet on both original and corrected labels, reporting strong accuracy on Clothing1M and Food101-N.","lead":"This paper presents a self-training method that corrects noisy image labels using multiple class prototypes discovered in the feature space of the same network being trained. The method reports state-of-the-art accuracy on two real-world noisy datasets without extra clean supervision.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'no assumption on noise distribution' claim is undermined by the density-cluster premise in Section 3.3; under structured label noise, prototype selection can pick wrong-labeled dense clusters and self-training may amplify errors.","rationale":"The reader's weakest_assumption isolates exactly the premise that correct labels form dense clusters and wrong labels are isolated. This is the most load-bearing concern because the entire label-correction phase depends on it: prototypes are selected by density in the feature space of a network trained on noisy labels, and corrected labels are assigned by average similarity to those prototypes. If the premise fails, the method loses its mechanism for distinguishing correct from incorrect labels, and the iterative self-training can propagate errors. The paper provides only a single illustrative density plot on one class interval in Clothing1M and does not test robustness to structured noise. A synthetic asymmetric-noise experiment is a direct, inexpensive check that would settle the concern. The reader's other points, about test-set hyperparameter selection and missing error bars, are real but are rigor issues that can be repaired without invalidating the approach; the density-cluster premise is more foundational because it determines whether the method works at all outside the two tested datasets. I therefore agree with the reader's assessment and would keep the verdict as CONDITIONAL rather than moving to accept or reject, pending this additional evidence.","tokens_in":12978,"tokens_out":5791,"duration_ms":67198,"concrete_test":"Construct a synthetic noisy benchmark, such as CIFAR-10, with two noise regimes: (1) symmetric random noise at 40% and (2) asymmetric structured noise that mimics visual confusions, e.g., flipping 40% of images from automobile to truck, cat to dog, and bird to airplane. Run the SMP pipeline in both regimes using a held-out validation set to select alpha and p, and report final test accuracy over at least three seeds. If SMP's gain over ordinary cross-entropy and a pseudo-label baseline shrinks or reverses in the structured-noise regime relative to the symmetric-noise regime, the density-cluster premise in Section 3.3 is violated and the 'no assumption on noise distribution' claim does not hold.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that SMP works on real noisy datasets without any assumption on the noise distribution. The method's actual load-bearing premise appears in Section 3.3: after training on noisy labels, correctly labeled images are dense in the deep feature space while incorrectly labeled images are isolated. This is stated directly: 'images with correct labels are more possible to have large rho value while those images with wrong labels appear in the region with low rho' (Figure 3, Section 3.3). That is an assumption about the geometry of the noise, and it is not guaranteed for real systematic confusions. When noise is structured and dense, for example a large visually similar subset of 'Shirt' images labeled as 'T-shirt', these wrong-labeled images form their own dense cluster. Density-based prototype selection (Eqs. 5-7) can then select wrong-labeled images as class prototypes for the noisy label, and Eq. 9 will propagate those errors into corrected labels. Because the corrected labels supervise the next training round via Eq. 4, the error can be amplified in subsequent iterations. The paper's empirical success on Clothing1M and Food101-N may reflect that these datasets' noise is not too densely structured in the relevant feature space, but that is an empirical fact about those datasets, not a consequence of the framework. A second, evaluative concern is that the key hyperparameters p and alpha are selected using test-set accuracy (Figures 4b and 4c), which inflates the reported comparisons; this is secondary but should be fixed with a validation split and repeated runs. The most foundational weakness remains the density-cluster premise: if it fails, the method's label-correction phase loses its justification.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes SMP (Self-Learning with Multi-Prototypes), an iterative framework for training convolutional networks on datasets with noisy labels. The method alternates between two phases: a training phase that minimizes a weighted cross-entropy loss combining the original noisy label and a corrected pseudo-label (Eq. 4), and a label-correction phase that selects multiple class prototypes per category using density and similarity measures computed from deep features of a sampled subset of each class (Eqs. 5-9). The authors claim the framework requires no assumption on the noise distribution and no extra clean supervision, and report state-of-the-art accuracy on Clothing1M and Food101-N, with ablations over the number of prototypes p, the weight factor alpha, the number of sampled images m, and the prototype-selection method.","tokens_in":13273,"tokens_out":3695,"duration_ms":39443,"significance":"If the claims hold, the work is practically significant because it offers a simple, single-network self-training recipe for learning from large-scale noisy web data without manual cleaning. The paper's strengths include experiments on two realistic noisy benchmarks, an extensive ablation suite, and the explicit verification that corrected labels improve from 61.74% to 77.36% on Clothing1M (Table 2). The central idea of using multiple prototypes per class rather than a single one is clearly motivated by Figure 1 and supported by the ablations in Figure 4b. However, the abstract's 'no assumption on the noise distribution' and 'without extra supervision' claims are stronger than what the method and experiments actually support, and the main hyperparameters are selected on the test set, which weakens the reported margins.","major_comments":[{"comment":"","section":"Abstract and Section 3.3"},{"comment":"","section":"Section 4.1, Table 1"},{"comment":"","section":"Section 4.2, Figure 4b-c"}],"minor_comments":[{"comment":"","section":"Section 3.3, Eq. (7)"},{"comment":"","section":"Section 3.3"},{"comment":"","section":"Table 2 and Section 4"},{"comment":"","section":"Abstract and Section 5"},{"comment":"","section":"Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable empirical contribution but its central claims are overstated relative to the evidence. The test-set-based hyperparameter selection in Figure 4 is a serious methodological issue that needs to be fixed before publication. I would also encourage the authors to compare with stronger recent baselines from 2018-2019, since the comparison set is mostly older transition-matrix and prototype methods; the reported margins are small and may not be robust across implementation details."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core of this paper is a genuinely practical self-training loop for web-scale noisy labels: learn a network on noisy labels, use its features to pick multiple prototypes per class via density peaks, correct labels by similarity to those prototypes, then retrain on a convex combination of original and corrected labels. That combination—multi-prototype, density-based selection, and retaining the original label—is not in CleanNet or Joint Optimization, and the ablations give it reasonable support. The empirical gains on Clothing1M and Food101-N look real, and the observation that a single prototype is insufficient for noisy classes is a useful one.\n\nThe soft spots are real but mostly addressable. The abstract says the method does not rely on any assumption about the noise distribution; the actual method leans on the density-cluster premise in Section 3.3—that correctly labeled images sit in dense regions while wrong labels are isolated. That is an assumption about label-noise geometry, and it can fail when mislabeled images form their own dense clusters (e.g., systematic confusion between visually similar classes). The paper even shows the expected case in Figure 3a. The benchmarks suggest this premise mostly holds for Clothing1M and Food101-N, but it is not assumption-free. Second, key hyperparameters p and alpha are selected using test-set accuracy in Figures 4b and 4c. That inflates the reported numbers relative to baselines that presumably did not get that treatment. A validation split and repeated runs with error bars would fix this, and the differences are not huge. The lack of released code also makes it harder to gauge the practical ease.\n\nThe circularity worry—the same network produces the corrected labels it is then trained on—is real, but the main claim is held-out test accuracy, so it does not invalidate the empirical result. It just means the method is heuristic, not principled. The paper is honest about this design; my concern is only the scope of the claim.\n\nWho gets value: anyone working on learning from noisy labels or practical training on web data. This is a credible baseline and a good candidate for comparison. It deserves serious peer review, not a desk reject. I would ask for a revised claim about assumptions, a validation-based hyperparameter selection, and repeated-run statistics. Even with those caveats, I would cite it and bring it to reading group.","headline":"Solid self-training approach for noisy labels with real empirical payoff; the 'no noise-distribution assumption' claim overreaches, but the method itself deserves a serious look.","tokens_in":13844,"tokens_out":1531,"would_cite":true,"duration_ms":18155,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes a self-learning framework that trains accurate image classifiers on real-world noisy datasets without any extra clean supervision, using multiple density-selected class prototypes to correct labels iteratively.","keywords":["noisy labels","self-learning","label correction","multi-prototype","prototype selection","image classification","Clothing1M","Food101-N"],"falsifier":"On a noisy dataset with systematic confusions between two visually similar classes, compute the paper's density ρ for each training image and measure how well a threshold on ρ separates correct from incorrect labels; if the separation is near chance, the prototype selector will pick mislabeled images and the corrected-label accuracy will not exceed the original label accuracy.","tokens_in":12755,"feed_emoji":"🏷️","tokens_out":5974,"duration_ms":55044,"temperature":0.7,"pith_summary":"Learning image classifiers from web-scraped labels fails when many labels are wrong. This paper tries to change that by showing a network can clean its own training set, provided labels are corrected with several representative images per class rather than one. The proposed framework, Self-Learning with Multi-Prototypes (SMP), alternates between training the network and relabeling images by comparing deep features against class prototypes. It claims to beat existing methods on the Clothing1M and Food101-N benchmarks using only the noisy labels, with no extra supervision. If true, noisy web-scale image collections become directly usable for training, without expensive manual cleaning.","feed_headline":"Multiple prototypes clean noisy image labels without extra supervision","feed_subtitle":"An iterative framework corrects image labels on Clothing1M and Food101-N using density-selected class prototypes.","key_machinery":"The central object is the multi-prototype label-correction mechanism. For each class c, m images sharing that noisy label are randomly sampled, their deep features G(x) are compared by cosine similarity (Eq. 5), and a density ρ_i counts how many neighbors each image has above a threshold S_c; a second measure η_i picks high-density samples that are not too close to each other, yielding p prototypes per class (Eqs. 6-7). The corrected label of an image is the class maximizing the average cosine similarity between its feature and the prototypes of that class (Eqs. 8-9). The training objective combines original label y and corrected label ŷ as L_total = (1−α)L(F(θ,x), y) + αL(F(θ,x), ŷ) (Eq. 4). This machinery carries the argument because the paper's performance gain rests on the corrected labels being accurate enough to guide training.","core_discovery":"The central claim is that a single prototype per class is insufficient to represent a noisy category, because samples in the same category form several clusters in deep feature space. SMP therefore selects for each class a small set of prototypes—images whose deep features have high local density and are spread apart—and corrects each training sample to the class whose prototypes are on average most similar to it. The network is then trained on a convex combination of the original noisy label and this corrected label, with weight α, and the two phases iterate until convergence. On Clothing1M and Food101-N the paper reports higher accuracy than previous methods, including ones that use additional clean labels, and ablation studies show the benefit of multiple prototypes and of mixing original with corrected labels.","pith_inferences":["The density-isolation premise implies the method should fail when label noise is systematic and clustered—for instance, when two visually similar classes are heavily confused, the mislabeled samples may form their own dense clusters that compete with the true prototypes.","The same multi-prototype selection could be applied to other self-training setups, such as semi-supervised learning or domain adaptation, wherever features are dense for confident samples.","Because the accuracy is insensitive to the number of sampled images m and to the clustering method, the core ingredient is likely the multi-prototype voting rule, not the specific density estimate.","The paper's Figure 3 density plots suggest a testable diagnostic: before training, one could compute the density separation between correct and wrong labels in feature space and use it to predict whether SMP will help."],"forward_implications":["A classifier can be trained on a noisy web-scale dataset without a manually cleaned subset, removing a major bottleneck in practice.","Using several prototypes per class instead of one provides a measurable accuracy boost, suggesting prototype-based label correction should use multi-modal representations.","Combining the corrected labels with the original noisy labels (α ≈ 0.5) works better than either alone, indicating the original labels still carry information about hard samples.","The framework's benefit extends to settings where a small clean set is available, where it also improves over prior methods."],"supporting_citations":[{"why":"Provides the density-based prototype selection method that SMP adapts with cosine similarity.","marker":"[31]"},{"why":"Supplies the single-prototype baseline (CleanNet) that SMP extends to multiple prototypes, and the Food101-N dataset.","marker":"[15]"},{"why":"Provides the Clothing1M benchmark and its noisy labels used in the main experiments.","marker":"[38]"},{"why":"A key baseline that replaces all labels with pseudo-labels; SMP outperforms it by retaining original labels.","marker":"[35]"},{"why":"A transition-matrix baseline that performs poorly on real noisy data; SMP compares against it.","marker":"[25]"},{"why":"A meta-learning baseline (MLNT-Teacher) whose reported accuracy SMP exceeds.","marker":"[16]"}],"fun_headline_variants":["Self-learning with multiple prototypes beats noisy-label training","No assumptions, no extra labels: self-learning fixes noisy image data","Iterative self-training uses multiple prototypes to correct noisy labels","Deep self-learning cleans noisy labels without external supervision","Multiple prototypes enable robust training on real noisy datasets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that images with correct labels are dense in the network's feature space while mislabeled images are isolated, so that picking high-density, spread-out prototypes recovers clean class representatives even from a network trained only on noisy labels.","fun_headline_variants_meta":{"raw":{"variants":["Self-learning with multiple prototypes beats noisy-label training","No assumptions, no extra labels: self-learning fixes noisy image data","Iterative self-training uses multiple prototypes to correct noisy labels","Deep self-learning cleans noisy labels without external supervision","Multiple prototypes enable robust training on real noisy datasets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000514,"raw_usage":{"total_tokens":2440,"prompt_tokens":829,"completion_tokens":1611,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":445,"completion_tokens_details":{"reasoning_tokens":1534}},"tokens_in":445,"tokens_out":1611,"duration_ms":10449,"temperature":1.0,"reasoning_tokens":1534,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:51:45.852989+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a noisy dataset with systematic confusions between two visually similar classes, compute the paper's density ρ for each training image and measure how well a threshold on ρ separates correct from incorrect labels; if the separation is near chance, the prototype selector will pick mislabeled images and the corrected-label accuracy will not exceed the original label accuracy.","supporting_citations":[{"cited_title":"Clustering by fast search and find of density peaks","cited_arxiv_id":null,"evidence_quote":"Provides the density-based prototype selection method that SMP adapts with cosine similarity."},{"cited_title":"Cleannet: Transfer learning for scalable image classi- fier training with label noise","cited_arxiv_id":null,"evidence_quote":"Supplies the single-prototype baseline (CleanNet) that SMP extends to multiple prototypes, and the Food101-N dataset."},{"cited_title":"Learning from massive noisy labeled data for im- age classification","cited_arxiv_id":null,"evidence_quote":"Provides the Clothing1M benchmark and its noisy labels used in the main experiments."},{"cited_title":"Joint optimization framework for learning with noisy labels","cited_arxiv_id":null,"evidence_quote":"A key baseline that replaces all labels with pseudo-labels; SMP outperforms it by retaining original labels."},{"cited_title":"Making deep neural networks robust to label noise: A loss correction approach","cited_arxiv_id":null,"evidence_quote":"A transition-matrix baseline that performs poorly on real noisy data; SMP compares against it."},{"cited_title":"Learning to learn from noisy labeled data","cited_arxiv_id":null,"evidence_quote":"A meta-learning baseline (MLNT-Teacher) whose reported accuracy SMP exceeds."}],"review_version":1}