{"id":"a8b5b9c8-177b-4251-99fc-f49c536353a5","arxiv_id":"2502.00386","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"ALR combines temporal-ensemble soft labels with an entropy regularizer to reduce overfitting to noisy labels, reporting accuracy gains over several baselines on CIFAR-10/100, ANIMAL-10N, Clothing1M, and WebVision.","lead":"Researchers propose ALR, a training recipe that replaces noisy hard labels with temporally averaged soft predictions and adds an entropy penalty to sharpen confident predictions. It is meant to make label-noise-robust training simpler than sample-selection methods, and it reports accuracy gains on CIFAR and real-world noisy datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The entropy regularization in Eq. (5) hardens whatever class the model is most confident about, not specifically clean labels; Theorem 1's premise that high confidence implies correctness is unverified and fails under class imbalance or asymmetric noise, so the 'outperforms SOTA' claim is not…","rationale":"The paper's central claim is that ALR outperforms state-of-the-art methods by decoupling avoidance of noisy-label fitting from thorough learning of clean samples. The mechanism that supposedly achieves the latter is the entropy regularizer of Eq. (5) combined with Theorem 1, which asserts that high-confidence samples are hardened. I read the method in good faith: the temporal ensembling of Eq. (3) is a reasonable extension of SELC, the ablation in Tables 8–9 shows the entropy term helps on the tested balanced CIFAR settings, and the reported gains over the listed baselines are plausible. The concern is not that the method is fraudulent or that the empirical wins are impossible; it is that the theoretical guarantee as stated does not follow from the premises, and the one real-world dataset the authors themselves identify as most challenging (WebVision) shows the method below ELR. The missing assumption is that confidence is a proxy for label correctness. Under symmetric noise on balanced CIFAR, early-learning makes this proxy reasonably good, which is why the method works there. Under class imbalance or asymmetric confusing-class noise, the model can be confidently wrong, and the entropy term will amplify that error because it only sees p, not the label or the truth. Since the paper explicitly acknowledges class imbalance for WebVision but does not test the entropy term's behavior there, the central claim is conditional on a regime check. The reader's weakest_assumption identified the same premise; my read adds the mechanism: the target t tracks p, so the cross-entropy term cannot rescue the confidently-wrong case. A conditional verdict is appropriate: require the imbalance test (or an equivalent confidence-correctness correlation analysis) and a comparison against DivideMix, which is conspicuously absent from the baselines despite being described in Section 2. The proof errors in Section 4.3 are real but secondary; fixing them would not address the premise.","tokens_in":20097,"tokens_out":8989,"duration_ms":98269,"concrete_test":"Run ALR on a class-imbalanced CIFAR-10 variant with imbalance ratio 100 and 40% symmetric noise, using the same hyperparameters as the paper (α=0.9, λ=0.2, m=30, ResNet-34), and compare ALR with and without the entropy term. Report tail-class test accuracy and the fraction of high-confidence (p[u] > ε) training samples whose target t[u] differs from the clean label. If removing the entropy term improves tail-class accuracy or reduces confident-wrong predictions, the Theorem 1 premise fails; this isolates the exact mechanism the paper claims is safe.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central justification of ALR rests on Theorem 1 in Section 4.3, which shows that for the top predicted class u, ∂ℓ_ALR/∂z[u] = p[u](1 − λ(log p[u] + H(p))) − t[u]. For p[u] > ε the entropy term makes this gradient smaller than the plain cross-entropy gradient p[u] − t[u], and the paper concludes that 'high-confidence samples are typically clean' so the method 'prioritizes fitting clean samples.' This is the load-bearing step, and it is not secure. The entropy term is a function only of the prediction p; it does not know whether u is the true class, the noisy label, or neither. For any sample on which the model is confidently wrong—e.g., a tail-class image confidently predicted as a head class under WebVision-style imbalance, or an instance whose asymmetric noisy label is a visually similar class—the same p[u] > ε condition holds and the entropy gradient drives z[u] upward, further hardening the wrong belief. The cross-entropy term does not reliably counteract this, because t is updated as t(e) = αt(e−1) + (1−α)p(e) (Eq. 3): when the model is consistently confident, t[u] tracks p[u], so p[u] − t[u] ≈ 0 and the entropy term dominates. The paper's own WebVision result is evidence of the risk: the authors state the dataset is highly class-imbalanced and ALR top-1 (74.72) is below ELR (76.26), contradicting the abstract's SOTA claim. No experiment measures the correlation between confidence and label correctness, so the regime in which the premise fails is never examined. The proof also has local errors—H(p) → 0 as p[u] → 1, so the claimed boundary behavior in Eq. (16) is not correct—but the substantive issue is the premise itself.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Adaptive Label Refinement (ALR), a training method for learning with noisy labels. After a warm-up phase, ALR replaces the original hard labels with temporally ensembled soft labels (Eq. 3) and adds a negative entropy regularizer (Eq. 5), training with L_ALR = L_CE + lambda * L_e. The authors argue that soft-label refurbishment reduces memorization of noisy labels while the entropy term progressively 'hardens' high-confidence targets, thereby improving learning from clean samples without explicit sample selection. The method is evaluated on CIFAR-10 and CIFAR-100 under symmetric and asymmetric noise and on ANIMAL-10N, Clothing1M, and mini-WebVision, with ablations and hyperparameter sensitivity experiments. The central claim is that ALR outperforms state-of-the-art methods while being simpler and requiring no noise-rate information or auxiliary clean data.","tokens_in":20467,"tokens_out":10670,"duration_ms":104590,"significance":"If the empirical results are taken at face value, the paper offers a simple and generally applicable recipe for label-noise robustness, with a plausible decoupling of noise avoidance and clean-sample learning. Strengths include fixed hyperparameters across the CIFAR noise conditions, ablations showing that both the label-refinement and entropy components contribute, and results on several real-world noisy datasets. The comparison is against external baselines, and no reported test number is fit by the method, so the circularity burden is low. However, the 'outperforms state-of-the-art' claim is weakened by the WebVision results, where ELR scores higher on the primary metric, and by the absence of DivideMix, a standard strong baseline. The theoretical analysis in Section 4.3 is also not as supportive as claimed: it establishes a gradient comparison for high-confidence predictions, not for clean samples. These issues are fixable but currently affect the paper's headline claims.","major_comments":[{"comment":"Theorem 1 as used in the text does not establish that ALR prioritizes clean samples. The inequality ∂ℓ_ALR/∂z[u] < p[u] − t[u] holds for every high-confidence prediction p[u] > ε, regardless of whether the predicted class is the true class, the noisy label, or neither. The entropy term depends only on p, so it cannot distinguish clean labels from noisy ones. Under class imbalance (acknowledged for WebVision) or asymmetric noise, a confidently wrong prediction satisfies the same condition, and the gradient then hardens the incorrect belief. Because t(e) in Eq. (3) tracks p(e) for consistently confident samples, the cross-entropy term does not reliably counteract this effect. No experiment in the paper measures the correlation between prediction confidence and label correctness, so the premise 'high-confidence samples are typically clean' is unverified. The theorem's conclusion should be restated as a property of high-confidence predictions, and the method's behavior under confident-but-wrong predictions should be examined empirically.","section":"§4.3, Eq. (7)-(8)"},{"comment":"The proof of Theorem 1 contains formal errors. Eq. (18) writes f'(p[u]) = 1/p[u] − (log p[k]+1) with the index k in place of u, and the derivative ∂H(p)/∂p[u] is computed as if the other probabilities were fixed even though the probability vector satisfies Σ p[k] = 1. As written, the existence and uniqueness of the threshold ε is not rigorously established; at best, the argument is a pointwise statement for a fixed distribution. The proof should be corrected or the theorem weakened accordingly before the result is cited as a guarantee. The final gradient formula in Eq. (7) is nevertheless correct, which suggests the issue is in the proof presentation rather than in the underlying calculation.","section":"§4.3, Eq. (17)-(18)"},{"comment":"The abstract's claim that 'ALR outperforms state-of-the-art methods' is contradicted by the authors' own Table 7. On WebVision top-1, ELR reaches 76.26 while ALR reaches 74.72; on top-5, ELR reaches 91.26 versus ALR's 90.84. The text acknowledges the result as 'slightly inferior' and attributes it to class imbalance, but no experiment in the paper tests that attribution. The abstract and conclusion should be qualified to reflect that ALR is competitive with, but not uniformly superior to, the compared methods on this benchmark.","section":"Abstract; §5.3, Table 7"},{"comment":"DivideMix, which the authors cite and describe in Section 2, is a standard strong baseline for the CIFAR and real-world label-noise benchmarks used in the paper, yet it is absent from all comparison tables. Without this baseline, the statement that ALR outperforms state-of-the-art methods is not supported by the evidence presented. The authors should either add DivideMix to the comparison (using published numbers under the same protocol) or explicitly restrict their superiority claims to the methods actually compared.","section":"§2, §5.2-5.3, Tables 2-7"}],"minor_comments":[{"comment":"The warm-up condition is inconsistent: Eq. (3) states that the warm-up phase is e ≤ m and refinement starts at e > m, while Algorithm 1 (line 3) switches to refinement when e < m. Please align the equation and the pseudocode.","section":"§4.2, Eq. (3) and Algorithm 1"},{"comment":"The sentence 'Substituting Eq. (3) into Eq. (3) gives the loss function' should read 'Substituting Eq. (3) into Eq. (1) gives the loss function.'","section":"§4.2, Eq. (4)"},{"comment":"The sensitivity analysis swaps the roles of α and λ. Section 5.6.1 tests the entropy-loss weight but labels it α and concludes 'we set α to 0.2'; Section 5.6.3 tests the temporal momentum but labels it λ and concludes 'we fix λ at 0.9.' Elsewhere in the paper α is the temporal momentum and λ is the entropy-loss weight. The same swap appears in the captions of Figures 7 and 9. In addition, Section 5.6.2 and Figure 8 call m the 'momentum hyperparameter' although m denotes the number of warm-up epochs.","section":"§5.6 and Figures 7-9"},{"comment":"The notation ˜yi ∈ (0,1)^K is inaccurate for a one-hot vector; the entries are 0 or 1, so the correct set is {0,1}^K (or the interval notation should be clarified).","section":"§3.1, Eq. (1)"},{"comment":"The proof contains typos: Eq. (11) differentiates t[k] instead of p[k], Eq. (13) contains a duplicated factor, and the line 'By combining Eq. (3) and (4)' refers to equations in the proof rather than to the main-text Eq. (3) and Eq. (4). These should be corrected for readability.","section":"§4.3, proof of Theorem 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of Neural Networks and the proposed method is simple and likely useful. My main concerns are the overbroad SOTA claim, the missing DivideMix baseline, and the fact that Theorem 1 is used to justify a claim it does not actually prove. None of these appear to be unresolvable, and I do not see evidence of questionable research practices. I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this paper gives you a simple, threshold-free label-refurbishment method—temporal ensembled soft labels plus a small entropy penalty—and on CIFAR-10/100 the numbers hold up. The ablations show both components matter, and the authors fix hyperparameters across datasets, which is good practice. The idea is not new in its parts (SELC + entropy minimization), but the combination is clean and easy to implement.\n\nWhat I'd flag: the theory in Section 4.3 is not sound as written. The proof of Theorem 1 treats H(p) as if it were independent of p[u] when differentiating, so the expression for f'(p[u]) is wrong. And the boundary claim that f(p[u]) -> H(p) > 0 as p[u]->1 is false because H(p) -> 0 when one class dominates. So the existence of a clean threshold ε is not actually established. The conceptual claim that 'high-confidence samples are typically clean' is also doing heavy lifting without direct evidence; under class imbalance (which they acknowledge in WebVision) confident predictions can be confidently wrong, and that's likely why ALR's WebVision top-1 (74.72) trails ELR (76.26). The abstract's 'outperforms state-of-the-art' doesn't survive that comparison, and DivideMix is discussed but never included in the tables.\n\nMinor but sloppy: the hyperparameter sensitivity section mixes up roles—Section 5.6.1 says α controls the entropy loss, Section 5.6.2 calls m the momentum, Section 5.6.3 describes λ as a warm-up phase. These are mechanical errors and easy to fix, but they should have been caught.\n\nOverall, the empirical contribution is worth a serious referee: it's a reproducible recipe with decent gains on artificial noise and on ANIMAL-10N/Clothing1M, and the flaws are correctable. I would send it out rather than desk reject, with a request to fix the proof, add a properly tuned DivideMix comparison, include a confidence-correctness correlation analysis, and rewrite the abstract to match the actual results.","headline":"Simple, solid label-refurbishment variant whose empirical core holds up on CIFAR, but the theory is shaky and the SOTA claim overreaches.","tokens_in":21038,"tokens_out":2340,"would_cite":true,"duration_ms":24776,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a two-term loss—cross-entropy against temporally ensembled soft labels plus a prediction-entropy regularizer—can avoid memorizing wrong labels while learning clean samples thoroughly, outperforming more complex…","keywords":["label noise","label refinement","temporal ensembling","entropy regularization","soft labels","deep neural networks","image classification","noisy labels"],"falsifier":"A concrete check: on CIFAR-10 with 40% asymmetric noise (for example, the paper's cat-to-dog and bird-to-airplane flips), record each mislabeled sample's top-1 confidence $p[u]$ during the refinement phase. If a large fraction of mislabeled samples exceed the hardening threshold $\\varepsilon$ and the model's accuracy is no better than the same model without the entropy term, then the claim that confident samples are clean is not doing the work claimed.","tokens_in":19911,"feed_emoji":"🏷️","tokens_out":12590,"duration_ms":111424,"temperature":0.7,"pith_summary":"Adaptive Label Refinement (ALR) is a training procedure that separates two problems usually solved together: not memorizing incorrect annotations, and learning as much as possible from correct ones. It replaces one-hot labels with soft labels formed by a temporal ensemble of the model's own past predictions, which dilutes the influence of any single noisy annotation. To stop those soft labels from making the model lazy on clean data, it adds a prediction-entropy term that gradually sharpens high-confidence soft labels back toward one-hot form. A gradient analysis in the paper shows that the combined loss pushes harder on confident samples while leaving low-confidence samples less forced, and the authors argue this is why the method can skip explicit clean/noisy partitioning. Experiments on CIFAR-10/100 with injected noise and on ANIMAL-10N, Clothing1M, and WebVision are reported as showing that this simple combination matches or beats the compared methods.","feed_headline":"Soft labels plus entropy loss top noisy-label benchmarks","feed_subtitle":"Temporally ensembled soft labels plus an entropy term beat prior noisy-label methods on five benchmarks.","key_machinery":"The load-bearing object is the combined loss $\\mathcal{L}_{\\mathrm{ALR}} = \\mathcal{L}_{\\mathrm{ce}} + \\lambda \\mathcal{L}_e$, whose two terms are assigned two different jobs. The temporal-ensemble target $t^{(e)}$ defined by Eq. (3) is the label-refinement mechanism: it blends the original noisy one-hot label with a running average of the model's predictions, so any single wrong annotation is diluted over time. The entropy term $\\mathcal{L}_e$ is the hardening mechanism: because its gradient contains $\\log p[u] + H(p)$, the regularizer changes sign at a confidence threshold $\\varepsilon$, so confident predictions are pushed toward one-hot while uncertain predictions are left alone. The theorem's sign analysis of this gradient is what connects the two terms: it shows that the entropy regularizer does not interfere with the soft-label dilution for low-confidence samples, and it supplies the extra learning pressure on confident samples that plain label refurbishment lacks.","core_discovery":"On the paper's own terms, the central claim is that the loss $\\mathcal{L}_{\\mathrm{ALR}} = \\mathcal{L}_{\\mathrm{ce}} + \\lambda \\mathcal{L}_e$—where the cross-entropy term is computed against temporally ensembled soft labels $t^{(e)} = \\alpha t^{(e-1)} + (1-\\alpha)p^{(e)}$ after a warm-up of $m$ epochs, and $\\mathcal{L}_e = -\\sum_i \\sum_k p_i[k]\\log p_i[k]$ is the entropy of the predictions—simultaneously resists memorizing wrong labels and learns clean labels thoroughly. The paper derives the per-sample gradient with respect to the top predicted logit $z[u]$ as $\\partial \\ell_{\\mathrm{ALR}}/\\partial z[u] = p[u]\\bigl(1-\\lambda(\\log p[u]+H(p))\\bigr)-t[u]$, and proves there is a unique confidence threshold $\\varepsilon$ such that for $p[u]>\\varepsilon$ the ALR gradient is smaller than the plain cross-entropy gradient while for $p[u]\\le \\varepsilon$ it is not. This is interpreted as giving high-confidence—hence, the paper assumes, clean—samples stronger learning pressure while not forcing low-confidence samples toward possibly wrong labels. The paper reports that this mechanism outperforms robust losses, regularizers, sample-selection methods, and label-refurbishment baselines on the five tested benchmarks.","pith_inferences":["Beyond the paper, the gradient formula suggests a family of confidence-gated losses: any regularizer whose derivative changes sign at a confidence threshold should reproduce the harden-confident, leave-uncertain behaviour, so the entropy term may be replaceable by cheaper or better-calibrated alternatives.","A stress test the paper does not run is class-imbalanced noise combined with asymmetric flips; the authors' own explanation for the WebVision shortfall identifies confidence on rare classes as the weak point, so a dataset with severe imbalance and high noise would show whether the high-confidence-equals-clean premise can be sustained.","One could also apply the same temporal-ensembled soft labels plus entropy hardening to semi-supervised or partial-label settings, where pseudo-labels are refined iteratively; this is an extension the paper does not claim.","A practical follow-up would be to log the threshold $\\varepsilon$ and the fraction of mislabeled samples whose confidence crosses it during training; if that fraction grows, a confidence-calibration or abstention mechanism would be needed."],"forward_implications":["In the paper's experiments, ALR outperforms the compared methods under both symmetric and asymmetric noise, including asymmetric settings where small-loss selection criteria are known to struggle.","A classifier can adopt ALR by changing only the loss and the label-update rule; no noise-rate estimate, clean auxiliary set, or explicit sample selection is required.","On clean data, the soft-label target and entropy term act as a regularizer, and the paper's ablations report that ALR still slightly beats plain cross-entropy on CIFAR-10 and CIFAR-100 without injected noise.","The hyperparameters $m$, $\\alpha$, and $\\lambda$ are reported to be stable across datasets, with the main caution being that a very large momentum $\\alpha$ hurts CIFAR-100 under asymmetric noise."],"supporting_citations":[{"why":"Supplies the early-learning regularization baseline ELR and the historical-prediction idea that ALR extends; its reported accuracy is the main comparison on CIFAR and WebVision.","marker":"Liu et al., 2020"},{"why":"Supplies the SELC temporal-ensembling label-correction strategy that directly inspires ALR's soft-label update rule.","marker":"Lu and He, 2022"},{"why":"Establishes the early-learning and memorization phenomenon that justifies the warm-up phase and the assumption that early confident predictions are reliable.","marker":"Arpit et al., 2017"},{"why":"Supplies the symmetric and asymmetric noise injection protocol used to create the CIFAR-10/100 benchmarks.","marker":"Patrini et al., 2017"},{"why":"Supplies the bootstrapping label-refurbishment idea of mixing one-hot labels with model predictions that ALR generalizes to temporal ensembling.","marker":"Reed et al., 2015"},{"why":"Supplies the ANIMAL-10N real-world noisy dataset and the SELFIE refurbishment baseline used in the evaluation.","marker":"Song et al., 2019"}],"fun_headline_variants":["Soft labels and entropy: simple noisy-label fix","Temporal ensembling plus entropy beats prior methods","ALR: decouple wrong-label avoidance from clean learning","Without noise rates, adaptive label refinement wins"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method rests on the premise that high-confidence predictions are almost always correct, so hardening them can only help; when class imbalance or asymmetric noise produces confidently wrong predictions, that premise fails and the entropy term would harden incorrect soft labels.","fun_headline_variants_meta":{"raw":{"variants":["Soft labels and entropy: simple noisy-label fix","Temporal ensembling plus entropy beats prior methods","ALR: decouple wrong-label avoidance from clean learning","Without noise rates, adaptive label refinement wins"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000375,"raw_usage":{"total_tokens":2057,"prompt_tokens":1059,"completion_tokens":998,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":675,"completion_tokens_details":{"reasoning_tokens":937}},"tokens_in":675,"tokens_out":998,"duration_ms":10293,"temperature":1.0,"reasoning_tokens":937,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T19:15:19.094462+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete check: on CIFAR-10 with 40% asymmetric noise (for example, the paper's cat-to-dog and bird-to-airplane flips), record each mislabeled sample's top-1 confidence $p[u]$ during the refinement phase. If a large fraction of mislabeled samples exceed the hardening threshold $\\varepsilon$ and the model's accuracy is no better than the same model without the entropy term, then the claim that confident samples are clean is not doing the work claimed.","supporting_citations":[{"cited_title":", author Niles-Weed, J","cited_arxiv_id":null,"evidence_quote":"Supplies the early-learning regularization baseline ELR and the historical-prediction idea that ALR extends; its reported accuracy is the main comparison on CIFAR and WebVision."},{"cited_title":", author Jastrz e bski, S","cited_arxiv_id":null,"evidence_quote":"Establishes the early-learning and memorization phenomenon that justifies the warm-up phase and the assumption that early confident predictions are reliable."},{"cited_title":", author Rozza, A","cited_arxiv_id":null,"evidence_quote":"Supplies the symmetric and asymmetric noise injection protocol used to create the CIFAR-10/100 benchmarks."},{"cited_title":", author Lee, H","cited_arxiv_id":null,"evidence_quote":"Supplies the bootstrapping label-refurbishment idea of mixing one-hot labels with model predictions that ALR generalizes to temporal ensembling."},{"cited_title":", author Kim, M","cited_arxiv_id":null,"evidence_quote":"Supplies the ANIMAL-10N real-world noisy dataset and the SELFIE refurbishment baseline used in the evaluation."}],"review_version":1}