Early Stopping Against Label Noise Without Validation Data

Lei Feng; Suqin Yuan; Tongliang Liu

REVIEW 3 major objections 6 minor 1 cited by

Early Stopping Against Label Noise Without Validation Data

T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Label Wave is an early-stopping rule that uses the first local minimum of smoothed prediction changes on the training set to select a model near best test accuracy, with no validation data needed.

desk verdict Label Wave is a genuinely useful validation-free early stopping rule for noisy labels, with broad and honest experiments, but the core argmin-alignment claim needs better evidence and the key hyperparameters should be reported. read the letter →

arxiv 2502.07551 v1 pith:TTXFVXAH submitted 2025-02-11 cs.LG

Suqin Yuan , Lei Feng , Tongliang Liu This is my paper · ORCID

classification cs.LG

keywords labelnoiseearlystoppingmodelselectionpredictionchangeslearningconfusingpatternsnoisylabelsdeepneuralnetworksnovalidationdata

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Label Wave is an early-stopping rule for training deep networks on datasets with label noise that needs no hold-out validation set. The paper argues that, during noisy training, the number of training examples whose predicted label flips between consecutive epochs—the prediction-change metric—drops while the model learns clean patterns and then rises once it starts fitting mislabeled examples. Stopping at the first local minimum of a smoothed version of this curve therefore selects a model close to the best test accuracy the run ever reaches, before the model overfits the noise. The authors demonstrate the rule across noise levels from 10% to 80%, several architectures and datasets, and show it often beats standard validation-set selection when combined with existing noisy-label methods. The paper also names the interval between the prediction-change minimum and the later stabilization point the 'learning confusing patterns' stage.

What carries the argument

The carrying object is the prediction-changes metric, $PC_t = \sum_i \mathbb{1}\{\hat{y}_t^i \neq \hat{y}_{t-1}^i\}$, counting how many training examples the model classifies differently than in the previous epoch. Smoothed by a moving average, $PC'_t$, this curve is the 'wave' whose first local minimum defines the stopping epoch. The paper uses it as a proxy for the transition between two stages it identifies: the model first 'learns simple patterns' with steadily dropping prediction fluctuations, then enters a transitional 'learning confusing patterns' stage in which fluctuations rise as the model fits mislabeled examples, before memorization eventually stabilizes them again. The method needs no labels beyond the noisy training labels and no validation split, because it only compares the model's own predictions across epochs.

What would settle it

Train a network on a noisy-label dataset under a regime the paper did not test, such as 5% symmetric noise or a strong robust regularizer, and record the test accuracy of the model at the first local minimum of the smoothed prediction-change curve. If that model is consistently more than a few points below the best test accuracy seen during training, or if the first local minimum occurs after the test error has already peaked, the central correlation fails in that regime.

Watch

Extended reading notes

Core claim

The paper's central claim is that the first local minimum of the smoothed prediction-changes curve marks the optimal early stopping point when training with label noise. Concretely, let $PC_t$ be the number of training examples whose predicted label changes from epoch $t-1$ to $t$, and let $PC'_t$ be its moving average over $k$ epochs; the selected model is the one at $t_{\text{first-min}}$, the first local minimum of $PC'_t$ (Eq. 4). The authors report that this point coincides with the epoch before the model begins to fit mislabeled examples, and that models selected this way land within roughly 0.1 to 0.9 percentage points of the global maximum test accuracy across symmetric, instance-dependent, and real-world noise, with and without other noisy-label learning methods. They further claim the method improves precision over hold-out validation of comparable size, as measured by Kendall $\tau$ correlation with test accuracy, and that it consistently outperforms validation-set selection with 5-30% hold-out data in the tested settings.

Load-bearing premise

The load-bearing premise is that the first local minimum of the smoothed prediction-change curve reliably marks the epoch just before the model starts overfitting mislabeled examples; the paper establishes this correlation empirically on the tested configurations and states it can fail when training shows no 'learning confusing patterns' stage, as with very low or no label noise or robust regularization.

Editorial extensions

If this is right

Training can use the full noisy training set instead of carving out a validation split, so the learning signal is not reduced.
When combined with existing noisy-label methods, the rule often beats hold-out validation using 5% to 30% of the training data on the tested CIFAR-10 and CIFAR-100 settings.
The same first-local-minimum rule transfers across ResNet depths, VGG, Inception-v3, DenseNet, multiple optimizers, batch sizes, and learning rates, and across image and text datasets.
The 'learning confusing patterns' stage gives a new way to describe training dynamics under label noise: a period where the model's predictions are least stable and both generalization and fitting performance decline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the prediction-change signal could be monitored online during training to trigger other interventions, such as reweighting or relabeling, rather than only stopping; the paper does not discuss this extension.
Editorial inference: if the observed correlation persists at larger scale, Label Wave could replace hold-out validation in production settings where clean labels are costly, but the paper tests only one text dataset and several image benchmarks, so transfer to other modalities remains open.
Editorial inference: the method as stated has no built-in detection of its own failure regime; a practical extension would flag when no clear first local minimum exists and fall back to a default training horizon.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The short version: Label Wave is a simple, cheap, validation-free early stopping rule for noisy-label training, with unusually broad empirical coverage. The catch is that the core assumption — that the first smoothed local minimum of prediction changes lines up with the best test epoch — is asserted, not shown, and the paper never reports two hyperparameters that matter for reproducing it.

What's new: instead of using prediction dynamics to select which examples to clean or reweight (the Late Stopping line of work), the authors use the whole training-set prediction-flip rate as a global stopping signal. That's a genuinely different use of the same machinery, and it works in practice in their tables: across 20–80% symmetric and instance-dependent noise, seven architectures, seven datasets, and seven existing noisy-label methods, the gap to the oracle best test accuracy is typically 0.1–0.9 points. They also beat validation-based early stopping with 5–30% hold-out, which is the right baseline for the no-validation-data selling point. The paper is honest about where it fails: Appendix C.3 says the method is not applicable at very low or zero noise, or under robust regularization, because the 'learning confusing patterns' stage may not appear. That is a real limitation, not a buried one.

The soft spots are real but narrower than the stress-test note implies. First, the main text's Algorithm 1 defines patience p but never reports its value; k appears only in Appendix E. That's a reproducibility gap, not evidence of cherry-picking, but it matters. Second, Appendix E reports Pearson correlation between PC′ and test accuracy, which is the wrong statistic: a signal can be well correlated globally while its first local minimum is early, late, or spurious. The distribution of the first local minimum relative to the oracle epoch is never shown. So the central claim is an empirical regularity with solid support inside the tested grid and unknown support outside it. The stress-test's caveat about a small spurious local minimum appearing before the real transition is exactly the scenario the missing statistic would rule out, so it's a fair request for revision.

Bottom line: this is a practitioner-friendly method with honest experiments, not a theoretical result. A serious referee should push for code, exact k and p, and the argmin-alignment plot, but the paper deserves that referee time.

Referee Report

3 major / 6 minor

Summary. The paper proposes Label Wave, an early stopping method for training deep networks under label noise that does not use a hold-out validation set. The method computes prediction changes (PC) on the training set, smooths them with a moving average (PC′), and stops at the first local minimum of PC′, interpreted as the moment when the model begins to overfit mislabeled examples. The authors support the method with an empirical three-stage description of training under noise, report small gaps between the test accuracy of the selected model and the global maximum test accuracy over training across many datasets, architectures, noise levels, and optimizers, and show that integrating Label Wave with existing noisy-label methods improves their test accuracy relative to hold-out validation. The paper is empirical in nature; the central claim is that the first local minimum of the smoothed prediction-change curve selects a near-optimal model, and this claim is supported by Table 1 and the integration experiments in Tables 2 and 3.

Significance. If the central claim holds, Label Wave is practically valuable: it replaces a validation split with a training-set statistic at negligible extra cost and improves several existing methods for learning with noisy labels. The empirical scope is broad, including symmetric, instance-dependent, and real-world noise; seven architectures; multiple datasets; and integration with seven existing methods. The use of means and standard deviations over five seeds in Table 1 and the candid limitations statement in Appendix C.3 are strengths, as is the sensitivity analysis for the moving-average window in Appendix E. The main liability is that the load-bearing argmin-alignment assumption—that the first local minimum of PC′ marks the best-generalizing epoch—is asserted rather than directly documented, and the method's operational hyperparameters are not fully disclosed in the main text.

major comments (3)

[§3.3, Eq. (4) vs. Algorithm 1] There is a mismatch between the stopping criterion stated in Eq. (4) and the pseudocode. Eq. (4) says the early stopping point is the first local minimum of PC′t, but Algorithm 1 stores a new candidate whenever PC′t is below the running minimum v, so if PC′t later falls below the value at the first confirmed local minimum, the returned model is a later, deeper minimum rather than the first local minimum. Please state precisely which object is evaluated, reconcile the pseudocode with the definition, and confirm that Table 1 reports results for that same object.
[§3.3 and Appendix E] The operational rule depends on the moving-average window k (Eq. 3) and the patience p (Algorithm 1), but the main text never reports a value for p, and k is only discussed in Appendix E. The Pearson correlations reported there (r between about −0.86 and −0.96) show that PC′ and test accuracy move in opposite directions overall, but they do not establish the argmin alignment asserted by Eq. (4): a strongly negatively correlated curve can have an early, late, or spurious first local minimum. Please report k and p for every main result and provide the distribution of the selected epoch relative to the epoch of maximum test accuracy, or a sweep of the Difference metric from Appendix C.6 over k and p, across the five seeds and the main settings of Table 1.
[Appendix C.3] The paper candidly concedes that in very low or no label noise, and under robust regularization, the training process may lack a 'learning confusing patterns' stage and the original Label Wave method may not identify an appropriate stopping point. Because Algorithm 1 unconditionally returns some first local minimum, the method will not fail gracefully in those regimes; it will halt at a possibly spurious minimum. This is a load-bearing limitation for the paper's framing that early stopping is needed 'in the presence of label noise.' Please either add an applicability condition that can be checked from PC′ alone, or clearly restrict the method's scope in the abstract and conclusion, and support the restriction with experiments at 0% and 5% noise instead of only the 10% cases in Table 1.

minor comments (6)

[Algorithm 1] In Algorithm 1, line 2 advances t by n training steps, while PC in Eq. (2) is defined per epoch; please clarify whether 'steps' means epochs or update iterations, since this changes the meaning of k in Eq. (3).
[§3.1] The stability metric in Eq. (1) is defined on the clean subset D_c, which is unavailable in the no-validation-data setting that motivates the method; please state explicitly that Eq. (1) is used only for the analysis in Section 3.2 and not by Algorithm 1.
[Abstract and §3.3] The abstract says 'minimum fluctuations in predictions typically occur at the training epoch before the model excessively fits mislabeled data,' but the method uses the first local minimum of the smoothed PC′t curve; please align the wording with Eq. (4) so the stopping point is not described as a single raw epoch.
[Appendix B] Appendix B states that 20% of the training data is held out for validation during training, which seems to conflict with the paper's no-validation-data premise; please clarify whether this hold-out is used only for the analysis in Section 3 and for baselines, and not for selecting the Label Wave model in Table 1.
[Table 5] The first row of Table 5 is labeled simply 'Label Wave' and reports much lower accuracy than the following rows; adding a footnote describing the exact configuration (e.g., no Batch Normalization or data augmentation) would prevent misinterpretation.
[Appendix C.3] There are minor typographical inconsistencies such as 'learning confusion patterns' instead of 'learning confusing patterns' and inconsistent capitalization of the coined stage; a careful copyedit is recommended.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: Label Wave's stopping rule is defined from a label-free prediction-change metric, and the near-oracle results are empirical outcomes rather than consequences of the definition.

full rationale

The claimed chain is: define PC_t from prediction flips on the training set (Eq. 2), smooth it into PC'_t (Eq. 3), and stop at the first local minimum (Eq. 4). None of these definitions uses test accuracy or the best-test model, so the small gaps in Table 1 are not forced by construction. Section 3.2 offers an empirical correlation, not a theorem; the possibility that the first local minimum misaligns with the test-error minimum is a transfer or correctness caveat, not circularity. The k-epoch learning metric is cited to the authors' prior work (Yuan et al., 2023) and used in the stability analysis, but the final algorithm depends only on PC, so the self-citation is not load-bearing. Appendix E selects k by Pearson correlation with test accuracy; this is an oracle-informed hyperparameter choice that should be reported in the main text, but the negative correlation is strong for every k shown, and the reported stopping-point outcomes are not algebraically identical to the correlation objective. Appendix C.3 honestly states that the method requires a learning-confusing-patterns stage. Overall this is a low circularity score: no enumerated circular step was identified.

Assumptions & free parameters 2 free parameters · 3 assumptions · 1 invented entities

The method depends on two empirical premises and two hand-set hyperparameters; no formal theorem is offered. The main load-bearing premise is that prediction-change turning points track test-error turning points, which the paper supports only by observation. Invented entities are limited to the conceptual stage label.

free parameters (2)

moving-average window k = tested 1, 2, 3, 5, 10; strongest Pearson correlation at k=3
Appears in Eq. (3) for PC′t. Appendix E selects it by correlation with test accuracy, which is a mild fit to the target metric; no default is stated for the main experiments.
patience p = unspecified
Algorithm 1 uses p as the number of worsening-PC epochs before stopping, but the paper does not give its value. Any re-implementation must choose it, and it affects the selected epoch.

assumptions (3)

domain assumption Noisy label training shows an early phase of falling test error followed by a later phase of rising test error.
Section 3.1 and Figure 2 depend on this two-phase behavior; it is standard empirical knowledge from Arpit et al. and Zhang et al., but it is assumed rather than proven.
domain assumption At the test-error minimum, fitting mislabeled examples starts to impair the model's overall fitting performance, and this impairment is visible in prediction fluctuations.
Section 3.2 states these as observations; the Label Wave stopping rule relies on the visible transition being synchronous with the generalization turning point.
ad hoc to paper The first local minimum of smoothed prediction changes is a reliable, transferable proxy for the best generalization epoch.
This is the operational assumption behind Eq. (4); it is supported only by the experiments in Section 4 and explicitly fails in low-noise or robust-regularization regimes per Appendix C.3.

invented entities (1)

learning confusing patterns
purpose: Names the proposed transitional training stage between Point 1 and Point 2 in learning with noisy labels.
Introduced in Section 3.2 and Appendix A; it is a descriptive label on the same prediction-fluctuation curves used to define it, with no independent measurement or falsifiable handle outside this paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Early Stopping Against Label Noise Without Validation Data." pith.science (2026). https://pith.science/paper/TTXFVXAH

@misc{pith2026250207551,
  author       = {Pith},
  title        = {Pith review of: Early Stopping Against Label Noise Without Validation Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TTXFVXAH}},
  note         = {Machine review of arXiv:2502.07551}
}

read the original abstract

Early stopping methods in deep learning face the challenge of balancing the volume of training and validation data, especially in the presence of label noise. Concretely, sparing more data for validation from training data would limit the performance of the learned model, yet insufficient validation data could result in a sub-optimal selection of the desired model. In this paper, we propose a novel early stopping method called Label Wave, which does not require validation data for selecting the desired model in the presence of label noise. It works by tracking the changes in the model's predictions on the training set during the training process, aiming to halt training before the model unduly fits mislabeled data. This method is empirically supported by our observation that minimum fluctuations in predictions typically occur at the training epoch before the model excessively fits mislabeled data. Through extensive experiments, we show both the effectiveness of the Label Wave method across various settings and its capability to enhance the performance of existing methods for learning with noisy labels.

Figures

Figures reproduced from arXiv: 2502.07551 by the authors.

**Figure 2.** Tracking test error and training error (mislabeled examples) in training process. In real-world applications, however, obtaining prior knowledge about which training examples are mislabeled examples is often impossible, thereby eliminating the possibility of relying on how well the model fits mislabeled examples as a criterion for early stopping. Furthermore, as we aim to eliminate the dependency on hold-out dataset… view at source ↗

**Figure 3.** Using stability and variability metrics to track [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: We aim to compare the test accuracy of models selected at the early stopping point by the [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

**Figure 5.** Figure 5: Based on the multi-metrics we are tracking for the model’s generalization and fitting [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

CLID-MU: Cross-Layer Information Divergence Based Meta Update Strategy for Learning with Noisy Labels
cs.LG 2025-07 conditional novelty 6.0 of 10

CLID-MU replaces the clean meta-dataset in meta-learning with an unsupervised cross-layer divergence metric, improving noisy-label and semi-supervised results on several benchmarks.

Reference graph

Works this paper leans on

22 extracted references · 10 canonical work pages · cited by 1 Pith paper

[1]

learning confusing patterns

Specifically, in Point 2, the fluc- tuations in predictions transitioned from increasing to decreasing trends, eventually stabilizing at a specific value. Here, we empirically explain the evolution of fluctuations in predictions during the training process. Before reaching Point 1, as training progressed, the model’s generalization perfor- mance improved....

work page 2014
[6]

Webvision database: Visual learning and understanding from web data

Wen Li, Limin Wang, Wei Li, Eirikur Agustsson, and Luc Van Gool. Webvision database: Visual learning and understanding from web data. arXiv preprint arXiv:1708.02862,

arXiv
[7]

On the over-memorization during natural, robust and catastrophic overfitting

Runqi Lin, Chaojian Yu, Bo Han, and Tongliang Liu. On the over-memorization during natural, robust and catastrophic overfitting. arXiv preprint arXiv:2310.08847,

arXiv
[8]

SELC: Self-Ensemble Label Correction Improves Learning with Noisy Labels

Sheng Liu, Zhihui Zhu, Qing Qu, and Chong You. Robust training under label noise by over- parameterization. In ICML, pp. 14153–14172, 2022a. Ziming Liu, Ouail Kitouni, Niklas S Nolte, Eric Michaud, Max Tegmark, and Mike Williams. To- wards understanding grokking: An effective theory of representation learning. NeurIPS, 2022b. Yangdi Lu and Wenbo He. Selc:...

work page Pith review arXiv
[9]

Early stopping without a validation set

11 Published as a conference paper at ICLR 2024 Maren Mahsereci, Lukas Balles, Christoph Lassner, and Philipp Hennig. Early stopping without a validation set. arXiv preprint arXiv:1703.09580,

arXiv 2024
[10]

Characterizing datapoints via second-split forgetting

Pratyush Maini, Saurabh Garg, Zachary Chase Lipton, and J Zico Kolter. Characterizing datapoints via second-split forgetting. In ICML 2022: Workshop on Spurious Correlations, Invariance and Stability. Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning . MIT press,

work page 2022
[15]

Selfie: Refurbishing unclean samples for robust deep learning

Hwanjun Song, Minseok Kim, and Jae-Gil Lee. Selfie: Refurbishing unclean samples for robust deep learning. In ICML, 2019a. Hwanjun Song, Minseok Kim, Dongmin Park, and Jae-Gil Lee. How does early stopping help generalization against label noise? arXiv preprint arXiv:1911.08059, 2019b. Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. R...

arXiv 1911
[16]

Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude

12 Published as a conference paper at ICLR 2024 Tijmen Tieleman, Geoffrey Hinton, et al. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine learning,

work page 2024

Show all 22 references

[17]

Learn- ing with noisy labels revisited: A study using real-world human annotations

Jiaheng Wei, Zhaowei Zhu, Hao Cheng, Tongliang Liu, Gang Niu, and Yang Liu. Learn- ing with noisy labels revisited: A study using real-world human annotations. arXiv preprint arXiv:2110.12088,

arXiv
[18]

13 Published as a conference paper at ICLR 2024 A D ISCUSSION ON learning confusing patterns In this section, we delve into Point 2, where the model’s test error transitions from increasing to sta- bilizing, by utilizing the metrics introduced in Section

2024
[22]

This method, as detailed in the paper, employs moving averages of Prediction Changes (PC) for early stopping in training models with noisy labels

45.03±0.38% 45.75 ±0.27% NLS (Wei et al., 2022a) 58.05±0.15% 58.32 ±0.35% SOP (Liu et al., 2022a) 68.53±0.30% 68.78 ±0.27% 20 Published as a conference paper at ICLR 2024 E S ENSITIVITY ANALYSIS OF k VALUE IN MOVING AVERAGES In this appendix, we present a sensitivity analysis ...

2024
[256]

Difference

• Learning Rates (LR.): 0.01, 0.05, 0.001. • Optimizers: – SGD with momentum (Robbins & Monro, 1951; Polyak, 1964): A widely-used opti- mizer, we gauged how its momentum-based optimization worked in tandem with the Label Wave method. – RMSProp (Tieleman et al., 2012): Known fo...

1951
[1938]

Adam: A method for stochastic optimization

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980,

arXiv
[1964]

Grokking: Gen- eralization beyond overfitting on small algorithmic datasets

Alethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, and Vedant Misra. Grokking: Gen- eralization beyond overfitting on small algorithmic datasets. arXiv preprint arXiv:2201.02177,

arXiv
[1999]

Very deep convolutional networks for large-scale image recognition

Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556,

arXiv
[2014]

On the importance of single directions for generalization

Ari S Morcos, David GT Barrett, Neil C Rabinowitz, and Matthew Botvinick. On the importance of single directions for generalization. arXiv preprint arXiv:1803.06959,

arXiv
[2015]

Predicting the generalization gap in deep networks with margin distributions

Yiding Jiang, Dilip Krishnan, Hossein Mobahi, and Samy Bengio. Predicting the generalization gap in deep networks with margin distributions. arXiv preprint arXiv:1810.00113,

arXiv
[2016]

A survey of label-noise representation learning: Past, present and future

Bo Han, Quanming Yao, Tongliang Liu, Gang Niu, Ivor W Tsang, James T Kwok, and Masashi Sugiyama. A survey of label-noise representation learning: Past, present and future. arXiv preprint arXiv:2011.04406,

2011 arXiv
[2017]

Machine vision therapy: Multimodal large language models can enhance visual robustness via denoising in-context learning

Zhuo Huang, Chang Liu, Yinpeng Dong, Hang Su, Shibao Zheng, and Tongliang Liu. Machine vision therapy: Multimodal large language models can enhance visual robustness via denoising in-context learning. arXiv preprint arXiv:2312.02546,

arXiv
[2019]

Table 4: Differences (mean±std) among the model selection methods

(CID-LDAM). Table 4: Differences (mean±std) among the model selection methods. Lower is better. Datasets Clothing1M WebVision Food101 Global Maximum (%) 70.56±0.11 57.58 ±0.14 80.73 ±1.46 Label Wave (%) 70.12±0.34 57.24 ±0.34 80.12 ±1.01 Difference 0.44% 0.34% 0.61% C.3 L ABEL...

2015
[2020]

Leveraging unlabeled data to track memoriza- tion

10 Published as a conference paper at ICLR 2024 Mahsa Forouzesh, Hanie Sedghi, and Patrick Thiran. Leveraging unlabeled data to track memoriza- tion. In ICLR,

2024
[2021]

Progress measures for grokking via mechanistic interpretability

Neel Nanda, Lawrence Chan, Tom Liberum, Jess Smith, and Jacob Steinhardt. Progress measures for grokking via mechanistic interpretability. arXiv preprint arXiv:2301.05217,

arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [1]

learning confusing patterns

Specifically, in Point 2, the fluc- tuations in predictions transitioned from increasing to decreasing trends, eventually stabilizing at a specific value. Here, we empirically explain the evolution of fluctuations in predictions during the training process. Before reaching Point 1, as training progressed, the model’s generalization perfor- mance improved....

work page 2014

[2] [6]

Webvision database: Visual learning and understanding from web data

Wen Li, Limin Wang, Wei Li, Eirikur Agustsson, and Luc Van Gool. Webvision database: Visual learning and understanding from web data. arXiv preprint arXiv:1708.02862,

arXiv

[3] [7]

On the over-memorization during natural, robust and catastrophic overfitting

Runqi Lin, Chaojian Yu, Bo Han, and Tongliang Liu. On the over-memorization during natural, robust and catastrophic overfitting. arXiv preprint arXiv:2310.08847,

arXiv

[4] [8]

SELC: Self-Ensemble Label Correction Improves Learning with Noisy Labels

Sheng Liu, Zhihui Zhu, Qing Qu, and Chong You. Robust training under label noise by over- parameterization. In ICML, pp. 14153–14172, 2022a. Ziming Liu, Ouail Kitouni, Niklas S Nolte, Eric Michaud, Max Tegmark, and Mike Williams. To- wards understanding grokking: An effective theory of representation learning. NeurIPS, 2022b. Yangdi Lu and Wenbo He. Selc:...

work page Pith review arXiv

[5] [9]

Early stopping without a validation set

11 Published as a conference paper at ICLR 2024 Maren Mahsereci, Lukas Balles, Christoph Lassner, and Philipp Hennig. Early stopping without a validation set. arXiv preprint arXiv:1703.09580,

arXiv 2024

[6] [10]

Characterizing datapoints via second-split forgetting

Pratyush Maini, Saurabh Garg, Zachary Chase Lipton, and J Zico Kolter. Characterizing datapoints via second-split forgetting. In ICML 2022: Workshop on Spurious Correlations, Invariance and Stability. Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning . MIT press,

work page 2022

[7] [15]

Selfie: Refurbishing unclean samples for robust deep learning

Hwanjun Song, Minseok Kim, and Jae-Gil Lee. Selfie: Refurbishing unclean samples for robust deep learning. In ICML, 2019a. Hwanjun Song, Minseok Kim, Dongmin Park, and Jae-Gil Lee. How does early stopping help generalization against label noise? arXiv preprint arXiv:1911.08059, 2019b. Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. R...

arXiv 1911

[8] [16]

Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude

12 Published as a conference paper at ICLR 2024 Tijmen Tieleman, Geoffrey Hinton, et al. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine learning,

work page 2024