REVIEW 4 major objections 5 minor 16 references
Tracing the Path to Grokking: Embeddings, Dropout, and Network Activation
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A spike in test-accuracy variance under dropout foretells grokking in a modular addition network.
desk verdict The variance-peak forecasting claim is an arithmetic artifact of the p(1-p) envelope; the embedding observations are worth a look. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central objects are the Dropout Robustness Curve (DRC), obtained by evaluating test accuracy over a range of dropout rates at fixed checkpoints, and the variance of test accuracy over 100 stochastic forward passes with MC dropout at rate 0.3. These are paired with three complementary statistical probes: the cosine similarity between embedding vectors, the mean and standard deviation of embedding and weight distributions, and the percentage of inactive neurons after the ReLU layer. Together they characterize the network's internal state during the memorization-to-generalization transition and are claimed to predict grokking before the test accuracy visibly rises.
What would settle it
Train the same modular addition MLP with several different dropout rates (e.g., 0.1, 0.2, 0.5) and multiple random seeds, then check whether the variance-in-accuracy peak always appears immediately before the sustained rise in test accuracy and disappears after generalization; if the peak moves, repeats, or fails to appear in a regime where grokking still occurs, the precursor claim is falsified.
Extended reading notes
Core claim
The paper claims that a rise in the variance of the test accuracy resulting from Monte Carlo dropout is a precursor of grokking: as the model moves from memorization to generalization, the variance over stochastic forward passes first grows rapidly, then collapses as test accuracy saturates. It further claims that a Dropout Robustness Curve—test accuracy versus dropout rate at successive training checkpoints—shows the network becoming progressively more robust to structural noise once grokking occurs. The paper also argues that grokking is accompanied by the emergence of a symmetric, bimodal distribution of embedding weights with periodic cosine-similarity patterns, and that this structure is intrinsic to learning rather than an artifact of the activation function, since it persists when a ReLU layer is added or the initialization is changed. The author concludes that grokking results from the symmetries of the data, not from overtraining, and that these metrics can predict and quantify the phenomenon.
Load-bearing premise
The load-bearing premise is that the variance of test accuracy over 100 Monte Carlo dropout forward passes at a fixed dropout rate of 0.3 faithfully measures the network's transition from memorization to generalization, even though this probe is only tested on one architecture and one dropout rate.
Editorial extensions
If this is right
- If the variance spike under dropout is a reliable precursor, practitioners could monitor this variance during training to anticipate grokking without waiting for test accuracy to climb.
- The Dropout Robustness Curve provides a training-stage-dependent measure of how sensitive a network is to structural perturbations, which could be used to schedule dropout or regularization before grokking occurs.
- The observed bimodal embedding distribution and periodic cosine-similarity patterns suggest that grokking reflects the network discovering dataset symmetries, which may guide architecture choices that accelerate or suppress such structure formation.
- Weight and embedding distribution statistics and neuron sparsity offer additional, easy-to-compute signals that can be tracked in parallel to predict the onset of generalization.
- The near-linear dependence of grokking delay on initialization scale indicates that initialization provides a control knob for timing the memorization-to-generalization transition.
Reading between the lines
- A natural extension is to test whether the dropout-variance precursor and the DRC shape appear in transformer architectures and on non-algorithmic datasets, since the paper only demonstrates them on a two-layer MLP trained on modular addition.
- The causal reading—that embedding symmetries cause grokking rather than merely coincide with it—is an inference the paper draws from co-occurrence and from a frozen-embedding experiment; a direct test would manipulate the symmetries themselves and observe whether grokking is accelerated or suppressed.
- The variance peak under MC dropout might be usable as a general uncertainty signal: if it marks the moment a network moves from memorization to a more structured solution, it could inform early-stopping or model-selection in settings where grokking is not explicitly observed.
- The finding that bimodal embeddings appear regardless of initialization suggests a route toward understanding grokking as a symmetry-breaking phase transition, which could be probed with order parameters beyond cosine similarity, such as participation ratios or spectral gaps of the embedding covariance.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies grokking in a small MLP trained on modular addition. It proposes several diagnostics: the variance of test accuracy over MC-dropout forward passes, a Dropout Robustness Curve (DRC), cosine-similarity structure of embeddings, weight/embedding distribution statistics, and neuron sparsity. The central claim, stated in the conclusion, is that a rise in the variance of test accuracy under dropout is a precursor of grokking and that the proposed metrics can be used to predict and quantify grokking. The paper also makes a causal claim that grokking results from dataset symmetries rather than from overtraining. The results are presented as qualitative observations along what appears to be a single training trajectory, with figures showing correlations between the proposed metrics and the delayed rise in test accuracy.
Significance. If the proposed metrics genuinely forecast grokking before it occurs, they would be a useful contribution to the grokking literature, which currently lacks cheap, early, model-internal predictors. The paper also connects representation structure (bimodal embeddings, cosine-similarity bands) to generalization, and the initialization experiments are systematic and clearly described. However, the central forecasting claim is not supported as stated: the variance peak under dropout is likely an arithmetic artifact of the [0,1] accuracy scale, and no evaluation on held-out runs, seeds, or architectures is provided. The causal claim about symmetries is not established. The paper is therefore better viewed as a collection of correlational observations than as a validated set of forecasting metrics.
major comments (4)
- [Section 3.1 / Fig. 1] The headline result that the variance of test accuracy under MC dropout peaks before or during grokking is not supported as evidence of a precursor. For any accuracy statistic A taking values in [0,1] with mean p, the Bhatia–Davis inequality gives Var(A) ≤ p(1−p). Since the reported test accuracy moves from near 0 during memorization to near 1 after grokking, a variance peak at intermediate accuracy is mathematically guaranteed for almost any stochastic evaluation procedure, independent of any change in network sensitivity. To establish the claimed forecasting property, the authors would need to normalize the variance by p(1−p), compare against a null accuracy process with the same mean, or otherwise show that the peak is not simply tracking this envelope. No such analysis, and no repeated-seed error bars, are provided.
- [Section 3.3 / Fig. 5 and Conclusion] The statement that "grokking results from these symmetries rather than from overtraining" is a causal claim that is not supported by the evidence presented. The paper shows that bimodal embeddings co-occur with the rise in test accuracy, but co-occurrence does not establish causation. To make this claim, the authors would need an intervention that removes or changes the dataset symmetries while holding training otherwise fixed, or some other causal identification strategy. As written, the evidence is correlational and the causal wording should be removed or substantially weakened.
- [Abstract / Conclusion] The central claim that the proposed metrics "can be employed to predict and quantify grokking" is never evaluated as a forecasting claim. The paper does not define an operational forecasting rule, does not specify the feature values or thresholds that signal an imminent grokking transition, and does not test any rule on unseen training runs, seeds, or architectures. Observations that a metric changes before the test accuracy rise constitute a correlation, not a prediction. A concrete forecasting evaluation, with lead time and error bars over multiple seeds, is needed.
- [All figures / Section 3] All claims are based on what appears to be a single training trajectory. Grokking is known to be highly sensitive to initialization and seed, and the authors themselves demonstrate strong initialization effects in Section 3.3. Without repeated seeds, error bars, or at least a statement of how many runs were performed, the reported peaks, distribution shifts, and sparsity curves cannot be distinguished from seed-specific fluctuations. This is particularly problematic for the variance-under-dropout metric, where the variance of a single trajectory's accuracy estimate is reported without any measure of uncertainty.
minor comments (5)
- [Section 2] There is a typo: "modular additional model" should be "modular addition model", and in the abstract "drop out rate" should be "dropout rate".
- [Section 3.2 / Fig. 3] The text says "the heatmap of Fig. 1(a) is random", but the heatmap being discussed is in Fig. 3; the figure references should be corrected.
- [Section 3.4 / Fig. 11] The caption of Fig. 11 states "The training and test accuracy as a function of the percentage of inactive neurons", but the text does not clearly explain how sparsity is placed on the axis or whether the curve is parametric in epoch; this should be clarified.
- [References] References [3] and [12] are the same paper (Varma et al., "Explaining grokking through circuit efficiency") and should be merged or removed.
- [Section 3.3] The sentence "That the distributions contract during initial training can also be employed to predict grokking" overstates the evidence; no predictive rule or quantitative threshold is given, so this is an observation, not a demonstrated prediction method.
Circularity Check
No circular derivation; the metrics are empirical correlates and the only self-citation is background, not load-bearing.
full rationale
The paper does not define grokking in terms of its proposed metrics. Grokking is defined at the outset as delayed generalization, i.e., test accuracy rising appreciably after training accuracy, while the variance under dropout, Dropout Robustness Curve, embedding cosine similarity, and sparsity are measured as separate observable quantities. None of these is statistically fitted to the test-accuracy trajectory and then renamed a prediction; the dropout rate of 0.3 and the 100 Monte Carlo forward passes are fixed probes, not tuned parameters. The central variance claim is an empirical observation about a local maximum during training, and a failure to control for the p(1-p) bound on Bernoulli-like accuracy would be a validity concern, not a circularity, because the peak is not imposed by construction. The only self-citation is the authors' prior work [4] in the Introduction, where it supports a background statement about controlling grokking by modifying the activation profile; it is not used to derive or justify the current metrics. The causal interpretation in Section 3.3 that grokking 'results from these symmetries rather than from overtraining' is post hoc and would need stronger control experiments, but that is an inference-strength issue rather than circular reasoning. No equation in the paper reduces a claimed prediction to its own input, and no load-bearing argument depends on an unverified self-citation. Accordingly, the circularity score is low.
Assumptions & free parameters
free parameters (2)
- Threshold for 'substantial rise' in test accuracy =
not stated
- Dropout probe rate =
0.3 (Fig. 1); DRC sweeps 0.0 to 0.9
assumptions (4)
- domain assumption The modular addition MLP with hidden size 256, ReLU, AdamW, weight decay 1, learning rate 3e-4 exhibits canonical grokking behavior.
- ad hoc to paper Test-time MC dropout variance at a fixed dropout rate is a valid measure of network sensitivity or robustness.
- ad hoc to paper The observed co-occurrence of structured embeddings and test accuracy rise reflects a causal role for dataset symmetries.
- domain assumption Single-run observations are sufficient to conclude that embedding and weight distributions converge independently of initialization.
Cite this review
Pith. "Pith review of Tracing the Path to Grokking: Embeddings, Dropout, and Network Activation." pith.science (2026). https://pith.science/paper/6VMJ5FBC
@misc{pith2026250711645,
author = {Pith},
title = {Pith review of: Tracing the Path to Grokking: Embeddings, Dropout, and Network Activation},
year = {2026},
howpublished = {\url{https://pith.science/paper/6VMJ5FBC}},
note = {Machine review of arXiv:2507.11645}
}
read the original abstract
Grokking refers to delayed generalization in which the increase in test accuracy of a neural network occurs appreciably after the improvement in training accuracy This paper introduces several practical metrics including variance under dropout, robustness, embedding similarity, and sparsity measures, that can forecast grokking behavior. Specifically, the resilience of neural networks to noise during inference is estimated from a Dropout Robustness Curve (DRC) obtained from the variation of the accuracy with the dropout rate as the model transitions from memorization to generalization. The variance of the test accuracy under stochastic dropout across training checkpoints further exhibits a local maximum during the grokking. Additionally, the percentage of inactive neurons decreases during generalization, while the embeddings tend to a bimodal distribution independent of initialization that correlates with the observed cosine similarity patterns and dataset symmetries. These metrics additionally provide valuable insight into the origin and behaviour of grokking.
Reference graph
Works this paper leans on
-
[1]
Grokking: Generalization beyond overfitting on small algorithmic datasets
Alethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, and Vedant Misra. Grokking: Generalization beyond overfitting on small algorithmic datasets. arXiv preprint arXiv:2201.02177, 2022
arXiv 2022
-
[2]
Omnigrok: Grokking beyond algorithmic data
Ziming Liu, Eric J Michaud, and Max Tegmark. Omnigrok: Grokking beyond algorithmic data. arXiv preprint arXiv:2210.01117, 2022
arXiv 2022
-
[4]
Controlling Grokking with Nonlinearity and Data Symmetry
Ahmed Salah, and David Yevick. “Controlling Grokking with Nonlinearity and Data Symmetry.” arXiv preprint arXiv:2411.05353, 2024
work page Pith review arXiv 2024
-
[5]
Towards understanding grokking: An effective theory of representation learning
Ziming Liu, Ouail Kitouni, Niklas Nolte, Eric J Michaud, Max Tegmark, and Mike Williams. Towards understanding grokking: An effective theory of representation learning. arXiv preprint arXiv:2205.10343, 2022
arXiv 2022
-
[6]
Sentence-bert: Sentence embeddings using siamese bert - networks
Nils Reimers, and Iryna Gurevych. “Sentence-bert: Sentence embeddings using siamese bert - networks.” arXiv preprint arXiv:1908.10084, 2019
arXiv 1908
-
[7]
Learning the symmetric group: large from small
Max Petschack, Alexandr Garbali, and Jan de Gier. “Learning the symmetric group: large from small.” arXiv preprint arXiv:2502.12717, 2025
work page Pith review arXiv 2025
-
[8]
Deep Grokking: Would Deep Neural Networks Generalize Better?
Simin Fan, Razvan Pascanu, and Martin Jaggi. “Deep Grokking: Would Deep Neural Networks Generalize Better?.” arXiv preprint arXiv:2405.19454, 2024
arXiv 2024
-
[9]
Hidden progress in deep learning: Sgd learns parities near the computational limit
Boaz Barak, Benjamin Edelman, Surbhi Goel, Sham Kakade, Eran Malach, and Cyril Zhang. Hidden progress in deep learning: Sgd learns parities near the computational limit. Advances in Neural Information Processing Systems, 35:21750–21764, 2022
work page 2022
Show all 16 references
-
[10]
Progress measures for grokking via mechanistic interpretability
Neel Nanda, Lawrence Chan, Tom Liberum, Jess Smith, and Jacob Steinhardt. Progress measures for grokking via mechanistic interpretability. arXiv preprint arXiv:2301.05217, 2023
2023 arXiv
-
[11]
Grokking modular arithmetic
Andrey Gromov. Grokking modular arithmetic. arXiv preprint arXiv:2301.02679, 2023. 15
2023 arXiv
-
[12]
Explaining grokking through circuit efficiency
Vikrant Varma, Rohin Shah, Zachary Kenton, János Kramár, and Ramana Kumar. Explaining grokking through circuit efficiency. arXiv preprint arXiv:2309.02390, 2023
2023 arXiv
-
[13]
Progress Measures for Grokking on Real -world Tasks
Satvik Golechha, “Progress Measures for Grokking on Real -world Tasks.” arXiv preprint arXiv:2405.12755, 2024
2024 arXiv
-
[14]
A tale of two circuits: Grokking as competition of sparse and dense subnetworks
William Merrill, Nikolaos Tsilivis, and Aman Shukla. “A tale of two circuits: Grokking as competition of sparse and dense subnetworks.” arXiv preprint arXiv:2303.11873, 2023
2023 arXiv
-
[15]
To grok or not to grok: Disentangling generalization and memorization on corrupted algorithmic datasets
Doshi Darshil, Aritra Das, Tianyu He, and Andrey Gromov. “To grok or not to grok: Disentangling generalization and memorization on corrupted algorithmic datasets. ” arXiv preprint arXiv:2310.13061 , 2024
2024 arXiv
-
[16]
Dropout as a Bayesian approximation: Representing model uncertainty in deep learning
Yarin Gal and Zoubin Ghahramani. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pages 1050–1059, 2016
2016
-
[17]
Tikeng Notsawo, Hattie Zhou, Mohammad Pezeshki, Irina Rish, and Guillaume Dumas
Pascal Jr. Tikeng Notsawo, Hattie Zhou, Mohammad Pezeshki, Irina Rish, and Guillaume Dumas. Predicting grokking long before it happens: A look into the loss landscape of models which grok, 2023
2023
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.