Pith. sign in

REVIEW 4 major objections 4 minor 47 references

Grokking is a glass-like kinetic arrest in parameter space, and a swap-based Monte Carlo perturbation triggered by low fractal dimension reduces grokking time from about 3000 to about 650 epochs.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-04 19:57 UTC pith:2GHVDBTX

load-bearing objection Interesting diagnostic trio and a swap trick, but the glass-arrest claim is confounded by AdamW's weight decay and the paper's own limitations concede the FD estimator isn't ready to carry the load. the 4 major comments →

arxiv 2608.01833 v1 pith:2GHVDBTX submitted 2026-08-03 cond-mat.dis-nn cs.LG

Tunneling the Loss Landscape: Bypassing Memorization with Monte Carlo Parameter Swapping

Lai Shun Chan , Xiaotian Zhang , Yue Shang , Ge Zhang , Entao Yang This is my paper
classification cond-mat.dis-nn cs.LG
keywords grokkingglass dynamicskinetic arrestparameter mobilityreplica correlationfractal dimensionswap Monte Carlogeneralization delay
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Grokking is the strange training regime where a network memorizes perfectly for thousands of epochs and then abruptly generalizes. This paper argues that the delay is not merely slow gradient flow but a glass-like kinetic arrest: during memorization, parameter mobility collapses, replicas branched from the same state stay highly correlated in a history-dependent way, and the trajectory becomes straight and channel-like with fractal dimension near one. It introduces SAM-Swap, a swap Monte Carlo-inspired intervention that randomly exchanges parameter values within layers when the fractal dimension drops below a threshold, rejecting changes that raise loss above the initial training value. In the paper's experiments, this reduces grokking time from about 3000 to about 650 epochs. The three observables jointly diagnose whether slow dynamics are just small updates or true confinement in parameter space.

Core claim

The central claim is that standard optimization on a grokking task leaves the network in a kinetically arrested, glass-like regime during memorization. Evidence comes from three observables: parameter mobility drops by orders of magnitude; replica correlation shows two-step relaxation and strong waiting-time dependence, a signature of aging; and trajectory fractal dimension collapses toward one, indicating low-dimensional, ballistic, channel-like motion. These signatures reverse near generalization. The same kinetic-arrest picture motivates SAM-Swap, a structured random parameter-swap perturbation triggered when fractal dimension falls below an empirical threshold, which restores mobility, d

What carries the argument

SAM-Swap is a swap Monte Carlo-inspired intervention: within each transformer layer, random pairs of parameter values are exchanged, preserving the parameter distribution while rearranging the configuration; a loss cap rejects swaps that push training loss above the initial value. It is triggered when the layer-level trajectory fractal dimension falls below 1.1. The supporting diagnostics are parameter mobility (Euclidean displacement over a 20-epoch window), replica correlation (Pearson correlation between independently perturbed replicas branched at waiting time), and fractal dimension (exponent relating squared displacement to arc length, estimated by log-log regression over a sliding 20-

Load-bearing premise

The central claim depends on the local fractal dimension estimate—log-log regression over a fixed 20-epoch window and a selected lag range—being a faithful trigger for the arrested regime; the paper concedes that this estimate is window-dependent and sensitive to sampling range and density.

What would settle it

Train the same (x^2+y) mod 67 transformer with fractal dimension estimated over multiple window sizes (e.g., 20, 100, and 500 epochs). If the collapse to D_f near 1 during memorization disappears at larger windows, the channel-like arrest diagnosis is an artifact of window choice. Alternatively, if SAM-Swap triggered by FD below 1.1 fails to accelerate grokking on a different modular arithmetic task, the trigger is not general.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If grokking is kinetic arrest, interventions that restore parameter mobility and exploration can shorten generalization delay without changing architecture or data.
  • The three observables form a diagnostic: low mobility alone does not imply glassy confinement; it must coincide with high replica correlation and low fractal dimension.
  • Structured non-local parameter swaps can be as effective as large additive Gaussian noise in accelerating generalization, suggesting multiple routes to break the arrested regime.
  • The timing of generalization is trajectory-dependent: accessible generalizing solutions may exist in the landscape while the optimizer is dynamically unable to reach them.
  • Acceleration is consistently associated with random exploration in parameter space, connecting diffusion-like dynamics to entropy increase and generalization.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the kinetic-arrest picture is generic, similar state-triggered perturbations could be applied to non-algorithmic grokking tasks, but the paper only demonstrates the effect on modular arithmetic; testing on addition or multiplication mod different primes would clarify generality.
  • The claim that the FD threshold is not critical as long as it exceeds one suggests the arrest is a qualitative regime rather than a sharp boundary; a multiscale FD estimator could turn this into a practical order parameter for monitoring training.
  • SAM-Swap's loss-cap rejection is reminiscent of Metropolis acceptance; formalizing the swap rate and acceptance criterion as a Markov chain could connect the intervention to an effective temperature and predict optimal swap schedules.
  • The framework implies that memorization and generalization are not just different loss optima but different dynamical phases, so measuring replica correlation and fractal dimension during training could inform adaptive, state-dependent optimization strategies.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a three-observable framework—parameter mobility (PM), replica correlation (RC), and trajectory fractal dimension (FD)—to characterize grokking as a glass-like kinetic arrest in parameter space. Using full-batch AdamW on a one-layer transformer trained on (x^2+y) mod 67, the authors report that during the memorization phase PM collapses, RC shows two-step relaxation with aging, and FD decays to approximately 1, consistent with a channel-like, arrested trajectory. They then introduce State-Aware Monte Carlo Parameter Swapping (SAM-Swap), an FD-triggered intervention that randomly swaps parameters within layers, and report that it reduces grokking time from about 3000 to about 650 epochs. They compare this with weight decay and large Gaussian noise and conclude that accelerated generalization is associated with random exploration in parameter space.

Significance. If the central evidence were robust, the paper would provide a practically useful dynamical diagnostic for grokking and strengthen the analogy between grokking and glassy relaxation. The framework is intuitive, the paper is clearly written, and the authors are commendably candid about limitations in the Limitation section. However, the evidence is currently undermined by a plausible confound with AdamW's decoupled weight decay, the fragility of the FD estimator, the single-task/tuned-hyperparameter setting, and missing error bars. These issues are addressable, and a careful revision could make the paper a meaningful contribution, but as it stands the empirical support is not yet convincing.

major comments (4)
  1. [Observation (Fig. 1); Methods 'Small Additive Gaussian Noise as Probe'] All three glass signatures (PM collapse, RC plateau, FD≈1) are reported for AdamW with lr=1e-2, weight decay=1e-1. Once the training loss gradient becomes negligible after memorization, AdamW's decoupled weight decay dominates the update: θ_{t+1} ≈ (1 − ηλ)θ_t. This is a purely radial contraction toward the origin. Such a trajectory is a straight line in parameter space, so the FD estimate D_f = 2/α with ΔR² ∝ Δs^α gives α=2 and D_f≈1 by construction; the displacement decays as ηλ‖θ_t‖; and any two replicas that differ by small additive noise are multiplied by the same scalar, so their Pearson correlation remains essentially constant. The manuscript does not report PM/RC/FD curves for the Adam (no weight decay) baseline in Fig. 2, so this null model is not ruled out. Please add an Adam-without-weight-decay control, or subtract the radial-contraction component before computing the diagnos
  2. [Limitation; Algorithm 1; 'State-Aware Monte Carlo Parameter Swapping'] FD is simultaneously the evidence for glass arrest and the trigger for the SAM-Swap intervention, but the Limitation section states that the FD estimate is window-dependent, sensitive to sampling range and density, and 'requires multiscale refinement before it can serve as a robust order parameter.' The log-log regression lag range is unspecified, the window T=20 is chosen empirically, and FD values below 1 are dismissed as artifacts. The acceleration result depends on threshold df.glass=1.1 and swap ratio r=1e-2, both chosen empirically. Crucially, no control is shown with swaps applied at fixed intervals or random times unlinked to FD, so the 'state-aware' aspect of SAM-Swap is not demonstrated. Please specify the lag range, provide threshold and ratio sensitivity curves, include error bars, and add a non-triggered swap control to isolate the effect of state-awareness.
  3. [Conclusion and Future Directions; Fig. 2 caption] The broad claim that 'standard optimization presents clear signatures of glass dynamics' is supported by a single task (one-layer transformer on (x^2+y) mod 67), and the caption of Fig. 2 states that all hyperparameters are selected to minimize grokking time. The comparison of SAM-Swap, weight decay, and Gaussian noise is therefore a post-hoc tuned demonstration, not a predictive test, and the paper's own conclusion concedes it is unclear whether FD-triggered perturbations generalize beyond modular arithmetic. Please add at least one additional algorithmic grokking task with pre-registered hyperparameters, or show that the qualitative results are robust across a reasonable hyperparameter grid without task-specific retuning.
  4. [Figures 1 and 2] Error bars are omitted for the key PM and FD curves in Fig. 1b/d and Fig. 2b/d, and no confidence intervals are given for the reported grokking times (~3000 vs ~650 epochs). The claims of 'collapse', 'plateau', and 'aging' are based on visual inspection, and the two-step RC relaxation is not quantified (e.g., no plateau-length or decorrelation-time metric). Without uncertainty quantification, the statistical strength of the central empirical claims cannot be assessed, especially given the potential weight-decay confound. Please include error bars or confidence bands, and report quantitative summary statistics for the observed dynamical regimes.
minor comments (4)
  1. [Methods, 'Fractal Window Displacement'] The equation for dR has a formatting issue: it should read dR = (1/|l|) Σ_l ‖R_{t+T} − R_t‖_2. The notation R_t is used inconsistently (sometimes 'Rt'). Please clarify.
  2. [Methods, 'Fractal Dimension of the Training Trajectory'] The notation 'τ=1, ....[t/2]' is informal. Use τ ∈ {1, ..., ⌊t/2⌋} and, importantly, specify the actual lag range used for the log-log regression. This is needed for reproducibility of the FD values.
  3. [Algorithm 1] The pseudocode computes a per-layer fractal dimension d_f^ℓ and applies swaps per layer, while the text and Fig. 1 describe FD as 'average across layers.' Clarify whether swaps are triggered by layer-specific or averaged FD, and how the loss cap is evaluated after sequential layer-wise swaps.
  4. [References] Several references are dated 2026 (e.g., Ersoy and Wiesner 2026; Li et al. 2026; Pakman et al. 2026). The authors should verify publication status and ensure all citations conform to the journal's guidelines for forthcoming work.

Circularity Check

2 steps flagged

Partial circularity: SAM-Swap's acceleration is tuned to the reported grokking time, and the FD≈1 'channel-like' signature is a mathematical consequence of AdamW weight decay.

specific steps
  1. fitted input called prediction [State-Aware Monte Carlo Parameter Swapping (SAM-Swap) section; Figure 2 caption]
    "We use a threshold on Df to indicate entry into the constrained regime, selected empirically as df.glass = 1.1 ... We find that r= 10−2 yields the most stable training and fastest generalization ... All hyperparameters are selected to minimize grokking time."

    The SAM-Swap intervention is presented as a test of the kinetic-arrest theory, but its trigger threshold (df.glass), swap ratio (r), and the compared noise amplitudes are explicitly chosen to minimize grokking time on the same benchmark. The reported reduction from ~3000 to ~650 epochs is therefore the selection objective itself, not an independent prediction. The external test-accuracy benchmark prevents complete circularity, but the quantitative acceleration is by construction.

  2. other [Fractal Dimension of the Training Trajectory (Methods); Observation: Dynamical Transition in Trajectory Geometry; Figure 1 caption]
    "All data are from AdamW with learning rate 10−2 and weight decay 10−1 ... Values of Df ≈ 1 indicate that the trajectory evolves in a relatively straight or ballistic manner ... Once training accuracy reaches 99%, Df collapses toward 1, indicating that the optimization trajectory becomes effectively low-dimensional and ballistic-like."

    FD is defined by Df = 2/α from ΔR² ∝ Δs^α, so any straight trajectory has α = 2 and Df = 1 by construction. Under AdamW with decoupled weight decay, once the data-gradient term is negligible the update is θ_{t+1} = (1−ηλ)θ_t, a radial contraction whose increments are all collinear; the trajectory is a straight line. The observed 'channel-like' FD≈1 is therefore a mathematical consequence of the chosen optimizer plus vanishing gradients, not an independent glass signature. The paper does not separately report Adam (no weight-decay) FD/PM/RC curves to exclude this trivial null model.

full rationale

The paper's central dynamical measurements (PM, RC, FD) are real observations against an external test-accuracy benchmark, and the glass interpretation is not a pure derivation from its own definitions. However, two concrete reductions weaken the claimed novelty. First, the SAM-Swap acceleration is not an out-of-sample prediction: the trigger threshold, swap ratio, and noise amplitudes are explicitly tuned to minimize grokking time, and the minimized time is then reported as evidence for the theory. Second, the FD observable is defined so that straight trajectories give Df = 1, and the baseline AdamW optimizer, once gradients vanish, produces exactly a straight radial contraction under weight decay; thus the 'channel-like' arrest signature may be an artifact of the optimizer choice rather than independent evidence. The self-citation of the authors' prior glass-relaxation paper (Zhang et al. 2025) is background rather than a uniqueness theorem and is not the main circularity source. Overall, partial circularity: score 5.

Axiom & Free-Parameter Ledger

8 free parameters · 7 axioms · 0 invented entities

The paper's central claims rest on several fitted hyperparameters, notably the FD trigger threshold and swap ratio, and on unvalidated domain assumptions inherited from the authors' prior glass theory. No new physical entities are introduced beyond the interpretive label 'glass-like kinetic arrested state'; SAM-Swap is a method, not an entity.

free parameters (8)
  • FD estimation window T = 20 epochs
    Chosen empirically to resolve dynamics at the 10^1-10^2 epoch timescale; longer windows obscure the collapse into low-dimensional regimes (Limitation section).
  • FD glass threshold df.glass = 1.1
    Selected empirically to align with the observed onset of arrested dynamics.
  • Swap ratio r = 1e-2
    Chosen because it yields the most stable training and fastest generalization; smaller ratios recover baseline grokking and larger ratios induce excessive fluctuations.
  • Probe noise sigma = 1e-5
    Chosen from control experiments to show no change in macroscopic training performance while enabling meaningful RC and FD evaluation.
  • Intervention Gaussian noise sigma = 1e-2
    Large additive noise used as an intervention; selected as part of 'all hyperparameters selected to minimize grokking time' (Figure 2 caption).
  • Learning rate for AdamW baseline = 1e-2
    Selected to minimize grokking time (Figure 2 caption).
  • Weight decay for AdamW baseline = 1e-1
    Selected to minimize grokking time (Figure 2 caption).
  • Log-log regression lag range
    The paper says the exponent is estimated over a 'selected range of tau' but does not specify how this range is chosen or its width; an unspecified analytical freedom affecting the FD estimate.
axioms (7)
  • domain assumption Training loss acts as an effective energy of a statistical-mechanical system with Boltzmann entropy counting solution states
    Imported from the authors' prior glass-relaxation theory (Zhang et al. 2025) without independent verification; underpins the entire glass analogy.
  • domain assumption Full-batch Adam/AdamW without dropout eliminates intrinsic sources of randomness
    Invoked in Methods to control noise level and treat RC and FD as meaningful measures above a deterministic baseline.
  • domain assumption Pearson correlation between flattened parameter vectors is a scale-invariant measure of replica similarity
    Foundation of the replica-correlation function; not derived from first principles.
  • domain assumption The scaling relation ΔR^2 ∝ Δs^α holds over a selected lag range, and the resulting exponent yields a meaningful fractal dimension
    Central to FD estimation; the paper gives no theoretical justification and does not specify the lag-range selection criterion.
  • domain assumption The 1-layer transformer on (x^2+y) mod 67 is representative of grokking phenomena
    All experiments use this single setup; the authors admit it is unclear whether the results generalize beyond modular arithmetic.
  • domain assumption Swapping parameter values within a layer preserves the parameter distribution and mimics swap Monte Carlo particle swaps
    Justifies the SAM-Swap intervention as an analog of thermal swaps; no proof of equivalence in neural networks.
  • domain assumption Median aggregation of path-length and displacement statistics is robust to non-stationarity
    Used to define Δs(τ) and ΔR^2(τ) for FD estimation; a heuristic choice with no formal support.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Tunneling the Loss Landscape: Bypassing Memorization with Monte Carlo Parameter Swapping." pith.science (2026). https://pith.science/paper/2GHVDBTX

@misc{pith2026260801833,
  author       = {Pith},
  title        = {Pith review of: Tunneling the Loss Landscape: Bypassing Memorization with Monte Carlo Parameter Swapping},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2GHVDBTX}},
  note         = {Machine review of arXiv:2608.01833}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Grokking is a striking phenomenon in neural network training, where a model can undergo a prolonged period of pure memorization before abrupt generalization. While previous works have attempted to interpret it through classical machine learning mechanisms like weight norm, recent research draws an analogy from statistical physics, framing grokking as a form of computational glass relaxation. This theory defines the initial memorization as a result of `fast cooling' where the training loss is reduced so quickly that a glass state is formed, followed by a `slow relaxation' towards final generalization. Although providing a unifying framework for representative grokking theories, this perspective has remained largely at the theoretical on macroscopic level without direct empirical validation on training dynamics. Here we introduce a three-component framework to directly characterize the training dynamics via parameter mobility (PM), and two representative measurements from glassy dynamics: replica correlation (RC) and fractal dimension (FD). We demonstrate that standard optimization presents clear signatures of glass dynamics and inherently traps the grokking network in a kinetic arrested memorization state with a collapsed mobility, strong history dependence, and channel-like motions. This quantitative agreement motivates us to introduce State-Aware Monte Carlo Parameter Swapping (SAM-Swap), an optimization plug-in that can accelerate generalization, inspired by swap Monte Carlo algorithm widely used in glass dynamics. Comparing SAM-Swap, weight decay, and Gaussian gradient noise, we find that accelerated generalization is consistently associated with random exploration in the parameter space, similar to diffusion in physics.

Figures

Figures reproduced from arXiv: 2608.01833 by Entao Yang, Ge Zhang, Lai Shun Chan, Xiaotian Zhang, Yue Shang.

Figure 1
Figure 1. Figure 1: Comparison across waiting times. All data are from AdamW with learning rate 10−2 and weight decay 10−1 , optimized for shortest grokking time. Baseline uses AdamW; probed model adds small Gaussian noise to test trajectory sensitivity. The x-axis is aligned to t + tw. (a) Test accuracy (solid lines show Test accuracy; translucent line shows training accuracy).(b) Parameter mobility, error bars were omitted … view at source ↗
Figure 2
Figure 2. Figure 2: Comparison across perturbation methods: All models uses learning rate of 10−2 , while AdamW addi￾tionally applies weight decay of 10−1 . Both models are probed using a weak Gaussian probe (σ = 10−5 ), whereas Adam+Noise employs additive Gaussian perturbations with σ = 10−2 and Adam+Swap uses the proposed SAM-Swap. All hyperparameters are selected to minimize grokking time. (a) Test accuracy (solid lines sh… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

47 extracted references

  1. [1]

    2016 , eprint =

    Zhang, Chiyuan and Bengio, Samy and Hardt, Moritz and Recht, Benjamin and Vinyals, Oriol , title =. 2016 , eprint =

    2016

  2. [2]

    2022 , eprint =

    Git Re-Basin: Merging Models Modulo Permutation Symmetries , author =. 2022 , eprint =

    2022

  3. [3]

    Proceedings of the National Academy of Sciences , volume =

    Reconciling modern machine-learning practice and the classical bias--variance trade-off , author =. Proceedings of the National Academy of Sciences , volume =. 2019 , publisher =

    2019

  4. [4]

    International Conference on Learning Representations (ICLR) , year =

    Chiang, Ping-Yeh and Ni, Renkun and Miller, David Yu and Bansal, Arpit and Geiping, Jonas and Goldblum, Micah and Goldstein, Tom , title =. International Conference on Learning Representations (ICLR) , year =

  5. [5]

    International Conference on Machine Learning , pages=

    Bias of Stochastic Gradient Descent or the Architecture: Disentangling the Effects of Overparameterization of Neural Networks , author=. International Conference on Machine Learning , pages=. 2024 , organization=

    2024

  6. [6]

    2026 , eprint =

    Pakman, Ari and Kreimer, Lior and Berchenko, Yakir , title =. 2026 , eprint =

    2026

  7. [7]

    2024 , eprint =

    Lee, Jaerin and Kang, Bong Gyun and Kim, Kihoon and Lee, Kyoung Mu , title =. 2024 , eprint =

    2024

  8. [8]

    International Conference on Learning Representations , year =

    Prieto, Lucas and Barsbey, Melih and Mediano, Pedro and Birdal, Tolga , title =. International Conference on Learning Representations , year =

  9. [9]

    2026 , eprint =

    Ersoy, Ibrahim Talha and Wiesner, Karoline , title =. 2026 , eprint =

    2026

  10. [10]

    2022 , eprint =

    Power, Alethea and Burda, Yuri and Edwards, Harri and Babuschkin, Igor and Misra, Vedant , title =. 2022 , eprint =

    2022

  11. [11]

    Proceedings of the 41st International Conference on Machine Learning , series=

    Deep networks always grok and here is why , author=. Proceedings of the 41st International Conference on Machine Learning , series=. 2024 , publisher=

    2024

  12. [12]

    Advances in Neural Information Processing Systems , volume=

    Towards understanding grokking: an effective theory of representation learning , author=. Advances in Neural Information Processing Systems , volume=

  13. [13]

    Proceedings of the International Conference on Learning Representations , year=

    Omnigrok: grokking beyond algorithmic data , author=. Proceedings of the International Conference on Learning Representations , year=

  14. [14]

    Physical Review Research , volume=

    Glassy dynamics in deep neural networks: a structural comparison , author=. Physical Review Research , volume=. 2025 , publisher=

    2025

  15. [15]

    Advances in Neural Information Processing Systems , volume=

    Is grokking a computational glass relaxation? , author=. Advances in Neural Information Processing Systems , volume=

  16. [16]

    Proceedings of the International Conference on Artificial Intelligence and Statistics , pages=

    The loss surfaces of multilayer networks , author=. Proceedings of the International Conference on Artificial Intelligence and Statistics , pages=. 2015 , publisher=

    2015

  17. [17]

    Physical Chemistry Chemical Physics , volume=

    Energy landscapes for machine learning , author=. Physical Chemistry Chemical Physics , volume=. 2017 , publisher=

    2017

  18. [18]

    Proceedings of the 35th International Conference on Machine Learning , volume=

    Comparing dynamics: deep neural networks versus glassy systems , author=. Proceedings of the 35th International Conference on Machine Learning , volume=. 2018 , publisher=

    2018

  19. [19]

    Physical Review E , volume=

    Jamming transition as a paradigm to understand the loss landscape of deep neural networks , author=. Physical Review E , volume=. 2019 , publisher=

    2019

  20. [20]

    SciPost Physics Core , volume=

    From complex to simple: hierarchical free-energy landscape renormalized in deep neural networks , author=. SciPost Physics Core , volume=

  21. [21]

    Physical Review Research , volume=

    Spatially heterogeneous learning by a deep student machine , author=. Physical Review Research , volume=. 2023 , publisher=

    2023

  22. [22]

    Entropy , volume=

    Spin glasses in a field show a phase transition varying the distance among real replicas , author=. Entropy , volume=

  23. [23]

    Axioms , volume=

    Pole analysis of the inter-replica correlation function in a two-replica system , author=. Axioms , volume=. 2024 , publisher=

    2024

  24. [24]

    Science Advances , volume =

    Universal activated aging and weak ergodicity breaking in spin and structural glasses , author =. Science Advances , volume =. 2026 , publisher =

    2026

  25. [25]

    Testing mode-coupling theory for a supercooled binary Lennard-Jones mixture. II. Intermediate scattering function and dynamic susceptibility , author=. Physical Review E , volume=. 1995 , publisher=

    1995

  26. [26]

    Physical review letters , volume=

    Aging effects in a Lennard-Jones glass , author=. Physical review letters , volume=. 1997 , publisher=

    1997

  27. [27]

    Proceedings of the National Academy of Sciences , year=

    Slow relaxation and landscape-driven dynamics in viscous ripening foams , author=. Proceedings of the National Academy of Sciences , year=

  28. [28]

    Physical Review E , volume =

    Fast Monte Carlo algorithm for supercooled soft spheres , author =. Physical Review E , volume =. 2001 , publisher =

    2001

  29. [29]

    Physical Review Letters , volume =

    Efficient Equilibration of Hard Spheres up to the Jamming Density and Beyond with Swap Monte Carlo , author =. Physical Review Letters , volume =. 2016 , publisher =

    2016

  30. [30]

    SN Computer Science , volume=

    Grokking in neural networks: a review , author=. SN Computer Science , volume=. 2025 , publisher=

    2025

  31. [31]

    npj Artificial Intelligence , volume=

    High-entropy advantage in neural networks' generalizability , author=. npj Artificial Intelligence , volume=. 2026 , publisher=

    2026

  32. [32]

    2023 , eprint =

    Grokking as the transition from lazy to rich training dynamics , author =. 2023 , eprint =

    2023

  33. [33]

    Proceedings of the International Conference on Learning Representations , year =

    Progress Measures for Grokking via Mechanistic Interpretability , author =. Proceedings of the International Conference on Learning Representations , year =

  34. [34]

    2023 , eprint =

    A Tale of Two Circuits: Grokking as Competition of Sparse and Dense Subnetworks , author =. 2023 , eprint =

    2023

  35. [35]

    2023 , eprint =

    Explaining Grokking through Circuit Efficiency , author =. 2023 , eprint =

    2023

  36. [36]

    NeurIPS 2022 Workshops: Human Interpretability of Machine Learning Systems (HITY) , year =

    The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenon , author =. NeurIPS 2022 Workshops: Human Interpretability of Machine Learning Systems (HITY) , year =

    2022

  37. [37]

    Advances in Neural Information Processing Systems , year =

    Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization , author =. Advances in Neural Information Processing Systems , year =

  38. [38]

    Journal of High School Science , volume =

    Investigating the Impact of Low to Moderate Level Label Noise on Neural Network Grokking Dynamics , author =. Journal of High School Science , volume =. 2025 , doi =

    2025

  39. [39]

    Proceedings of the International Conference on Learning Representations (ICLR) , year =

    On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima , author =. Proceedings of the International Conference on Learning Representations (ICLR) , year =

  40. [40]

    Proceedings of the International Conference on Learning Representations (ICLR) , year =

    Entropy-SGD: Biasing Gradient Descent Into Wide Valleys , author =. Proceedings of the International Conference on Learning Representations (ICLR) , year =

  41. [41]

    Proceedings of the ICML Workshop on Principled Approaches to Deep Learning , year =

    Empirical Analysis of the Hessian of Over-Parametrized Neural Networks , author =. Proceedings of the ICML Workshop on Principled Approaches to Deep Learning , year =

  42. [42]

    Journal of Machine Learning Research , volume =

    Group Invariance, Stability to Deformations, and Complexity of Deep Convolutional Representations , author =. Journal of Machine Learning Research , volume =. 2019 , publisher =

    2019

  43. [43]

    Advances in Neural Information Processing Systems 29 (NeurIPS) , pages =

    End-to-End Kernel Learning with Supervised Convolutional Kernel Networks , author =. Advances in Neural Information Processing Systems 29 (NeurIPS) , pages =. 2016 , url =

    2016

  44. [44]

    Reviews of Modern Physics , volume =

    Theoretical perspective on the glass transition and amorphous materials , author =. Reviews of Modern Physics , volume =. 2011 , publisher =

    2011

  45. [45]

    Reviews of Modern Physics , volume =

    The glass transition and elastic models of glass-forming liquids , author =. Reviews of Modern Physics , volume =. 2006 , publisher =

    2006

  46. [46]

    Structural Glasses and Supercooled Liquids: Theory, Experiment, and Applications , editor =

    Mézard, Marc and Parisi, Giorgio , title =. Structural Glasses and Supercooled Liquids: Theory, Experiment, and Applications , editor =. 2012 , chapter =

    2012

  47. [47]

    Chaos: An Interdisciplinary Journal of Nonlinear Science , volume =

    Estimating fractal dimensions: A comparative review and open source implementations , author =. Chaos: An Interdisciplinary Journal of Nonlinear Science , volume =. 2023 , publisher =

    2023

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.