Grokking Explained: A Statistical Phenomenon

Artur S. d'Avila Garcez; Breno W. Carvalho; Em\'ilio Vital Brazil; Lu\'is C. Lamb

REVIEW 4 major objections 5 minor 14 references

Grokking Explained: A Statistical Phenomenon

T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that grokking—delayed generalization—is fundamentally a statistical consequence of distribution shift between training and test data, not of data sparsity or regularization, and that it can be systematically induced by…

desk verdict Useful synthetic benchmarks and a plausible hypothesis, but the causal claim about distribution shift is confounded by partial subclass coverage and the paper's own weight-decay explanation. read the letter →

arxiv 2502.01774 v1 pith:HQJE2HMG submitted 2025-02-03 cs.LG cs.AI

Breno W. Carvalho , Artur S. d'Avila Garcez , Luís C. Lamb , Emílio Vital Brazil This is my paper · ORCID

classification cs.LGcs.AI

keywords grokkingdelayedgeneralizationdistributionshiftclasshierarchysubclasssamplingsyntheticdatasetequivariantmaplate

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Grokking is the phenomenon where a model's test performance suddenly jumps long after training loss has converged. This paper argues that the underlying cause is a distribution shift between training and test data, and that data sparsity and regularization merely amplify this shift rather than cause it. To support this, the authors build two synthetic datasets with controlled class–subclass structure and induce grokking by under-sampling specific subclasses, even with thousands of samples and minimal hyperparameter tuning. They also reproduce the effect on MNIST by clustering digit representations, and show the behavior appears in both MLPs and transformers. The payoff is a systematic, testable account of late generalization and a basis for better stopping criteria in training.

What carries the argument

The central machinery is a pair of synthetic datasets built from multivariate normal subclasses organized into classes. In the equidistant dataset, all subclass centroids are equally far apart; in the equivariant dataset, subclasses within a class are closer to each other than to subclasses of other classes. A sampling fraction $f \in [0,1]$ controls how many examples are kept from selected subclasses, creating a controlled distribution shift. These datasets let the authors isolate whether grokking tracks sample size, regularization, or the shift itself; the class–subclass geometry also lets them test whether relational proximity among subclasses can substitute for missing samples. A formal Definition 3.1 of grokking, based on the transition interval between memorization and generalization, provides the measurement used throughout.

What would settle it

Train the same architecture on the equidistant dataset with $f=0$ (subclass absent) while sweeping weight decay and initialization scale: if grokking appears under some hyperparameters, the effect is not due to the shift alone. Alternatively, compare $f=0.01$ imbalanced sampling with a balanced sample of the same size; if balanced sampling also groks, the distribution shift is not the determining factor.

Watch

Extended reading notes

Core claim

The paper's central claim is that grokking is a statistical phenomenon: it occurs when the training distribution is shifted away from the test distribution, specifically when some subclasses of a class hierarchy are under-represented. The authors formalize grokking with a definition based on inflection points $\alpha_0$ and $\alpha_1$ and the gap $\Delta S$ between low- and high-performing test regions. Using synthetic data where classes are unions of equidistant or equivariant subclasses, they show that a faint training signal from a down-sampled subclass—even 1% of its samples—can trigger late generalization, while removing the subclass entirely ($f=0$) blocks it in the equidistant setting. In the equivariant setting, where subclasses of a class are close to each other, grokking occurs even when a subclass is completely absent, because nearby subclasses leak information. They conclude that small sample sizes are only a convenient route to the necessary distribution shift, not the cause, and that grokking can be induced with dense data and minimal tuning.

Load-bearing premise

The load-bearing premise is that under-sampling a subclass counts as a distribution shift and that the delayed generalization is caused by that shift, rather than by simply having a few examples of each subclass; the paper's own result that $f=0$ blocks grokking while $f=0.01$ triggers it makes this distinction the crux, and the fixed weight decay and initialization scale are not ablated.

Editorial extensions

If this is right

Grokking can be induced systematically in class-hierarchical data by under-sampling subclasses, without extreme sparsity or heavy regularization.
A very small number of examples from an under-represented subclass ($f=0.01$) is enough to produce late generalization, while zero examples blocks it in equidistant structure.
When subclasses are close within a class, grokking can occur even with a subclass completely absent, showing that relational structure in the data carries part of the training signal.
The phenomenon is architecture-independent: similar late-generalization curves appear for MLPs and transformers.
MNIST experiments with clustered latent representations show induced distribution shifts can reproduce grokking outside synthetic data, supporting new stopping criteria based on the likelihood of late generalization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If shift magnitude rather than mere presence of samples were the driver, grokking probability should vary monotonically with $f$; the $f=0$ versus $f=0.01$ discontinuity suggests a threshold experiment worth running: sweep $f$ finely and check whether grokking appears abruptly or gradually.
The equivariant result implies that datasets with hierarchical or geometric label structure may be prone to grokking under mild imbalance; an untested extension is that curriculum or active-learning methods that deliberately expose a few examples from each subclass could trigger or suppress grokking on demand.
A practical consequence the paper leaves implicit: train–test distribution divergence measured during training (e.g., by two-sample tests on hidden representations) could serve as an early-warning signal for imminent grokking, enabling adaptive stopping rather than waiting for the late jump.
The definition of grokking via $\Delta S$ and the support interval $S$ could be turned into an automated detector, though the paper does not propose thresholds for these quantities.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper's real contribution is the two synthetic datasets, which let you induce grokking on demand by undersampling subclasses. The equivariant dataset result—grokking even when a subclass is entirely absent—is genuinely interesting and not in the prior literature. If you work on grokking, these benchmarks are worth having.

The central hypothesis, that grokking is caused by a distribution shift between train and test, is not established by the experiments. The most direct problem is internal: in the equidistant dataset, removing a subclass entirely (f=0) prevents grokking, while keeping 1% of it (f=0.01) triggers late generalization. If bigger shift meant more grokking, f=0 should be the strongest case. The pattern is equally explained by the presence of a weak training signal from a few labeled examples. The paper never measures the shift (e.g., KL or total variation between train and test subclass proportions), so "distribution shift" is an inferred label for a confounded manipulation.

The paper also undercuts its own "regularization is merely an amplifier" claim. In Section 5.2 it attributes the late improvement to weight decay leading to a sparse internal representation. With fixed weight decay (1e-4) and initialization scale 8—both values known to promote grokking—and no ablations, the separate roles of shift, sparsity, and regularization remain tangled.

Completeness is an issue: the intro and contribution list promise MNIST and transformer experiments, but neither appears in the main text or appendix provided. The appendix only contains the dataset construction procedure. Error bars are absent despite 10 runs; that is minor but easy to fix.

On the positive side, the definition attempt is reasonable, the dataset construction is clearly described, code is provided, and the limitations section is honest. The citation of Levi et al. is appropriate; this paper extends their insight to class–subclass structure.

I would send this to peer review rather than desk-reject, because the benchmarks are useful and the hypothesis is testable. But a serious referee should push for a direct quantification of distribution shift, an ablation of weight decay and initialization, and the missing experiments. As it stands, the explanation is overclaimed.

Referee Report

4 major / 5 minor

Summary. The paper argues that grokking (delayed generalization) is a statistical phenomenon caused by a distribution shift between training and test data, with data sparsity and regularization acting only as amplifiers rather than causes. The authors define grokking informally, introduce two synthetic datasets with class-subclass hierarchies, and induce distribution shifts by downsampling subclasses by a fraction f. Experiments with MLPs and a two-layer transformer show grokking across several settings, including dense datasets, and the authors claim that grokking can be reproduced systematically with minimal hyperparameter tuning. The paper also announces MNIST experiments validating the findings beyond synthetic data, and provides code and datasets for replication.

Significance. If the central hypothesis were convincingly established, the paper would offer a unifying statistical explanation of grokking and provide useful controlled benchmarks for studying delayed generalization. The synthetic dataset construction and the systematic f-sweep are valuable contributions, and the release of code and data is a strength. However, the causal claim is currently under-supported: the operationalization of 'distribution shift' is confounded with partial subclass coverage, the shift itself is never quantified, and the fixed grokking-promoting hyperparameters are not ablated. The reported MNIST validation is missing from the manuscript. These issues currently prevent the paper from supporting its main claim, though the underlying hypothesis remains plausible and testable.

major comments (4)

[§5.2, Figure 3b] The non-monotonic relationship between f and grokking undercuts the causal attribution to distribution shift magnitude. Removing a subclass entirely (f = 0) prevents grokking, while retaining only 1-20% of that subclass (f = 0.01-0.2) induces it. If a larger distribution shift caused more grokking, f = 0 should grok the most; instead, the pattern is equally or better explained by whether any correctly labeled examples of each subclass remain (partial coverage / weak training signal). The manuscript should quantify the induced shift (e.g., KL divergence or total variation between training and test subclass proportions) and test monotonicity, or reframe the claim as requiring partial subclass coverage rather than shift magnitude.
[§5.2, Figure 3a] Balanced downsampling of the equidistant dataset, with no subclass forced to be sampled less than another, still produces grokking at small sample sizes. If the test set is the full balanced 10,000-sample set, the small training set has the same expected subclass proportions but a different finite-sample realization; whether this constitutes 'distribution shift' is not established. Without a direct measure of the shift, small sample size itself remains a viable candidate cause, which conflicts with the paper's claim that sparsity is merely a mechanism for achieving distribution shift.
[§5.1 and §5.2] The experiments fix weight decay at 1e-4 and initialization scale at 8, both known grokking-promoting choices from Liu et al. (2022), yet no ablation is reported. Moreover, §5.2 states that 'weight decay led the network to a more sparse internal representation,' which attributes the late improvement to regularization. This is difficult to reconcile with the claim that regularization merely amplifies and does not cause grokking. The authors should either ablate these hyperparameters or clarify the precise role of regularization in their account.
[Introduction and Abstract] The abstract and introduction claim that MNIST experiments validate the findings beyond synthetic data, but no MNIST results appear in the main text or in Appendix A. This claim is currently unsupported and must either be substantiated with results or removed from the paper.

minor comments (5)

[Figure 3a caption and §5.2] The caption says 'It occurs in every case,' while the text says grokking 'only existing in considerably small sample sizes.' These statements are contradictory and should be reconciled.
[Appendix A.2] The inequality |C(Ci,j) - C(Ci,l)| > |C(Ci,j) - C(Ck,l)| for i≠k appears to contradict the prose that subclasses of the same class are closer than subclasses of different classes; if the intended property is the opposite, the inequality should use '<'.
[§5, experimental setup] The text states results are aggregated from 10 independent runs, but the figures show only average curves without error bars or confidence bands; given that grokking detection is sensitive to training dynamics, variance information should be included.
[§3, Definition 3.1] Definition 3.1 relies on informal thresholds and symbols such as α0, α1, and 'S ≫ (α1 - α0)' without operational criteria; please specify how these quantities are estimated from training curves in practice.
[§2 and References] There are minor typographical issues, including 'liner models' instead of 'linear models,' and the reference to Quiñonero-Candela et al. (2022) should cite the original 2008 work on dataset shift.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the distribution-shift account is an empirical interpretation of controlled sampling experiments, not a definitional reduction or a fitted prediction.

full rationale

The paper's central claim (Section 3) is a hypothesis, not a derivation: grokking is defined operationally in Definition 3.1 purely in terms of training and test curve geometry, with no reference to distribution shift. The experiments then manipulate subclass sampling fraction f (Section 4, Eq. 1) and report measured test curves; the presence of grokking is read off the curves, while the shift is an intervention, never a fitted parameter used to produce those curves. No equation or definition makes the occurrence of grokking equivalent to the value of f, and no parameter is fit to a subset of curves and then 'predicted' on a closely related quantity. The self-citation (d'Avila Garcez et al., 2009) appears only as a contextual pointer to neuro-symbolic approaches and is not load-bearing; the grokking-promoting hyperparameters (weight decay 1e-4, initialization scale 8) are adopted from external work by Liu et al. (2022), not from a self-citation chain. The Limitations section concedes that the paper cannot predict when late generalization will occur, which further shows the claim is not a closed tautology. Concerns that the causal attribution is confounded (e.g., Figure 3b's non-monotonic f=0 result and the absence of a direct shift measurement) are underdetermination or validity objections, not circular reductions; they do not raise the circularity score.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper rests on an unmeasured causal variable (distribution shift), a descriptive definition of grokking, and hyperparameters that are fixed rather than swept. It also relies on a particular synthetic geometry that is asserted to represent relational structure. No new entities are postulated.

free parameters (5)

downsampling fraction f = grid: 0, 0.01, 0.05, 0.2
Hand-chosen values control how much each subclass is undersampled; the qualitative outcome (grokking vs no grokking) depends on these values, and f=0 vs f=0.01 flips the result.
weight decay = 1e-4
Fixed in all experiments; the paper attributes late generalization partly to weight decay producing sparse representations but does not vary it or measure sparsity.
initialization scale factor = 8
Weights are scaled by 8 following Liu et al.; this is a known grokking-amplifying choice, so 'minimal hyper-parameter tuning' is not fully supported.
learning rate = 1e-3
Fixed across experiments; no sensitivity analysis is reported.
number of binary positions p and dimension r = p=9, r=13
Dataset construction choices that determine the geometry (512 classes, dimension 117); results may depend on this geometry.

assumptions (5)

ad hoc to paper Definition 3.1 captures grokking
The paper asserts a formal definition of grokking as late test-set improvement after training convergence, but the thresholds (sigma_S, sigma_E, alpha_0, alpha_1, S) are not operationalized or measured automatically.
domain assumption Train/test distribution shift is the active causal variable
The hypothesis in Section 3 is treated as the explanation; experiments vary subclass sampling but do not measure distribution shift directly.
domain assumption Subclass proximity in the equivariant dataset provides information leakage
The paper attributes grokking without subclass examples to similarity and proximity between subclasses, but no quantitative measure of equivariance or information flow is provided.
domain assumption Weight decay leads to sparse representations that enable late generalization
Invoked in Section 5.2 to explain why a tiny sample fraction only helps after convergence; sparsity is not measured.
domain assumption Linearly separable Gaussian subclasses are representative of real data structure
The synthetic datasets use multivariate normal subclasses; whether this captures real grokking settings is assumed.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Grokking Explained: A Statistical Phenomenon." pith.science (2026). https://pith.science/paper/HQJE2HMG

@misc{pith2026250201774,
  author       = {Pith},
  title        = {Pith review of: Grokking Explained: A Statistical Phenomenon},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HQJE2HMG}},
  note         = {Machine review of arXiv:2502.01774}
}

read the original abstract

Grokking, or delayed generalization, is an intriguing learning phenomenon where test set loss decreases sharply only after a model's training set loss has converged. This challenges conventional understanding of the training dynamics in deep learning networks. In this paper, we formalize and investigate grokking, highlighting that a key factor in its emergence is a distribution shift between training and test data. We introduce two synthetic datasets specifically designed to analyze grokking. One dataset examines the impact of limited sampling, and the other investigates transfer learning's role in grokking. By inducing distribution shifts through controlled imbalanced sampling of sub-categories, we systematically reproduce the phenomenon, demonstrating that while small-sampling is strongly associated with grokking, it is not its cause. Instead, small-sampling serves as a convenient mechanism for achieving the necessary distribution shift. We also show that when classes form an equivariant map, grokking can be explained by the model's ability to learn from similar classes or sub-categories. Unlike earlier work suggesting that grokking primarily arises from high regularization and sparse data, we demonstrate that it can also occur with dense data and minimal hyper-parameter tuning. Our findings deepen the understanding of grokking and pave the way for developing better stopping criteria in future training processes.

Figures

Figures reproduced from arXiv: 2502.01774 by the authors.

**Figure 1.** Schematic representation of the two synthetic datasets used in this study. The equidistant subclass dataset (a) and [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

**Figure 2.** Grokking example: assume that the training set [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Average test set curves after the training loss has converged to a very high training set accuracy in the equidistant [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Investigation of grokking in the equivariant dataset [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 7 canonical work pages

[1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
[2]

M., Bruna, J., Cohen, T., and Velickovic, P

Bronstein, M. M., Bruna, J., Cohen, T., and Velickovic, P. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. CoRR, abs/2104.13478, 2021. URL https://arxiv.org/abs/2104.13478

arXiv 2021
[3]

S., Lamb, L

d'Avila Garcez, A. S., Lamb, L. C., and Gabbay, D. M. Neural-Symbolic Cognitive Reasoning. Cognitive Technologies. Springer, 2009. ISBN 978-3-540-73245-7. URL http://dblp.uni-trier.de/db/series/cogtech/index.html#GarcezLG2009

work page 2009
[4]

J., and Pehlevan, C

Kumar, T., Bordelon, B., Gershman, S. J., and Pehlevan, C. Grokking as the transition from lazy to rich training dynamics. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=vt5mnLVIVo

work page 2024
[5]

Grokking in linear estimators - A solvable model that groks without understanding

Levi, N., Beck, A., and Bar - Sinai, Y. Grokking in linear estimators - A solvable model that groks without understanding. CoRR, abs/2310.16441, 2023. doi:10.48550/ARXIV.2310.16441. URL https://doi.org/10.48550/arXiv.2310.16441

arXiv doi:10.48550/arxiv.2310.16441 2023
[6]

Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll \'a r, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp.\ 740--755. Springer, 2014

2014
[7]

S., Michaud, E., Tegmark, M., and Williams, M

Liu, Z., Kitouni, O., Nolte, N. S., Michaud, E., Tegmark, M., and Williams, M. Towards understanding grokking: An effective theory of representation learning. Advances in Neural Information Processing Systems, 35: 0 34651--34663, 2022

2022
[8]

Progress measures for grokking via mechanistic interpretability

Nanda, N., Chan, L., Lieberum, T., Smith, J., and Steinhardt, J. Progress measures for grokking via mechanistic interpretability. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.net, 2023. URL https://openreview.net/pdf?id=9XFSbDPmdW

work page 2023

Show all 14 references

[9]

Grokking: Generalization beyond overfitting on small algorithmic datasets

Power, A., Burda, Y., Edwards, H., Babuschkin, I., and Misra, V. Grokking: Generalization beyond overfitting on small algorithmic datasets. CoRR, abs/2201.02177, 2022. URL https://arxiv.org/abs/2201.02177

2022 arXiv
[10]

Qui \ n onero-Candela, J., Sugiyama, M., Schwaighofer, A., and Lawrence, N. D. Dataset shift in machine learning. MIT Press, 2022

2022
[11]

Failing loudly: An empirical study of methods for detecting dataset shift

Rabanser, S., G\" u nnemann, S., and Lipton, Z. Failing loudly: An empirical study of methods for detecting dataset shift. In Wallach, H., Larochelle, H., Beygelzimer, A., d Alch\' e -Buc, F., Fox, E., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, ...

2019
[12]

Are emergent abilities of large language models a mirage? In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S

Schaeffer, R., Miranda, B., and Koyejo, S. Are emergent abilities of large language models a mirage? In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems, volume 36, pp.\ 55565--55581. Curran Ass...

2023
[13]

S., and Kersting, K

Seng, J., Ze c evi \'c , M., Dhami, D. S., and Kersting, K. Learning large DAG s is harder than you think: Many losses are minimal for the wrong DAG . In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=gwbQ2YwLhD

2024
[14]

and Ilievski, E

Z unkovi c , B. and Ilievski, E. Grokking phase transitions in learning local rules with gradient descent. arXiv preprint arXiv:2210.15435, 2022

2022 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

[2] [2]

M., Bruna, J., Cohen, T., and Velickovic, P

Bronstein, M. M., Bruna, J., Cohen, T., and Velickovic, P. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. CoRR, abs/2104.13478, 2021. URL https://arxiv.org/abs/2104.13478

arXiv 2021

[3] [3]

S., Lamb, L

d'Avila Garcez, A. S., Lamb, L. C., and Gabbay, D. M. Neural-Symbolic Cognitive Reasoning. Cognitive Technologies. Springer, 2009. ISBN 978-3-540-73245-7. URL http://dblp.uni-trier.de/db/series/cogtech/index.html#GarcezLG2009

work page 2009

[4] [4]

J., and Pehlevan, C

Kumar, T., Bordelon, B., Gershman, S. J., and Pehlevan, C. Grokking as the transition from lazy to rich training dynamics. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=vt5mnLVIVo

work page 2024

[5] [5]

Grokking in linear estimators - A solvable model that groks without understanding

Levi, N., Beck, A., and Bar - Sinai, Y. Grokking in linear estimators - A solvable model that groks without understanding. CoRR, abs/2310.16441, 2023. doi:10.48550/ARXIV.2310.16441. URL https://doi.org/10.48550/arXiv.2310.16441

arXiv doi:10.48550/arxiv.2310.16441 2023

[6] [6]

Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll \'a r, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp.\ 740--755. Springer, 2014

2014

[7] [7]

S., Michaud, E., Tegmark, M., and Williams, M

Liu, Z., Kitouni, O., Nolte, N. S., Michaud, E., Tegmark, M., and Williams, M. Towards understanding grokking: An effective theory of representation learning. Advances in Neural Information Processing Systems, 35: 0 34651--34663, 2022

2022

[8] [8]

Progress measures for grokking via mechanistic interpretability

Nanda, N., Chan, L., Lieberum, T., Smith, J., and Steinhardt, J. Progress measures for grokking via mechanistic interpretability. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.net, 2023. URL https://openreview.net/pdf?id=9XFSbDPmdW

work page 2023