pith. sign in

arxiv: 2201.02177 · v1 · pith:2SAIDR7Mnew · submitted 2022-01-06 · 💻 cs.LG

Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets

Alethea Power , Yuri Burda , Harri Edwards , Igor Babuschkin , Vedant Misra This is my paper

Pith reviewed 2026-05-11 19:22 UTC · model grok-4.3

classification 💻 cs.LG
keywords grokkinggeneralizationoverfittingneural networksalgorithmic datasetsmemorizationdeep learningoptimization
0
0 comments X

The pith

Neural networks can suddenly achieve perfect generalization on small algorithmic tasks long after they have overfitted the training data.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines generalization in neural networks using small, algorithmically generated datasets that allow precise control over every aspect of the data. It shows that networks sometimes first overfit by memorizing training examples to the point of poor test performance, then later improve dramatically to perfect generalization through a process the authors call grokking. This jump occurs after far more optimization steps than needed to reach overfitting. The authors also find that smaller datasets require progressively more training to reach good generalization. These controlled settings are presented as a way to study how overparameterized networks can generalize beyond simply memorizing the finite training set.

Core claim

In some situations we show that neural networks learn through a process of grokking a pattern in the data, improving generalization performance from random chance level to perfect generalization, and that this improvement in generalization can happen well past the point of overfitting. We also study generalization as a function of dataset size and find that smaller datasets require increasing amounts of optimization for generalization. These datasets provide a fertile ground for studying generalization of overparameterized neural networks beyond memorization of the finite training dataset.

What carries the argument

The grokking process on small algorithmic datasets, in which generalization jumps from chance to perfect performance after overfitting has already occurred.

If this is right

  • Generalization performance can continue to improve substantially after a model has already overfit the training data.
  • Smaller dataset sizes require more optimization steps before generalization is achieved.
  • These algorithmically generated datasets make it possible to isolate and measure the transition from memorization to pattern-based generalization.
  • Overparameterized networks are capable of generalizing beyond rote memorization of the training examples.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Tracking test accuracy over very long training runs could reveal similar late-stage improvements in other domains where only short training is usually examined.
  • The separation between overfitting and later generalization phases suggests optimization trajectories may contain distinct stages that current early-stopping practices might miss.
  • If grokking depends on the structure of algorithmic data, it may be possible to design synthetic training distributions that deliberately induce this delayed generalization in practical tasks.

Load-bearing premise

The grokking behavior observed on these specific small algorithmic datasets reveals a general mechanism of neural network generalization rather than an artifact limited to the chosen tasks, architectures, and optimization regimes.

What would settle it

Finding that on other datasets or with other architectures generalization either stays at chance level after overfitting or improves only gradually without a sudden late jump would falsify the existence of a distinct grokking process.

read the original abstract

In this paper we propose to study generalization of neural networks on small algorithmically generated datasets. In this setting, questions about data efficiency, memorization, generalization, and speed of learning can be studied in great detail. In some situations we show that neural networks learn through a process of "grokking" a pattern in the data, improving generalization performance from random chance level to perfect generalization, and that this improvement in generalization can happen well past the point of overfitting. We also study generalization as a function of dataset size and find that smaller datasets require increasing amounts of optimization for generalization. We argue that these datasets provide a fertile ground for studying a poorly understood aspect of deep learning: generalization of overparametrized neural networks beyond memorization of the finite training dataset.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 3 minor

Summary. The manuscript proposes small, algorithmically generated datasets (e.g., modular arithmetic tasks) as a controlled testbed for studying neural network generalization. It reports the 'grokking' phenomenon in which test accuracy jumps from chance level to perfect generalization well after training accuracy has saturated at 100%, and shows that smaller training sets require substantially more optimization steps before generalization occurs. The authors argue these setups are useful for examining generalization beyond memorization in overparameterized models.

Significance. If the observations hold under the reported conditions, the work supplies a clean, fully reproducible experimental platform for dissecting delayed generalization. The use of perfectly structured, small-scale algorithmic tasks enables precise tracking of learning dynamics that are difficult to isolate on standard benchmarks. Credit is due for the emphasis on controlled, open setups that facilitate community follow-up rather than asserting a universal mechanism.

minor comments (3)
  1. [Figures 1-2] Figure 1 and Figure 2: the x-axis (training steps) scaling and lack of shaded variance bands across random seeds make it difficult to judge how consistently the grokking transition occurs and at what exact step count.
  2. [Section 3.1] Section 3.1: the precise definition of 'overfitting' (e.g., whether it is the first step at which train accuracy reaches 1.0 or a sustained plateau) should be stated explicitly so readers can replicate the 'well past' timing claim.
  3. [Experimental details] The manuscript would be strengthened by a short table summarizing hyper-parameters (learning rate, weight decay, batch size, architecture depth/width) for each reported experiment.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive assessment of our work and the recommendation for minor revision. We appreciate the recognition that small algorithmic datasets provide a clean, reproducible platform for studying delayed generalization in overparameterized models.

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper is an empirical observational study of neural network behavior on small algorithmic datasets. It reports the grokking phenomenon as an observed pattern without any derivation chain, equations, or fitted parameters that reduce the reported results to self-referential definitions or inputs. Claims are scoped to 'in some situations' on the chosen testbeds and do not rely on self-citations, uniqueness theorems, or ansatzes for their central content.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The paper is an empirical study relying on standard assumptions of neural network training; no free parameters, new axioms, or invented entities are introduced in the abstract.

axioms (1)
  • domain assumption Gradient-based optimization of neural networks on finite datasets can produce both memorization and later generalization.
    Standard background assumption in machine learning invoked to frame the experiments.

pith-pipeline@v0.9.0 · 5437 in / 1107 out tokens · 58879 ms · 2026-05-11T19:22:31.706868+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 60 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Dendritic In-Context Learning in a Single-Layer Spiking Neural Network

    cs.NE 2026-07 unverdicted novelty 8.0

    A single-layer compartmental SNN with apical recurrence matching leaky online Widrow-Hoff LMS achieves seed-stable ICL on high-dimensional Garg-2022 tasks where Transformers fail, with a linear probe recovering the LM...

  2. Neural Networks Provably Learn Spectral Representations for Group Composition

    cs.LG 2026-06 unverdicted novelty 8.0

    Two-layer neural networks provably converge almost surely to irreducible representations of finite groups when trained on the group composition task, with the dynamics governed by Riemannian gradient ascent on a repre...

  3. Canonical Regularisation of Wide Feature-Learning Neural Networks

    stat.ML 2026-05 unverdicted novelty 8.0

    Derives geodesic ridge regularization and Riemannian Gibbs Process prior for feature-learning wide neural networks, generalizing kernel-regime results via function-space axiomatization.

  4. Spherical Boltzmann machines: a solvable theory of learning and generation in energy-based models

    cs.LG 2026-05 unverdicted novelty 8.0

    In the high-dimensional limit the spherical Boltzmann machine admits exact equations for training dynamics, Bayesian evidence, and cascades of phase transitions tied to mode alignment with data, which connect to gener...

  5. The Spectral Lifecycle of Transformer Training: Transient Compression Waves, Persistent Spectral Gradients, and the Q/K--V Asymmetry

    cs.LG 2026-04 unverdicted novelty 8.0

    Transformer weight spectra exhibit transient compression waves that propagate layer-wise, persistent non-monotonic depth gradients in power-law exponents, and Q/K-V asymmetry, with the spectral exponent alpha predicti...

  6. When Does Content-Based Routing Work? Representation Requirements for Selective Attention in Hybrid Sequence Models

    cs.LG 2026-03 conditional novelty 8.0

    Content-based routing succeeds only when models provide bidirectional context and perform pairwise comparisons, with bidirectional Mamba plus rank-1 projection reaching 99.7% precision at linear inference cost.

  7. Algorithmic Task Capture, Computational Complexity, and Inductive Bias of Infinite Transformers

    cs.LG 2026-03 unverdicted novelty 8.0

    Infinite-width transformers exhibit an inductive bias against high-complexity polynomial-time algorithms, with derived upper bounds on capturable tasks like sorting and string matching.

  8. Low-Dimensional and Transversely Curved Optimization Dynamics in Grokking

    cs.LG 2026-02 unverdicted novelty 8.0

    Grokking reflects escape from a metastable low-dimensional regime where transverse curvature accumulates before generalization, with subspace motion necessary but curvature boost insufficient.

  9. The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery

    cs.AI 2024-08 unverdicted novelty 8.0

    The AI Scientist framework enables LLMs to independently conduct the full scientific process from idea generation to paper writing and review, demonstrated across three ML subfields with papers costing under $15 each.

  10. Progress measures for grokking via mechanistic interpretability

    cs.LG 2023-01 accept novelty 8.0

    Grokking arises from gradual amplification of a Fourier-based circuit in the weights followed by removal of memorizing components.

  11. Toy Models of Superposition

    cs.LG 2022-09 accept novelty 8.0

    Toy models demonstrate that polysemanticity arises when neural networks store more sparse features than neurons via superposition, producing a phase transition tied to polytope geometry and increased adversarial vulne...

  12. Measuring Dead Directions: Decomposing and Classifying Singular Structure off Canonical Alignment

    cs.LG 2026-07 unverdicted novelty 7.0

    A descent-free method recovers the singularity order k of dead directions in neural networks from the directional-Fisher rate, classifies them, and assembles global learning coefficients matching closed forms.

  13. Learning as Observable Matrix Dynamics: Diffusive Relaxations versus Phase Transitions

    cs.LG 2026-06 unverdicted novelty 7.0

    Observable Matrix Dynamics (OMD) is a new diagnostic framework that uses random matrix theory on distance matrices to distinguish diffusive relaxations from phase-transition-like reorganizations during neural network ...

  14. Natural Ungrokking: Asymmetric Control of Which Rules Survive Pretraining

    cs.LG 2026-06 unverdicted novelty 7.0

    During pretraining, language models exhibit natural ungrokking where learned rules are forgotten based on their support frequency in the corpus, with asymmetric editability of rule survival.

  15. The Generalization Spectrum: A Chromatographic Approach to Evaluating Learning Algorithms

    cs.LG 2026-06 unverdicted novelty 7.0

    Introduces the Generalization Spectrum evaluation framework to track per-example generalization across transfer distances in competitive programming tasks.

  16. Dead-Direction Signatures: A Cheap Spectral Reading of Singular Complexity

    cs.LG 2026-06 unverdicted novelty 7.0

    Dead-Direction Signatures provide closed-form spectral readings of dead directions in network activations and gradients that track rank deficits at singular minima, offering a cheap directional alternative to SGLD-based LLC.

  17. Circuit Synchronization Precedes Generalization: A Causal Precursor to Grokking

    cs.LG 2026-06 conditional novelty 7.0

    FSD, a permutation-tested metric of Fourier circuit synchronization, precedes grokking by a mean of 1722 steps across nine modular addition setups and causally controls grokking timing when weight decay is varied at t...

  18. When Probing Accuracy Saturates, Fragility Resolves: A Complementary Metric for LLM Pre-Training Analysis

    cs.CL 2026-06 unverdicted novelty 7.0

    Fragility, the activation noise level causing probe accuracy collapse, reveals evolving lexical-to-compositional moral encoding, layer robustness gradients, and fine-tuning differences invisible to saturated probing accuracy.

  19. Dead Directions: Geometric Singular Learning

    cs.LG 2026-06 unverdicted novelty 7.0

    Dead directions recover Watanabe's RLCT contribution and triple (λ, m, ν) from directional Fisher curvature decay rates in original parameter space for singular models, extended via K-FAC to networks and gauge-equivar...

  20. Dynamics and Representation Structure of Local Approximations to Gradient-Based Learning in Linear Recurrent Neural Networks

    cs.NE 2026-05 unverdicted novelty 7.0

    RFLO learning restricts solutions to low-rank perturbations of initial parameters in linear RNNs and produces qualitatively different stability and convergence behavior than BPTT.

  21. Does Capability Transfer to Subjective Behavior -- and Would Our Instruments Tell Us? A Self-Evolving, Trust-by-Construction Evaluation Paradigm

    cs.CL 2026-05 unverdicted novelty 7.0

    Self-evolving rubric with anti-gaming fitness reveals that objective capability scaling fails to transfer to subjective LLM behaviors, with advice-restraint as the universal lowest dimension that can regress.

  22. Phantom transitions in language model fine-tuning

    cs.CL 2026-05 accept novelty 7.0

    Apparent phase transitions during fine-tuning on near-synonym tasks are phantoms originating in the softmax readout; an order parameter isolates kinematic and structural failure modes and a few dimensionless quantitie...

  23. Feature Learning in Linear-Width Two-Layer Networks: Two vs. One Step of Gradient Descent

    stat.ML 2026-05 unverdicted novelty 7.0

    In the linear-width regime, the second GD step yields a spiked random matrix whose number of outliers is floor(alpha2 / (1/2 - alpha1)), and batch reuse enables learning directions with information exponent greater th...

  24. Feature Learning in Linear-Width Two-Layer Networks: Two vs. One Step of Gradient Descent

    stat.ML 2026-05 unverdicted novelty 7.0

    Two steps of gradient descent on first-layer weights in linear-width two-layer networks produce a spiked random matrix with floor(alpha2/(1/2-alpha1)) outliers, each a learned direction, and batch reuse allows capturi...

  25. Bounded-Rationality, Hedging, and Generalization

    cs.LG 2026-05 unverdicted novelty 7.0

    Generalization is a testable hedging property of the learner's response law, recovered via f-divergence regularizers that induce information-geometric curves between training loss and sample dependence.

  26. The Benefits of Temporal Correlations: SGD Learns k-Juntas from Random Walks Efficiently

    cs.LG 2026-05 unverdicted novelty 7.0

    Temporal correlations from lazy random walks enable efficient SGD learning of k-juntas via temporal-difference loss on ReLU networks, achieving linear sample complexity in d.

  27. The Geometric Structure of Models Learning Sparse Data

    cs.LG 2026-05 unverdicted novelty 7.0

    In sparse regimes, models exploit normal alignment of Jacobians to minimize loss and maximize robustness; GrokAlign induces this alignment to accelerate training and RFAMs improve adversarial robustness.

  28. The Geometric Structure of Models Learning Sparse Data

    cs.LG 2026-05 unverdicted novelty 7.0

    Normal alignment is the rank-one Jacobian structure that lets classifiers minimize loss and maximize local robustness in sparse regimes; the paper proves its optimality and uses it to create GrokAlign and RFAMs.

  29. Topological Signatures of Grokking

    cs.LG 2026-05 unverdicted novelty 7.0

    Persistent homology detects a sharp increase in maximum and total H1 persistence during grokking on modular arithmetic, offering a topological diagnostic that links representation geometry to generalization.

  30. Grokking or Glitching? How Low-Precision Drives Slingshot Loss Spikes

    cs.LG 2026-05 unverdicted novelty 7.0

    Slingshot loss spikes are produced by low-precision arithmetic that breaks the zero-sum gradient constraint and drives exponential growth via Numerical Feature Inflation.

  31. Grokking or Glitching? How Low-Precision Drives Slingshot Loss Spikes

    cs.LG 2026-05 unverdicted novelty 7.0

    Slingshot loss spikes arise from floating-point precision limits that round correct-class gradients to zero, breaking zero-sum constraints and driving exponential parameter growth through numerical feature inflation.

  32. Grokking or Glitching? How Low-Precision Drives Slingshot Loss Spikes

    cs.LG 2026-05 unverdicted novelty 7.0

    Slingshot loss spikes result from floating-point precision limits that round correct-class gradients to zero, triggering Numerical Feature Inflation and breaking gradient zero-sum constraints.

  33. Estimating Implicit Regularization in Deep Learning

    stat.ML 2026-05 unverdicted novelty 7.0

    Gradient matching empirically recovers implicit regularization effects such as l2 penalties from early stopping and dropout in neural networks.

  34. Layerwise LQR for Geometry-Aware Optimization of Deep Networks

    cs.LG 2026-05 unverdicted novelty 7.0

    Steepest descent under divergence-induced quadratic models equals an LQR problem, enabling learning of diagonal or Kronecker-factored inverse preconditioners via a global layerwise objective for scalable geometry-awar...

  35. A Theory of Generalization in Deep Learning

    cs.LG 2026-05 unverdicted novelty 7.0

    A theory shows SGD accumulates coherent signal via linear drift in NTK signal directions while trapping noise in orthogonal low-eigenvalue dimensions, enabling generalization even under O(1) kernel evolution and yield...

  36. ILDR: Geometric Early Detection of Grokking

    cs.LG 2026-04 unverdicted novelty 7.0

    ILDR detects the geometric reorganization preceding grokking by measuring when inter-class centroid separation exceeds intra-class scatter by 2.5 times its baseline in penultimate-layer representations.

  37. Grokking of Diffusion Models: Case Study on Modular Addition

    cs.LG 2026-04 unverdicted novelty 7.0

    Diffusion models show grokking on modular addition by composing periodic operand representations in simple data regimes or by separating arithmetic computation from visual denoising across timesteps in varied regimes.

  38. Scalable Neural Decoders for Practical Fault-Tolerant Quantum Computation

    quant-ph 2026-04 unverdicted novelty 7.0

    Neural decoder for quantum LDPC codes achieves ~10^{-10} logical error at 0.1% physical error with 17x improvement and high throughput, enabling practical fault tolerance at modest code sizes.

  39. Is your algorithm unlearning or untraining?

    cs.LG 2026-04 conditional novelty 7.0

    Machine unlearning conflates reversing the influence of specific training examples (untraining) with removing the full underlying distribution or behavior (unlearning).

  40. Spectral Edge Dynamics Reveal Functional Modes of Learning

    cs.LG 2026-04 unverdicted novelty 7.0

    Spectral edge dynamics during grokking reveal task-dependent low-dimensional functional modes over inputs, such as Fourier modes for modular addition and cross-term decompositions for x squared plus y squared.

  41. Dimensional Criticality at Grokking Across MLPs and Transformers

    cs.LG 2026-04 unverdicted novelty 7.0

    Effective cascade dimension D(t) crosses D=1 at the grokking transition in MLPs and Transformers, with opposite directions for modular addition versus XOR, consistent with attraction to a shared critical manifold.

  42. The Long Delay to Arithmetic Generalization: When Learned Representations Outrun Behavior

    cs.LG 2026-03 unverdicted novelty 7.0

    The grokking delay in encoder-decoder models on one-step Collatz prediction stems from decoder inability to use early-learned encoder representations of parity and residue structure, with numeral base acting as a stro...

  43. Rethinking the Harmonic Loss via Non-Euclidean Distance Layers

    cs.LG 2026-03 unverdicted novelty 7.0

    Non-Euclidean distance variants of harmonic loss improve accuracy, gradient stability, and energy efficiency over cross-entropy and Euclidean harmonic loss in vision backbones and large language models.

  44. The Norm-Separation Delay Law of Grokking: A First-Principles Theory of Delayed Generalization

    cs.AI 2026-03 conditional novelty 7.0

    Grokking delay follows T_grok - T_mem = Θ(γ_eff^{-1} log(‖θ_mem‖² / ‖θ_post‖²)), derived from norm separation in regularized optimization and validated with high correlations across 293 runs.

  45. The Geometry of Multi-Task Grokking: Transverse Instability, Superposition, and Weight Decay Phase Structure

    cs.LG 2026-02 unverdicted novelty 7.0

    Multi-task grokking in Transformers produces staggered generalization, low-dimensional manifolds, weight-decay phase structure, holographic solutions, and transverse redundancy.

  46. Egalitarian Gradient Descent: A Simple Approach to Accelerated Grokking

    cs.LG 2025-10 unverdicted novelty 7.0

    EGD equalizes gradient speeds across singular directions, eliminating or shortening grokking plateaus on modular addition and sparse parity problems.

  47. Reinforcement Learning for Reasoning in Large Language Models with One Training Example

    cs.LG 2025-04 accept novelty 7.0

    One training example via RLVR boosts LLM math reasoning from 17.6% to 35.7% average across six benchmarks.

  48. In-context Learning and Induction Heads

    cs.LG 2022-09 unverdicted novelty 7.0

    Induction heads, which implement pattern completion in attention, develop at the same training stage as a sudden rise in in-context learning, providing evidence they are the primary mechanism for in-context learning i...

  49. Radial Suppression Accelerates Algorithmic Generalization: A Geometric Analysis of Delayed Generalization

    cs.LG 2026-06 unverdicted novelty 6.0

    A norm penalty constraining activations to a sqrt(d)-radius hypersphere accelerates grokking by up to 6x on modular arithmetic via radial suppression in activation dynamics.

  50. The Generalization Spectrum: A Chromatographic Approach to Evaluating Learning Algorithms

    cs.LG 2026-06 unverdicted novelty 6.0

    The Generalization Spectrum framework shows RL turns memorization into near-transfer more efficiently than SFT, ICL transfer depends on correspondence, and local gains often fail to expand far-transfer radius.

  51. Patnaik-Pearson intrinsic dimension for internal representations of neural networks

    math.ST 2026-06 unverdicted novelty 6.0

    Introduces the Patnaik-Pearson intrinsic dimension estimator, proves some of its properties, relates it to HTSR/SETOL for Pareto spectra, and applies it to track embedding dimension evolution in BERT-base and DeepSeek...

  52. Patnaik-Pearson intrinsic dimension for internal representations of neural networks

    math.ST 2026-06 unverdicted novelty 6.0

    Introduces the Patnaik-Pearson intrinsic dimension estimator, relates it to HTSR/SETOL for Pareto spectral densities, and applies it to measure embedding dimension evolution in BERT-base and DeepSeek-R1-Distill-Qwen-1.

  53. Repeated Shared Access Enables Grokking, but Edit Propagation Depends on an Addressable Memory

    cs.AI 2026-06 unverdicted novelty 6.0

    A 2x2 ablation shows repeated shared access enables grokking while addressable memory (not recurrence) enables edit propagation in transformer variants on synthetic KG QA.

  54. Learning Dynamics of Chain-of-Thought State Tracking in a Solvable Transformer Model

    cond-mat.dis-nn 2026-06 unverdicted novelty 6.0

    Mean-field equations for attention retrieval, teacher alignment, and logic overlap quantitatively match simulations and predict a sharp accuracy transition in a solvable transformer for permutation state tracking.

  55. XtrAIn: Training-Guided Occlusion for Feature Attribution

    cs.LG 2026-06 unverdicted novelty 6.0

    XtrAIn shifts occlusion from input space to parameter space along the training trajectory to produce cleaner feature attributions than standard methods.

  56. Deciphering Two Training Clocks in Grokking via Deep Linear Network Theory with Conditional ReLU Reduction

    cs.LG 2026-06 unverdicted novelty 6.0

    Deep linear network theory derives logarithmic decay for cross-entropy loss under gap-growth conditions versus polynomial closure for Schatten-regularized structural energy under late-time KL tails, separating fitting...

  57. Arithmetic Pedagogy for Language Models

    cs.CL 2026-06 unverdicted novelty 6.0

    A small GPT-2 model trained from scratch on GASING-derived CoT supervision for arithmetic reaches over 80% held-out accuracy, exhibits three learning phases, and develops both procedural and associative reasoning.

  58. Consistency Training while Mitigating Obfuscation via Rate Matching

    cs.CL 2026-06 unverdicted novelty 6.0

    RMCT matches the rate of target behaviors like bias-following across input perturbations to reduce sycophancy in LLMs while preserving verbalization of bias cues.

  59. Emergent Ordinal Geometry in Transformers Trained on Local Comparisons

    cs.AI 2026-05 unverdicted novelty 6.0

    Transformers trained on local comparisons spontaneously form a rank-aligned one-dimensional embedding manifold that reproduces the symbolic distance effect.

  60. A Pre-Training Analogue of Grokking in Language Models: Tracing Delayed Grammatical Generalization

    cs.LG 2026-05 unverdicted novelty 6.0

    An exposure-based split on BLiMP data reveals delayed generalization in five grammatical phenomena during LLM pre-training, with post-generalization shifts in concept vector predictiveness and attention patterns.

Reference graph

Works this paper leans on

21 extracted references · 21 canonical work pages · cited by 118 Pith papers · 7 internal anchors

  1. [1]

    Proceedings of the National Academy of Sciences , volume =

    Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal. Reconciling modern machine learning practice and the bias-variance trade-off. arXiv preprint arXiv:1812.11118,

    work page arXiv

  2. [2]

    Triple descent and the two kinds of overfitting: Where & why do they appear? arXiv preprint arXiv:2006.03509,

    St´ephane d’Ascoli, Levent Sagun, and Giulio Biroli. Triple descent and the two kinds of overfitting: Where & why do they appear? arXiv preprint arXiv:2006.03509,

    work page arXiv 2006

  3. [3]

    Universal Transformers

    Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, andŁukasz Kaiser. Universal transformers. arXiv preprint arXiv:1807.03819,

    work page internal anchor Pith review arXiv

  4. [4]

    Adaptive Computation Time for Recurrent Neural Networks

    Alex Graves. Adaptive computation time for recurrent neural networks. arXiv preprint arXiv:1603.08983,

    work page internal anchor Pith review arXiv

  5. [5]

    Neural Turing Machines

    Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines. arXiv preprint arXiv:1410.5401,

    work page internal anchor Pith review arXiv

  6. [6]

    Jiang, B

    Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantastic generalization measures and where to find them. arXiv preprint arXiv:1912.02178,

    work page arXiv 1912

  7. [7]

    Neural GPUs Learn Algorithms

    Łukasz Kaiser and Ilya Sutskever. Neural gpus learn algorithms. arXiv preprint arXiv:1511.08228,

    work page Pith review arXiv

  8. [9]

    On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima

    URL http://arxiv.org/abs/1609.04836. Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101,

    work page internal anchor Pith review arXiv

  9. [10]

    arXiv , year =

    Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Ilya Sutskever. Deep double descent: Where bigger models and more data hurt. arXiv preprint arXiv:1912.02292,

    work page arXiv 1912

  10. [11]

    Adding Gradient Noise Improves Learning for Very Deep Networks

    Arvind Neelakantan, Luke Vilnis, Quoc V Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach, and James Martens. Adding gradient noise improves learning for very deep networks. arXiv preprint arXiv:1511.06807,

    work page Pith review arXiv

  11. [12]

    Neural Programmer-Interpreters

    Scott Reed and Nando De Freitas. Neural programmer-interpreters. arXiv preprint arXiv:1511.06279,

    work page Pith review arXiv

  12. [13]

    Analysing Mathematical Reasoning Abilities of Neural Models

    David Saxton, Edward Grefenstette, Felix Hill, and Pushmeet Kohli. Analysing mathematical reasoning abilities of neural models. arXiv preprint arXiv:1904.01557,

    work page Pith review arXiv 1904

  13. [14]

    Dropout: a simple way to prevent neural networks from overfitting

    Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929–1958,

    work page 1929

  14. [15]

    Attention Is All You Need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. arXiv preprint arXiv:1706.03762,

    work page internal anchor Pith review Pith/arXiv arXiv

  15. [16]

    Memory Networks

    Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. arXiv preprint arXiv:1410.3916,

    work page Pith review arXiv

  16. [17]

    Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks

    5 Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M Rush, Bart van Merri¨enboer, Armand Joulin, and Tomas Mikolov. Towards ai-complete question answering: A set of prerequisite toy tasks. arXiv preprint arXiv:1502.05698,

    work page Pith review arXiv

  17. [18]

    Reinforcement learning neural turing machines-revised

    Wojciech Zaremba and Ilya Sutskever. Reinforcement learning neural turing machines-revised. arXiv preprint arXiv:1505.00521,

    work page internal anchor Pith review arXiv

  18. [19]

    Understanding deep learning requires rethinking generalization

    Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530,

    work page internal anchor Pith review arXiv

  19. [20]

    For each training run, we chose a fraction of all available equations at random and declared them to be the training set, with the rest of equations being the validation set

    A A PPENDIX A.1 A DDITIONAL EXPERIMENTAL DETAILS A.1.1 B INARY OPERATIONS The following are the binary operations that we have tried (for a prime numberp = 97): x ◦y =x +y (mod p) for 0 ≤x,y <p x ◦y =x −y (mod p) for 0 ≤x,y <p x ◦y =x/y (mod p) for 0 ≤x<p , 0<y <p x ◦y = [x/y (mod p) ify is odd, otherwisex −y (mod p)] for 0 ≤x,y <p x ◦y =x2 +y2 (mod p) fo...

    work page 2017

  20. [21]

    grok-like

    In Figure 5 we show an example of a binary operation table that the network can actually solve. Figure 4: The loss curves for modular division, train and validation. We see the validation loss increases from 102 to about 105 optimization steps before it begins a second descent. A.3 R ELATED WORK In this paper we study training and generalization dynamics ...

    work page 2014

  21. [22]

    We interpret this as additional evidence that the capacity of the network and optimization procedure is well beyond the capacity needed for 9 memorizing all the labels on the training data, and that generalization happening at all requires a non-trivial explanation. A.5 G ENERALIZATION MEASURES We believe it is useful to explore how predictive common gene...

    work page 1997