Pith Number

pith:2SAIDR7M

pith:2022:2SAIDR7M2WPV7CH7FZDBIZHXK4

not attested not anchored not stored refs resolved

Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets

Alethea Power, Harri Edwards, Igor Babuschkin, Vedant Misra, Yuri Burda

Neural networks can suddenly achieve perfect generalization on small algorithmic tasks long after they have overfitted the training data.

arxiv:2201.02177 v1 · 2022-01-06 · cs.LG

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{2SAIDR7M2WPV7CH7FZDBIZHXK4}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

In some situations we show that neural networks learn through a process of 'grokking' a pattern in the data, improving generalization performance from random chance level to perfect generalization, and that this improvement in generalization can happen well past the point of overfitting.

C2weakest assumption

That the grokking behavior observed on these specific small algorithmic datasets reveals a general mechanism of neural network generalization rather than an artifact limited to the chosen tasks, architectures, and optimization regimes.

C3one line summary

Neural networks exhibit grokking on small algorithmic datasets, achieving perfect generalization well after overfitting.

References

21 extracted · 21 resolved · 7 Pith anchors

[1] Proceedings of the National Academy of Sciences , volume =

[2] Triple descent and the two kinds of overﬁtting: Where & why do they appear? arXiv preprint arXiv:2006.03509 2006

[3] Universal Transformers · arXiv:1807.03819

[4] Adaptive Computation Time for Recurrent Neural Networks · arXiv:1603.08983

[5] Neural Turing Machines · arXiv:1410.5401

Cited by

128 papers in Pith

Natural Ungrokking: Asymmetric Control of Which Rules Survive Pretraining

The Generalization Spectrum: A Chromatographic Approach to Evaluating Learning Algorithms

Emergent Capabilities Arise Randomly from Learning Sparse Attention Patterns

Parallel Manifold Steering: Efficient Adaptation of Large Associative Memories via Residual Energy Shaping

The Generalization Spectrum: A Chromatographic Approach to Evaluating Learning Algorithms

Receipt and verification

First computed	2026-07-05T03:46:30.868533Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

d48081c7ecd59f5f88ff2e461464f757251ab620ace18acefaa0fcc821db0179

Aliases

arxiv: 2201.02177 · arxiv_version: 2201.02177v1 · doi: 10.48550/arxiv.2201.02177 · pith_short_12: 2SAIDR7M2WPV · pith_short_16: 2SAIDR7M2WPV7CH7 · pith_short_8: 2SAIDR7M

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/2SAIDR7M2WPV7CH7FZDBIZHXK4 \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: d48081c7ecd59f5f88ff2e461464f757251ab620ace18acefaa0fcc821db0179

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "9511d9be986d9d2a086afb5f53cb431ab6087348ade0e71e49051b0cff454e96",
    "cross_cats_sorted": [],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.LG",
    "submitted_at": "2022-01-06T18:43:37Z",
    "title_canon_sha256": "aae4d6517ec56064910e587d7f48ff6067f4834ca66c4d0cf2947700346a86e2"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2201.02177",
    "kind": "arxiv",
    "version": 1
  }
}