Pith Number

pith:6FLJ2FRK

pith:2021:6FLJ2FRK3565SJWTG3QRIA4TF7

not attested not anchored not stored refs resolved

Scaling Language Models: Methods, Analysis & Insights from Training Gopher

Adhiguna Kuncoro, Aidan Clark, Aida Nematzadeh, Albin Cassirer, Amelia Glaese, Amy Wu, Angeliki Lazaridou, Antonia Creswell, Arthur Mensch, Aurelia Guy, Blake Hechtman, Chris Dyer, Chris Jones, Cyprien de Masson d'Autume, Daniel Toyama, David Budden, Demis Hassabis, Diego de las Casas, Domenic Donato, Doug Fritz, Ed Lockhart, Elena Buchatskaya, Elena Gribovskaya, Eliza Rutherford, Erich Elsen, Esme Sutherland, Francis Song, Geoffrey Irving, George van den Driessche, Iason Gabriel, Igor Babuschkin, Irina Higgins, Jack W. Rae, Jacob Menick, James Bradbury, Jean-Baptiste Lespiau, Jeff Stanway, Johannes Welbl, John Aslanides, John Mellor, Jonathan Uesato, Jordan Hoffmann, Kareem Ayoub, Karen Simonyan, Katie Millican, Koray Kavukcuoglu, Laura Rimell, Laura Weidinger, Laurent Sifre, Lena Martens, Lisa Anne Hendricks, Lorrayne Bennett, Mantas Pajarskas, Maria Tsimpoukelli, Maribeth Rauh, Matthew Johnson, Michela Paganini, Nat McAleese, Nikolai Grigorev, Oriol Vinyals, Po-Sen Huang, Richard Powell, Roman Ring, Saffron Huang, Sarah Henderson, Sebastian Borgeaud, Siddhant Jayakumar, Simon Osindero, Sumanth Dathathri, Susannah Young, Tayfun Terzi, Thibault Sottiaux, Toby Pohlen, Tom Hennigan, Trevor Cai, Vladimir Mikulik, William Isaac, Xiang Lorraine Li, Yujia Li, Zhitao Gong

Larger language models up to 280 billion parameters reach state-of-the-art results on most of 152 tasks, with scale helping reading and fact-checking most.

arxiv:2112.11446 v2 · 2021-12-08 · cs.CL · cs.AI

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{6FLJ2FRK3565SJWTG3QRIA4TF7}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

These models are evaluated on 152 diverse tasks, achieving state-of-the-art performance across the majority. Gains from scale are largest in areas such as reading comprehension, fact-checking, and the identification of toxic language, but logical and mathematical reasoning see less benefit.

C2weakest assumption

That performance differences across scales are primarily driven by model size rather than confounding factors such as dataset composition, training details, or evaluation choices, and that the 152 tasks sufficiently represent broader capabilities.

C3one line summary

Gopher, a 280 billion parameter language model, achieves state-of-the-art performance on the majority of 152 tasks with largest gains in reading comprehension, fact-checking, and toxic language detection.

References

79 extracted · 79 resolved · 2 Pith anchors

[1] URL https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb49674 18bfb8ac142f64a-Paper.pdf. J. Buckman. Fair ML tools require problematic ML models.https://jacobbuckman.com/2021- 02-15-fair-ml-too 2020 · arXiv:1812.01193

[2] doi: 10.18653/v1/2020.findings-emnlp.301 1902 · doi:10.18653/v1/2020.findings-emnlp.301

[3] , author Barocas, S 2009 · doi:10.1145/3351095.3372826

[4] In: Cohn, T., He, Y., Liu, Y 2011 · doi:10.18653/v1/2020.findings-emnlp.372

[5] Jouppi, Doe Hyun Yoon, George Kurian, Sheng Li, Nishant Patil, James Laudon, Cliff Young, and David A 2016 · doi:10.1145/3360307

Cited by

119 papers in Pith

Neural Scaling Universality: If Exponents Are Fixed, Time to Understand Coefficients

Interleaved Speech Language Models Latently Work In Text

Efficient Neural Network Model Selection for Few-Class Application Datasets

Characterizing Narrative Content in Web-scale LLM Pretraining Data

BLADE: Scalable Bi-level Adaptive Data Selection for LLM Training

Receipt and verification

First computed	2026-07-05T03:50:27.025729Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

f1569d162adf7dd926d336e11403932fc838cd859c5f0bd7573161321d5279e0

Aliases

arxiv: 2112.11446 · arxiv_version: 2112.11446v2 · doi: 10.48550/arxiv.2112.11446 · pith_short_12: 6FLJ2FRK3565 · pith_short_16: 6FLJ2FRK3565SJWT · pith_short_8: 6FLJ2FRK

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/6FLJ2FRK3565SJWTG3QRIA4TF7 \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: f1569d162adf7dd926d336e11403932fc838cd859c5f0bd7573161321d5279e0

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "acb60e61a0467c59a348897bf95d6b42216d5834406a43f0c68ae829e874e70c",
    "cross_cats_sorted": [
      "cs.AI"
    ],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2021-12-08T19:41:47Z",
    "title_canon_sha256": "3398a07379e6d2893eae299e46e16b8b3997b3d623cc8b591e7e13c1ce02542d"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2112.11446",
    "kind": "arxiv",
    "version": 2
  }
}