Pith Number

pith:FZMDE3LS

pith:2020:FZMDE3LSPVOF2LQ5KLUU6Z3LD5

not attested not anchored not stored refs resolved

Language Models are Few-Shot Learners

Aditya Ramesh, Alec Radford, Amanda Askell, Ariel Herbert-Voss, Arvind Neelakantan, Benjamin Chess, Benjamin Mann, Christopher Berner, Christopher Hesse, Clemens Winter, Daniel M. Ziegler, Dario Amodei, Eric Sigler, Girish Sastry, Gretchen Krueger, Ilya Sutskever, Jack Clark, Jared Kaplan, Jeffrey Wu, Mark Chen, Mateusz Litwin, Melanie Subbiah, Nick Ryder, Prafulla Dhariwal, Pranav Shyam, Rewon Child, Sam McCandlish, Sandhini Agarwal, Scott Gray, Tom B. Brown, Tom Henighan

Scaling language models to 175 billion parameters enables strong few-shot performance on NLP tasks without any fine-tuning.

arxiv:2005.14165 v4 · 2020-05-28 · cs.CL

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{FZMDE3LSPVOF2LQ5KLUU6Z3LD5}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches. Specifically, we train GPT-3, an autoregressive language model with 175 billion parameters, 10x more than any previous non-sparse language model, and test its performance in the few-shot setting.

C2weakest assumption

That the few-shot examples placed in the prompt allow genuine generalization rather than the model simply recalling near-duplicates from its web-scale training corpus, and that the chosen evaluation tasks are not contaminated by that corpus.

C3one line summary

GPT-3 shows that scaling an autoregressive language model to 175 billion parameters enables strong few-shot performance across diverse NLP tasks via in-context prompting without fine-tuning.

References

86 extracted · 86 resolved · 27 Pith anchors

[1] Massively multilingual neural machine translation 2019

[2] arXiv preprint arXiv:2005.14050 , year = 2005

[3] Semantic parsing on freebase from question-answer pairs 2013

[4] 2004.10151 , archiveprefix = 2004

[5] PIQA: Reasoning about Physical Commonsense in Natural Language 1911 · arXiv:1911.11641

Cited by

433 papers in Pith

RPAM: A Principled Metric for Evaluating Associations in Language Models with High Predictive Validity in Downstream Outputs

Persona Cartography: Charting Language Model Personality Traits in Weight Space

From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents

COBART: Controlled, Optimized, Bidirectional and Auto-Regressive Transformer for Ad Headline Generation

Rethinking Code Performance Benchmarks for LLMs

Receipt and verification

First computed	2026-07-05T01:21:37.693472Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

2e58326d727d5c5d2e1d52e94f676b1f7d9bb1e34d3edaa3506004c4be861e15

Aliases

arxiv: 2005.14165 · arxiv_version: 2005.14165v4 · doi: 10.48550/arxiv.2005.14165 · pith_short_12: FZMDE3LSPVOF · pith_short_16: FZMDE3LSPVOF2LQ5 · pith_short_8: FZMDE3LS

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/FZMDE3LSPVOF2LQ5KLUU6Z3LD5 \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 2e58326d727d5c5d2e1d52e94f676b1f7d9bb1e34d3edaa3506004c4be861e15

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "dd4f1ea04b346c922e20a115ded0b792e59dd2c63c094228c64c13601df96e76",
    "cross_cats_sorted": [],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2020-05-28T17:29:03Z",
    "title_canon_sha256": "70b62cc6c025dabeba9b7459e0825b9cc8e3c74057130bb182bdfc6a5cb482f5"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2005.14165",
    "kind": "arxiv",
    "version": 4
  }
}