Pith Number

pith:W5AW73QH

pith:2022:W5AW73QHXHRS4ZAC566MZ6OMRR

not attested not anchored not stored refs resolved

Teaching language models to support answers with verified quotes

Francis Song, Geoffrey Irving, Jacob Menick, John Aslanides, Lucy Campbell-Gillingham, Maja Trebacz, Martin Chadwick, Mia Glaese, Nat McAleese, Susannah Young, Vladimir Mikulik

A 280 billion parameter model can be trained to answer questions with specific cited evidence from documents and to abstain when uncertain.

arxiv:2203.11147 v1 · 2022-03-21 · cs.CL · cs.LG

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{W5AW73QHXHRS4ZAC566MZ6OMRR}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

Our 280 billion parameter model, GopherCite, is able to produce answers with high quality supporting evidence and abstain from answering when unsure. The model's response is found to be high-quality 80% of the time on this Natural Questions subset, and 67% of the time on the ELI5 subset. Abstaining from the third of questions for which it is most unsure improves performance to 90% and 80% respectively.

C2weakest assumption

That human raters' preferences for 'high quality supporting evidence' during RLHP training generalize to produce reliable citations and that the model's internal uncertainty signal for abstention is well-calibrated without introducing new biases.

C3one line summary

GopherCite produces answers with supporting evidence citations, rated high-quality 80% of the time on Natural Questions and 67% on ELI5, improving to 90% and 80% with abstention on uncertain questions.

References

14 extracted · 14 resolved · 0 Pith anchors

[1] ISBN 9781450349147 2018 · doi:10.1145/3041021.3053375

[2] road draft tube 2018 · doi:10.1177/0894439317715434

[3] {url} • {claim} See this fragment from "{title}"[1]: {quote}

[4] {quote}" Source:

[5] What happens if you smash a mirror? 2021

Cited by

30 papers in Pith

Solving math word problems with process- and outcome-based feedback

Gemini: A Family of Highly Capable Multimodal Models

Reinforcement Learning from Human Feedback

ZeroSearch: Incentivize the Search Capability of LLMs without Searching

LLM-Oriented Information Retrieval: A Denoising-First Perspective

Receipt and verification

First computed	2026-05-17T23:38:14.206266Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

b7416fee07b9e32e6402efbcccf9cc8c75905c39fdbe93293cc1ef5a6e1101d5

Aliases

arxiv: 2203.11147 · arxiv_version: 2203.11147v1 · doi: 10.48550/arxiv.2203.11147 · pith_short_12: W5AW73QHXHRS · pith_short_16: W5AW73QHXHRS4ZAC · pith_short_8: W5AW73QH

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/W5AW73QHXHRS4ZAC566MZ6OMRR \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: b7416fee07b9e32e6402efbcccf9cc8c75905c39fdbe93293cc1ef5a6e1101d5

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "3f27c4af1e562a931e61026b868ea299ed8a7336b9e159d0dbe738400d973823",
    "cross_cats_sorted": [
      "cs.LG"
    ],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2022-03-21T17:26:29Z",
    "title_canon_sha256": "13c59520008b99b844030d7803e45e1caa24dcb0fc66e41169eb41631e273ec5"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2203.11147",
    "kind": "arxiv",
    "version": 1
  }
}