Pith Number

pith:33NV57HY

pith:2023:33NV57HYMIFM5GWDBTWSEYIN2F

not attested not anchored not stored refs resolved

Scalable Extraction of Training Data from (Production) Language Models

A. Feder Cooper, Christopher A. Choquette-Choo, Daphne Ippolito, Eric Wallace, Florian Tram\`er, Jonathan Hayase, Katherine Lee, Matthew Jagielski, Milad Nasr, Nicholas Carlini

Adversaries can extract gigabytes of training data from language models including ChatGPT by querying them without prior knowledge of the data.

arxiv:2311.17035 v1 · 2023-11-28 · cs.LG · cs.CL · cs.CR

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{33NV57HYMIFM5GWDBTWSEYIN2F}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

Our methods show practical attacks can recover far more data than previously thought, and reveal that current alignment techniques do not eliminate memorization.

C2weakest assumption

That the strings emitted by the models are verifiably present in the original training datasets rather than plausible generations, and that the divergence attack requires no prior knowledge of the training data.

C3one line summary

Adversaries can scalably extract gigabytes of training data from open, semi-open, and closed language models via querying attacks, including a divergence method that increases extraction rates 150x on aligned models like ChatGPT.

References

64 extracted · 64 resolved · 5 Pith anchors

[1] Sequential Good-Turing and the miss- ing species problem

[2] M., F IRAT, O., ET AL 2023

[3] Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback 2022 · arXiv:2204.05862

[4] Recon- structing training data with informed adversaries 2022

[5] A., P UROHIT , S., P RASHANTH , U 2023

Formal links

1 machine-checked theorem link

Cited by

38 papers in Pith

Preserving Knowledge in Large Language Model with Model-Agnostic Self-Decompression

Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Survey

Towards the Anonymization of the Language Modeling

Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety

Gemma 3 Technical Report

Receipt and verification

First computed	2026-05-17T23:38:50.501353Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

dedb5efcf8620ace9ac30ced22610dd1616a0f2592cb05ab0854df3c2d44b3c6

Aliases

arxiv: 2311.17035 · arxiv_version: 2311.17035v1 · doi: 10.48550/arxiv.2311.17035 · pith_short_12: 33NV57HYMIFM · pith_short_16: 33NV57HYMIFM5GWD · pith_short_8: 33NV57HY

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/33NV57HYMIFM5GWDBTWSEYIN2F \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: dedb5efcf8620ace9ac30ced22610dd1616a0f2592cb05ab0854df3c2d44b3c6

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "78c268ca3f6d957e3e6106181db95edd8ea82003d47c3b317c03666251909969",
    "cross_cats_sorted": [
      "cs.CL",
      "cs.CR"
    ],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.LG",
    "submitted_at": "2023-11-28T18:47:03Z",
    "title_canon_sha256": "b92f16cf18c2856205cecdb2cb789e5f9b1896bee9511d819789558b5381838d"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2311.17035",
    "kind": "arxiv",
    "version": 1
  }
}