Pith Number

pith:GKEBQHZO

pith:2022:GKEBQHZOUDEA77XMILT44NLAZP

not attested not anchored not stored refs resolved

Locating and Editing Factual Associations in GPT

Alex Andonian, David Bau, Kevin Meng, Yonatan Belinkov

Factual associations in GPT models are stored in localized mid-layer feed-forward computations that can be directly edited via rank-one weight updates.

arxiv:2202.05262 v5 · 2022-02-10 · cs.CL · cs.LG

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{GKEBQHZOUDEA77XMILT44NLAZP}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

We find that ROME is effective on a standard zero-shot relation extraction (zsRE) model-editing task, comparable to existing methods. To perform a more sensitive evaluation, we also evaluate ROME on a new dataset of counterfactual assertions, on which it simultaneously maintains both specificity and generalization, whereas other methods sacrifice one or another.

C2weakest assumption

The assumption that the causal intervention correctly isolates the decisive feed-forward computations for factual recall, and that a rank-one update to those weights changes the association without creating unmeasured side effects on the broader distribution of model behavior.

C3one line summary

Factual associations in autoregressive transformers are localized to mid-layer feed-forward modules and can be edited via rank-one model editing while preserving both specificity and generalization on counterfactual tests.

References

43 extracted · 43 resolved · 3 Pith anchors

[1] Fine-grained analysis of sentence embeddings using auxiliary prediction tasks 2017

[2] Anderson, J. A. A simple neural network generating an interactive memory. Mathematical biosciences, 14 0 (3-4): 0 197--220, 1972 1972

[3] Rewriting a deep generative model 2020

[4] Probing Classifiers: Promises, Shortcomings, and Advances 2021 · doi:10.1162/coli_a_00422

[5] Analysis methods in neural language processing: A survey 2019 · doi:10.1162/tacl_a_00254

Formal links

2 machine-checked theorem links

Cited by

28 papers in Pith

GPT-NeoX-20B: An Open-Source Autoregressive Language Model

Learning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillation

From Correlation to Cause: A Five-Stage Methodology for Feature Analysis in Transformer Language Models

Painless Activation Steering: An Automated, Lightweight Approach for Post-Training Large Language Models

Meltdown: Circuits and Bifurcations in Point-Cloud-Conditioned 3D Diffusion Transformers

Receipt and verification

First computed	2026-05-17T23:38:48.267193Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

3288181f2ea0c80ffeec42e7ce3560cbc408fdfa9fc77f12611401d53aadf9a5

Aliases

arxiv: 2202.05262 · arxiv_version: 2202.05262v5 · doi: 10.48550/arxiv.2202.05262 · pith_short_12: GKEBQHZOUDEA · pith_short_16: GKEBQHZOUDEA77XM · pith_short_8: GKEBQHZO

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/GKEBQHZOUDEA77XMILT44NLAZP \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 3288181f2ea0c80ffeec42e7ce3560cbc408fdfa9fc77f12611401d53aadf9a5

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "4b53bc8ae790872206209c4c8d74e42d44cc03ad4c0074f1bcba9a2de9b43519",
    "cross_cats_sorted": [
      "cs.LG"
    ],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2022-02-10T18:59:54Z",
    "title_canon_sha256": "5ce9b4bbd2053dd244ea67114c94a80075b0edf61ce03c55d302752266c983ec"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2202.05262",
    "kind": "arxiv",
    "version": 5
  }
}