Pith Number

pith:UPAT3JQC

pith:2024:UPAT3JQCIUJVCYHS3ACGTXFFEP

not attested not anchored not stored refs resolved

Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2

Anca Dragan, Arthur Conmy, J\'anos Kram\'ar, Lewis Smith, Neel Nanda, Nicolas Sonnerat, Rohin Shah, Senthooran Rajamanoharan, Tom Lieberum, Vikrant Varma

Gemma Scope releases JumpReLU sparse autoencoders for every layer and sub-layer of the Gemma 2 2B and 9B models.

arxiv:2408.05147 v2 · 2024-08-09 · cs.LG · cs.AI · cs.CL

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{UPAT3JQCIUJVCYHS3ACGTXFFEP}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

We introduce Gemma Scope, an open suite of JumpReLU SAEs trained on all layers and sub-layers of Gemma 2 2B and 9B and select layers of Gemma 2 27B base models.

C2weakest assumption

That standard reconstruction and sparsity metrics are sufficient to establish that the released SAEs will be useful for downstream interpretability and safety research.

C3one line summary

Gemma Scope supplies trained sparse autoencoders for all layers of Gemma 2 2B and 9B plus select 27B layers, with public weights and benchmark scores.

References

14 extracted · 14 resolved · 4 Pith anchors

[1] Towards Automated Circuit Discovery for Mechanistic Interpretability 2024

[2] Gemini: A Family of Highly Capable Multimodal Models 2024 · arXiv:2312.11805

[3] Universal neurons in gpt2 language models 2024

[4] Adam: A Method for Stochastic Optimization 2023 · arXiv:1412.6980

[5] Efficient Estimation of Word Representations in Vector Space 2005 · doi:10.18653/v1/2023.blackboxnlp-1

Cited by

33 papers in Pith

Mechanistic Interpretability of EEG Foundation Models via Sparse Autoencoders

Beyond Cross-Modal Alignment: Measuring and Leveraging Modality Gap in Vision-Language Models

Reading Task Failure Off the Activations: A Sparse-Feature Audit of GPT-2 Small on Indirect Object Identification

DECO: Sparse Mixture-of-Experts with Dense-Comparable Performance on End-Side Devices

WriteSAE: Sparse Autoencoders for Recurrent State

Receipt and verification

First computed	2026-05-17T23:38:53.348400Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

a3c13da60245135160f2d80469dca523f04f957f0dc80cc61c3134d1d39b30ff

Aliases

arxiv: 2408.05147 · arxiv_version: 2408.05147v2 · doi: 10.48550/arxiv.2408.05147 · pith_short_12: UPAT3JQCIUJV · pith_short_16: UPAT3JQCIUJVCYHS · pith_short_8: UPAT3JQC

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/UPAT3JQCIUJVCYHS3ACGTXFFEP \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: a3c13da60245135160f2d80469dca523f04f957f0dc80cc61c3134d1d39b30ff

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "f00bffffe26e5b1c282893880845be5db0e2652950fad224bec2c430385fd3da",
    "cross_cats_sorted": [
      "cs.AI",
      "cs.CL"
    ],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.LG",
    "submitted_at": "2024-08-09T16:06:42Z",
    "title_canon_sha256": "a1c5ea48af15d6dd66bb04ab12f2de102010e702c9478725bb6d6659c541ae0b"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2408.05147",
    "kind": "arxiv",
    "version": 2
  }
}