Pith Number

pith:RQHZAYWW

pith:2024:RQHZAYWWV6IWIKPHD7XLFKSSAP

not attested not anchored not stored refs pending

Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders

Arthur Conmy, J\'anos Kram\'ar, Neel Nanda, Nicolas Sonnerat, Senthooran Rajamanoharan, Tom Lieberum, Vikrant Varma

JumpReLU sparse autoencoders deliver higher reconstruction fidelity than Gated or TopK SAEs at matched sparsity on Gemma 2 activations.

arxiv:2407.14435 v3 · 2024-07-19 · cs.LG

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{RQHZAYWWV6IWIKPHD7XLFKSSAP}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

JumpReLU SAEs achieve state-of-the-art reconstruction fidelity at a given sparsity level on Gemma 2 9B activations, compared to other recent advances such as Gated and TopK SAEs, and this improvement does not come at the cost of interpretability.

C2weakest assumption

That straight-through estimators applied to the discontinuous JumpReLU produce gradients that reliably optimize the intended sparse reconstruction objective rather than converging to a different local minimum.

C3one line summary

JumpReLU SAEs deliver higher reconstruction fidelity than Gated or TopK SAEs at fixed sparsity on Gemma 2 9B activations while preserving interpretability.

Formal links

2 machine-checked theorem links

Cited by

29 papers in Pith

Geometry-Adaptive Explainer for Faithful Dictionary-Based Interpretability under Distribution Shift

SwordBench: Evaluating Orthogonality of Steering Image Representations

To Call or Not to Call: Diagnosing Intrinsic Over-Calling Bias in LLM Agents

Mechanistic Interpretability with Sparse Autoencoder Neural Operators

Safe-SAIL: Towards a Fine-grained Safety Landscape of Large Language Models via Sparse Autoencoder Interpretation Framework

Receipt and verification

First computed	2026-05-17T23:38:50.468390Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

8c0f9062d6af916429e71feeb2aa5203da4f5109a78c81c141926122bf5f2fee

Aliases

arxiv: 2407.14435 · arxiv_version: 2407.14435v3 · doi: 10.48550/arxiv.2407.14435 · pith_short_12: RQHZAYWWV6IW · pith_short_16: RQHZAYWWV6IWIKPH · pith_short_8: RQHZAYWW

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/RQHZAYWWV6IWIKPHD7XLFKSSAP \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 8c0f9062d6af916429e71feeb2aa5203da4f5109a78c81c141926122bf5f2fee

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "b50802e452220cb41cc41233ea1ba2f2115e3db7957ec175d305f65ac0749979",
    "cross_cats_sorted": [],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.LG",
    "submitted_at": "2024-07-19T16:07:19Z",
    "title_canon_sha256": "ddba3bc248051a0a77b174db45bc08eacc7bda1175ce1108901b70d6287f634e"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2407.14435",
    "kind": "arxiv",
    "version": 3
  }
}