Pith Number

pith:NZLN6I2X

pith:2025:NZLN6I2X2WA5FPSCBY6Q3SMJHN

not attested not anchored not stored refs pending

Autoregressive Language Models are Secretly Energy-Based Models: Insights into the Lookahead Capabilities of Next-Token Prediction

Germain Vivier-Ardisson, Mathieu Blondel, Michael E. Sander, Tianlin Liu, Vincent Roulet

Autoregressive language models are equivalent to energy-based models in function space through a bijection from the chain rule of probability.

arxiv:2512.15605 v4 · 2025-12-17 · cs.LG · stat.ML

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{NZLN6I2X2WA5FPSCBY6Q3SMJHN}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

we establish an explicit bijection between ARMs and EBMs in function space, which we show to correspond to a special case of the soft Bellman equation in maximum entropy reinforcement learning

C2weakest assumption

The chain rule of probability directly yields the claimed bijection in function space with no additional restrictions on model capacity, training dynamics, or the form of the energy function.

C3one line summary

Autoregressive language models are equivalent to energy-based models through a bijection that corresponds to the soft Bellman equation, explaining their lookahead capabilities despite next-token training.

Formal links

2 machine-checked theorem links

Cited by

3 papers in Pith

Ontology-Constrained Neural Reasoning in Enterprise Agentic Systems: A Neurosymbolic Architecture for Domain-Grounded AI Agents

LoopUS: Recasting Pretrained LLMs into Looped Latent Refinement Models

Receipt and verification

First computed	2026-05-26T02:05:04.609257Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

6e56df2357d581d2be420e3d0dc9893b5e1d80f70d01841c9f563a8685a7227c

Aliases

arxiv: 2512.15605 · arxiv_version: 2512.15605v4 · doi: 10.48550/arxiv.2512.15605 · pith_short_12: NZLN6I2X2WA5 · pith_short_16: NZLN6I2X2WA5FPSC · pith_short_8: NZLN6I2X

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/NZLN6I2X2WA5FPSCBY6Q3SMJHN \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 6e56df2357d581d2be420e3d0dc9893b5e1d80f70d01841c9f563a8685a7227c

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "54408919a33aa2c84ace259bb2729e36799fda794e784c2511a017f86d2e6e2a",
    "cross_cats_sorted": [
      "stat.ML"
    ],
    "license": "http://creativecommons.org/licenses/by-nc-sa/4.0/",
    "primary_cat": "cs.LG",
    "submitted_at": "2025-12-17T17:14:26Z",
    "title_canon_sha256": "4a21800a0e271758192b0ab58fafe5ab76d07572b85c36e0e9d446a15cf28341"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2512.15605",
    "kind": "arxiv",
    "version": 4
  }
}