Pith Number

pith:Q54ABSZ3

pith:2026:Q54ABSZ3GIQGJCWNTEOIVCHJKV

not attested not anchored not stored refs resolved

A Hierarchical Language Model with Predictable Scaling Laws and Provable Benefits of Reasoning

Allan Sly, Elchanan Mossel, Frederic Koehler, Jason Gaitonde, Joonhyung Shin

Bounded-context autoregressive models require linear context to sample hierarchical languages faithfully, while reasoning models succeed with only logarithmic memory.

arxiv:2605.13687 v1 · 2026-05-13 · cs.LG · cs.AI · stat.ML

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{Q54ABSZ3GIQGJCWNTEOIVCHJKV}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

an autoregressive reasoning model with only Θ(log n) working memory can sample exactly from the true language — an exponential improvement.

C2weakest assumption

The exact k-gram ansatz serves as a faithful substitute for transformers with context length k; this substitution is central to all derivations and is only validated empirically rather than proven.

C3one line summary

Hierarchical synthetic languages require Ω(n) context length for faithful autoregressive sampling but only Θ(log n) working memory with reasoning for exact generation from the true distribution.

References

16 extracted · 16 resolved · 0 Pith anchors

[1] In this case, with probabilityρ, the root of the subtree remains the same as the root of the full tree and we get the same quantity of the subtree one level below

[2] There ared·(d−1)choices for them (where the first choice is taken twice), and given this choice, there are three permutations

[3] Conditioned on the root, each retains the signal with probabilityρindependently, and given they all do, the corresponding means areM d,ρ,h−1(1) = (dρ)h−1

[4] • Ifr= (1,· · ·,1), then we recursively sample through the broadcast channelκ X∅, X(1), X(1,1),· · ·, X r

[5] We sample a(d, w, κ, δ Xr)-language and setY ′ to be the sampled leaves

Receipt and verification

First computed	2026-05-18T02:44:16.984366Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

877800cb3b3220648acd991c8a88e95547e9984c594131dc1b0ac231fc644a4a

Aliases

arxiv: 2605.13687 · arxiv_version: 2605.13687v1 · doi: 10.48550/arxiv.2605.13687 · pith_short_12: Q54ABSZ3GIQG · pith_short_16: Q54ABSZ3GIQGJCWN · pith_short_8: Q54ABSZ3

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/Q54ABSZ3GIQGJCWNTEOIVCHJKV \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 877800cb3b3220648acd991c8a88e95547e9984c594131dc1b0ac231fc644a4a

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "c5346584ee4e8c5ba1eb6df317d00ddfb89ab9d1990303db47278d79f8bc7f43",
    "cross_cats_sorted": [
      "cs.AI",
      "stat.ML"
    ],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.LG",
    "submitted_at": "2026-05-13T15:42:26Z",
    "title_canon_sha256": "aad599750dd22b88303927a7baeecd30a60ca37ee333cd4ae88f04f2c0ad292c"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.13687",
    "kind": "arxiv",
    "version": 1
  }
}