Pith Number

pith:DIYGYPBP

pith:2026:DIYGYPBPELPZL2USZDUPXAUCO2

not attested not anchored not stored refs pending

On the Expressive Power of Contextual Relations in Transformers

Demi\'an Fraiman

Transformers can approximate any contextual relation by treating it as a probability distribution or coupling.

arxiv:2603.25860 v3 · 2026-03-26 · stat.ML · cs.LG

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{DIYGYPBPELPZL2USZDUPXAUCO2}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

we establish a universal approximation theorem for contextual systems using standard Softmax Attention and alternately Sinkhorn normalization. These results show that Transformer architectures can approximate arbitrary contextual relations rules, and that the choice of normalization determines how these relations are represented.

C2weakest assumption

Contextual relations can be fully and faithfully modeled as probabilistic objects, either as conditional distributions or as joint distributions (couplings), and that this modeling captures what Transformers actually compute.

C3one line summary

Transformers using softmax or Sinkhorn attention can universally approximate any contextual relation modeled as a probabilistic coupling or conditional distribution.

Cited by

2 papers in Pith

One Operator for Many Densities: Amortized Approximation of Conditioning by Neural Operators

Receipt and verification

First computed	2026-05-20T00:03:09.568965Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

1a306c3c2f22df95ea92c8e8fb828276946d0edc0c8babaa8eca278235aae91a

Aliases

arxiv: 2603.25860 · arxiv_version: 2603.25860v3 · doi: 10.48550/arxiv.2603.25860 · pith_short_12: DIYGYPBPELPZ · pith_short_16: DIYGYPBPELPZL2US · pith_short_8: DIYGYPBP

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/DIYGYPBPELPZL2USZDUPXAUCO2 \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 1a306c3c2f22df95ea92c8e8fb828276946d0edc0c8babaa8eca278235aae91a

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "d2465bc01ba689734b1490f1615a780cba58c267db80c2714de7180db49df244",
    "cross_cats_sorted": [
      "cs.LG"
    ],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "stat.ML",
    "submitted_at": "2026-03-26T19:30:36Z",
    "title_canon_sha256": "329b9ea9b336ce35e190fd72b669845d80e143f20af353d534cb05f38058f467"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2603.25860",
    "kind": "arxiv",
    "version": 3
  }
}