Pith Number

pith:XRDMPDN3

pith:2021:XRDMPDN35CM74LPWONCUIMD6WN

not attested not anchored not stored refs pending

Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP

Arun Raja, Beno\^it Sagot, Chenglei Si, Colin Raffel, Elizabeth Salesky, Manan Dey, Matthias Gall\'e, Sabrina J. Mielke, Samson Tan, Wilson Y. Lee, Zaid Alyafeai

arxiv:2112.10508 v1 · 2021-12-20 · cs.CL · cs.LG

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{XRDMPDN35CM74LPWONCUIMD6WN}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Cited by

13 papers in Pith

MinGram: A Minimalist Unigram Tokenizer with High Compression and Competitive Morphological Alignment

Phonemes to the Rescue: Multilingual Tokenization Based on International Phonetic Alphabet

The price of incrementality in k-center clustering

Kronecker Embeddings: Byte-Level Structured Token Representations for Parameter-Efficient Language Models

Translating Signals to Languages for sEMG-Based Activity Recognition

Receipt and verification

First computed	2026-07-05T03:42:16.043988Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

bc46c78dbbe899fe2df6734544307eb369f39e2b06c9390deb847ecee83cb639

Aliases

arxiv: 2112.10508 · arxiv_version: 2112.10508v1 · doi: 10.48550/arxiv.2112.10508 · pith_short_12: XRDMPDN35CM7 · pith_short_16: XRDMPDN35CM74LPW · pith_short_8: XRDMPDN3

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/XRDMPDN35CM74LPWONCUIMD6WN \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: bc46c78dbbe899fe2df6734544307eb369f39e2b06c9390deb847ecee83cb639

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "d67fa28882069912077842f82a1f5f66ce88a5581199e0bf21ea72a28dd6ff7a",
    "cross_cats_sorted": [
      "cs.LG"
    ],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2021-12-20T13:04:18Z",
    "title_canon_sha256": "cfc2e9d664aac3bba150360940b4e93b04cb44466959dd558922b9c5b5811c2f"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2112.10508",
    "kind": "arxiv",
    "version": 1
  }
}