Pith Number

pith:JL5RKDIC

pith:2021:JL5RKDICUQBJAFXXHOL72SV7ID

not attested not anchored not stored refs pending

Accounting for Variance in Machine Learning Benchmarks

Assya Trofimov, Brennan Nichyporuk, Chris Pal, Dmitriy Serdyuk, Edward Raff, Ga\"el Varoquaux, Justin Szeto, Kanika Madan, Mirko Bronzi, Naz Sepah, Pascal Vincent, Pierre Delaunay, Samira Ebrahimi Kahou, Tal Arbel, Vikram Voleti, Vincent Michalski, Xavier Bouthillier

arxiv:2103.03098 v1 · 2021-03-01 · cs.LG · stat.ML

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{JL5RKDICUQBJAFXXHOL72SV7ID}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Cited by

4 papers in Pith

From Forecasting Leaderboards to Deployment Decisions: A Fail-Closed Certification Protocol

A Pre-Registered Causal Partition of Self-Consistency Elicitation and Reward Design in RLVR

SafetyRepro: Configuration-Conditional Rank Instability on Alignment Benchmarks

QuickScope: Certifying Hard Questions in Dynamic LLM Benchmarks

Receipt and verification

First computed	2026-07-05T02:20:24.278242Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

4afb150d02a4029016f73b97fd4abf40c1f04fe32f3e2fe2850490d743e54e37

Aliases

arxiv: 2103.03098 · arxiv_version: 2103.03098v1 · doi: 10.48550/arxiv.2103.03098 · pith_short_12: JL5RKDICUQBJ · pith_short_16: JL5RKDICUQBJAFXX · pith_short_8: JL5RKDIC

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/JL5RKDICUQBJAFXXHOL72SV7ID \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 4afb150d02a4029016f73b97fd4abf40c1f04fe32f3e2fe2850490d743e54e37

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "9a3fa58118b1397f9df18aaf8f09bb4b2c24f65dd58941492f6e8c7f66514849",
    "cross_cats_sorted": [
      "stat.ML"
    ],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.LG",
    "submitted_at": "2021-03-01T22:39:49Z",
    "title_canon_sha256": "27a4645364d0de9f49b7663bb976996657450ce524f4a58528af07900c8427fd"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2103.03098",
    "kind": "arxiv",
    "version": 1
  }
}