Pith Number

pith:XR2UHLWN

pith:2024:XR2UHLWNHHFSD4ZJJXWRU6R6QP

not attested not anchored not stored refs pending

MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs

Cong Xie, Ding Zhou, Haibin Lin, Haohan Xu, Haoran Wei, Hongmin Chen, Jianxi Ye, Leqi Zou, Liang Xiang, Pengfei Nie, Qi Hou, Qi Huang, Shibiao Nong, Shipeng Yan, Sida Zhao, Sun He, Xiang Li, Xiaoying Jia, Xin Jin, Xin Liu, Yanghua Peng, Yangrui Chen, Yinmin Zhong, Yiyao Sheng, Yulu Jia, Zhang Zhang, Zhe Li, Zherui Liu, Zhihao Bai, Zhi Zhang, Zhuo Jiang, Ziheng Jiang

arxiv:2402.15627 v1 · 2024-02-23 · cs.LG · cs.DC

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{XR2UHLWNHHFSD4ZJJXWRU6R6QP}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Cited by

6 papers in Pith

PipeFusion: Patch-level Pipeline Parallelism for Diffusion Transformers Inference

Instant GPU Efficiency Visibility at Fleet Scale

Chameleon: Adaptive Fault Tolerance for Distributed Training via Real-time Policy Selection

MegaScale-Omni: A Hyper-Scale, Workload-Resilient System for MultiModal LLM Training in Production

TACO: Efficient Communication Compression of Intermediate Tensors for Scalable Tensor-Parallel LLM Training

Receipt and verification

First computed	2026-07-05T07:48:55.792483Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

bc7543aecd39cb21f3294ded1a7a3e83e3d13c5d82445f0e623f0f8f8faf88f6

Aliases

arxiv: 2402.15627 · arxiv_version: 2402.15627v1 · doi: 10.48550/arxiv.2402.15627 · pith_short_12: XR2UHLWNHHFS · pith_short_16: XR2UHLWNHHFSD4ZJ · pith_short_8: XR2UHLWN

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/XR2UHLWNHHFSD4ZJJXWRU6R6QP \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: bc7543aecd39cb21f3294ded1a7a3e83e3d13c5d82445f0e623f0f8f8faf88f6

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "ce0c3f4ade76bd25b3ab5b2cd3576980c8a1f3e0d142a7c6f3f912088a62ca84",
    "cross_cats_sorted": [
      "cs.DC"
    ],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.LG",
    "submitted_at": "2024-02-23T22:10:59Z",
    "title_canon_sha256": "965ecaee53eed9d5aab76f4c165b841a1f89677bff99362b775bbc76a9d427f7"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2402.15627",
    "kind": "arxiv",
    "version": 1
  }
}