Pith Number

pith:XZHAFFC6

pith:2022:XZHAFFC6JMMM7TVM2LSTLSKROG

not attested not anchored not stored refs pending

Multi-lingual Evaluation of Code Generation Models

Arash Farahani, Baishakhi Ray, Ben Athiwaratkun, Bing Xiang, Dan Roth, Haifeng Qian, Hantian Ding, Ming Tan, Mingyue Shang, Murali Krishna Ramanathan, Nathan Fulton, Parminder Bhatia, Qing Sun, Ramesh Nallapati, Robert Giaquinto, Sanjay Krishna Gouda, Shiqi Wang, Siddhartha Jain, Sudipta Sengupta, Sujan Kumar Gonugondla, Varun Kumar, Wasi Uddin Ahmad, Xiaopeng Li, Yuchen Tian, Zijian Wang

arxiv:2210.14868 v3 · 2022-10-26 · cs.LG · cs.CL

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{XZHAFFC6JMMM7TVM2LSTLSKROG}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Cited by

19 papers in Pith

Multi-LCB: Extending LiveCodeBench to Multiple Programming Languages

Beyond Pass Rate: A Multilingual, Execution-Grounded Evaluation of Open Code LLMs

SWE-InfraBench: Evaluating Language Models on Cloud Infrastructure Code

CodegenBench: Can LLMs Write Efficient Code Across Architectures?

Evaluating LLM-Generated Code: A Benchmark and Developer Study

Receipt and verification

First computed	2026-07-05T05:56:00.543306Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

be4e02945e4b18cfceacd2e535c95171a6c8452dc3a43c6f656c5f5e1140cebd

Aliases

arxiv: 2210.14868 · arxiv_version: 2210.14868v3 · doi: 10.48550/arxiv.2210.14868 · pith_short_12: XZHAFFC6JMMM · pith_short_16: XZHAFFC6JMMM7TVM · pith_short_8: XZHAFFC6

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/XZHAFFC6JMMM7TVM2LSTLSKROG \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: be4e02945e4b18cfceacd2e535c95171a6c8452dc3a43c6f656c5f5e1140cebd

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "843a0554b790fd983192b6f4791c963d96fd24b7d68d39245c73bf87e0cbf8d8",
    "cross_cats_sorted": [
      "cs.CL"
    ],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.LG",
    "submitted_at": "2022-10-26T17:17:06Z",
    "title_canon_sha256": "4cf2b945ed9b44810504d5f096c47add37ebfb8680bc0b28a910ac3acbf58492"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2210.14868",
    "kind": "arxiv",
    "version": 3
  }
}