The reviewed record of science sign in
Pith
Pith Number

pith:I7R2ORFJ

pith:2023:I7R2ORFJJS2MTPLW4MU4YAAJS7
not attested not anchored not stored refs pending

ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code

Arman Cohan, Baobao Chang, Haozhe Zhao, Helan hu, Junjie Lu, Kaikai An, Liang Chen, Mark Gerstein, Ruijun Huang, Sheng Chen, Shuzheng Si, Tianyu Liu, Wangchunshu Zhou, Xiangru Tang, Yanjun Shao, Yan Wang, Yichi Zhang, Yilun Zhao, Yin Fang, Yujia Qin, Yuliang Liu, Zefan Cai, Zexuan Deng, Zhiwei Jiang

arxiv:2311.09835 v5 · 2023-11-16 · cs.CL · cs.AI

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?
Add to your LaTeX paper 
\usepackage{pith}
\pithnumber{I7R2ORFJJS2MTPLW4MU4YAAJS7}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp
2 Internet Archive
3 Author claim open · sign in to claim
4 Citations open
5 Replications open
✓ Portable graph bundle live · download bundle · merged state
The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Cited by

7 papers in Pith

ProfiliTable: Profiling-Driven Tabular Data Processing via Agentic Workflows
MLS-Bench: A Holistic and Rigorous Assessment of AI Systems on Building Better AI
MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
From Standalone LLMs to Integrated Intelligence: A Survey of Compound Al Systems
ProfiliTable: Profiling-Driven Tabular Data Processing via Agentic Workflows
Receipt and verification
First computed 2026-07-05T08:57:34.749564Z
Builder pith-number-builder-2026-05-17-v1
Signature Pith Ed25519 (pith-v1-2026-05) · public key
Schema pith-number/v1.0

Canonical hash

47e3a744a94cb4c9bd76e329cc000997c78e6c16db887ef51f2d10e8cb522125

Aliases

arxiv: 2311.09835 · arxiv_version: 2311.09835v5 · doi: 10.48550/arxiv.2311.09835 · pith_short_12: I7R2ORFJJS2M · pith_short_16: I7R2ORFJJS2MTPLW · pith_short_8: I7R2ORFJ
Agent API
Resolver JSON Graph JSON Events JSON Schema Signing key
Verify this Pith Number yourself 
curl -sH 'Accept: application/ld+json' https://pith.science/pith/I7R2ORFJJS2MTPLW4MU4YAAJS7 \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 47e3a744a94cb4c9bd76e329cc000997c78e6c16db887ef51f2d10e8cb522125
Canonical record JSON 
{
  "metadata": {
    "abstract_canon_sha256": "1445b578891a6698e4f704b2ab8c63ddb62a09ce00385ac4c007fd65516d61cf",
    "cross_cats_sorted": [
      "cs.AI"
    ],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2023-11-16T12:03:21Z",
    "title_canon_sha256": "d852bbc0aead1ab4c321944655dab59321ac7bc3894c8148e4ac8e8c1fa35585"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2311.09835",
    "kind": "arxiv",
    "version": 5
  }
}