Pith Number

pith:BNQSVT3X

pith:2023:BNQSVT3XLHJMNRJGH6HSF73VSM

not attested not anchored not stored refs pending

OpsEval: A Comprehensive IT Operations Benchmark Suite for Large Language Models

Bohan Chen, Changhua Pei, Dan Pei, Gaogang Xie, Haiming Zhang, Jianhui Li, Kun Wang, Longlong Xu, Minghua Ma, Mingze Sun, Shenglin Zhang, Xiaohui Nie, Xidao Wen, Yongqian Sun, Yuhe Liu, Zhirui Zhang

arxiv:2310.07637 v5 · 2023-10-11 · cs.AI · cs.NI

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{BNQSVT3XLHJMNRJGH6HSF73VSM}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Cited by

5 papers in Pith

A Multi-Dataset Benchmark for Evaluating LLM Agents in Microservice Failure Diagnosis

SREGym: A Live Benchmark for AI SRE Agents with High-Fidelity Failure Scenarios

OpsLLM: Construction of Large Language Model for Software Operations with Multi-stage Learning

SREGym: A Live Benchmark for AI SRE Agents with High-Fidelity Failure Scenarios

OpsLLM: Construction of Large Language Model for Software Operations with Multi-stage Learning

Receipt and verification

First computed	2026-07-05T11:22:28.911015Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

0b612acf7759d2c6c5263f8f22ff759337bdcfb745141dbff634dd808caefa4e

Aliases

arxiv: 2310.07637 · arxiv_version: 2310.07637v5 · doi: 10.48550/arxiv.2310.07637 · pith_short_12: BNQSVT3XLHJM · pith_short_16: BNQSVT3XLHJMNRJG · pith_short_8: BNQSVT3X

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/BNQSVT3XLHJMNRJGH6HSF73VSM \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 0b612acf7759d2c6c5263f8f22ff759337bdcfb745141dbff634dd808caefa4e

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "03402873e46989fbf41525cc887817afe7960135593c3f50cd86af6a69e5f423",
    "cross_cats_sorted": [
      "cs.NI"
    ],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.AI",
    "submitted_at": "2023-10-11T16:33:29Z",
    "title_canon_sha256": "df284e98b36e1071928033a89a010a670d3ed765bd49e5c3d23d68a452ed528d"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2310.07637",
    "kind": "arxiv",
    "version": 5
  }
}