Pith Number

pith:EE4GJ4QH

pith:2026:EE4GJ4QH7BJJGQYHBHD6BCJ3F6

not attested not anchored not stored refs pending

ORACLE-SWE: Quantifying the Contribution of Oracle Information Signals on SWE Agents

Chaoyun Zhang, Dongmei Zhang, Elsie Nallipogu, Kenan Li, Liao Zhu, Qingwei Lin, Qirui Jin, Saravan Rajmohan, Wenke Lee, Xiaosong Huang, Xin Zhang, Yijia Wu, Yikai Zhang, Yufan Huang, Yu Kang, Zijian Jin

Oracle-SWE isolates perfect versions of key signals from SWE benchmarks to measure their separate effects on agent success rates.

arxiv:2604.07789 v2 · 2026-04-09 · cs.MA · cs.CL · cs.SE

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{EE4GJ4QH7BJJGQYHBHD6BCJ3F6}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

We introduce Oracle-SWE, a unified method to isolate and extract oracle information signals from SWE benchmarks and quantify the impact of each signal on agent performance.

C2weakest assumption

That isolating signals as perfect oracles and measuring performance gains accurately reflects their real-world contribution where signals are noisy, interdependent, and obtained imperfectly by agents.

C3one line summary

ORACLE-SWE isolates oracle signals such as reproduction tests, regression tests, edit locations, execution context, and API usage from SWE benchmarks to quantify their individual contributions to agent performance.

Receipt and verification

First computed	2026-05-29T01:05:09.004828Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

213864f207f85293430709c7e0893b2f918eb32a02aa4af1394e876c7487ef56

Aliases

arxiv: 2604.07789 · arxiv_version: 2604.07789v2 · doi: 10.48550/arxiv.2604.07789 · pith_short_12: EE4GJ4QH7BJJ · pith_short_16: EE4GJ4QH7BJJGQYH · pith_short_8: EE4GJ4QH

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/EE4GJ4QH7BJJGQYHBHD6BCJ3F6 \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 213864f207f85293430709c7e0893b2f918eb32a02aa4af1394e876c7487ef56

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "ecb1ee58f75ae6650058a15e0d74739ee37c69b21bfcaaac6283bbe96b3d978f",
    "cross_cats_sorted": [
      "cs.CL",
      "cs.SE"
    ],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.MA",
    "submitted_at": "2026-04-09T04:37:24Z",
    "title_canon_sha256": "fc9fcf02cc8a477b5ae738b369d31e19308b2043ae7b8e5d1c65e59cf33b5641"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2604.07789",
    "kind": "arxiv",
    "version": 2
  }
}