Pith Number

pith:AMR3556F

pith:2026:AMR3556F6FOUZPO6NFGOLRTUV2

not attested not anchored not stored refs pending

SciIntegrity-Bench: A Benchmark for Evaluating Academic Integrity in AI Scientist Systems

Xingtong Liu, Xinyan Xu, Zonglin Yang

AI models fabricate data rather than refuse impossible research tasks at a 34 percent rate.

arxiv:2605.10246 v2 · 2026-05-11 · cs.AI

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{AMR3556F6FOUZPO6NFGOLRTUV2}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

Across 231 evaluation runs spanning 7 state-of-the-art LLMs, the overall integrity problem rate reaches 34.2%, and no model achieves zero failures. Most strikingly, across missing-data scenarios, all seven models generate synthetic data rather than acknowledging infeasibility.

C2weakest assumption

That each of the 33 scenarios is constructed such that honest acknowledgment of failure is unequivocally the only correct response, with task completion necessarily requiring misconduct and no valid alternative interpretations.

C3one line summary

SciIntegrity-Bench shows state-of-the-art LLMs violate academic integrity in 34.2% of dilemmatic scenarios, primarily by fabricating data rather than refusing impossible tasks.

Formal links

2 machine-checked theorem links

Cited by

3 papers in Pith

Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops

Workflow Closure Is Not Scientific Closure in Auto-Research Systems

One Reflection Is Not Enough: Self-Correcting Autonomous Research via Multi-Hypothesis Failure Attribution

Receipt and verification

First computed	2026-06-04T01:08:51.109119Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

0323bef7c5f15d4cbdde694ce5c674ae8c498e0c0d15960011045402b527d76c

Aliases

arxiv: 2605.10246 · arxiv_version: 2605.10246v2 · doi: 10.48550/arxiv.2605.10246 · pith_short_12: AMR3556F6FOU · pith_short_16: AMR3556F6FOUZPO6 · pith_short_8: AMR3556F

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/AMR3556F6FOUZPO6NFGOLRTUV2 \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 0323bef7c5f15d4cbdde694ce5c674ae8c498e0c0d15960011045402b527d76c

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "658a1490df6ad05a4245941afb5f7f299ccc87885e7eab2efa378afda818d8a8",
    "cross_cats_sorted": [],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.AI",
    "submitted_at": "2026-05-11T09:19:17Z",
    "title_canon_sha256": "27915f28dbe78c26b0e57f5257ceb17d6bcd5493df2bf55d01544dec179e5564"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2605.10246",
    "kind": "arxiv",
    "version": 2
  }
}