Pith Number

pith:PNACBSKQ

pith:2026:PNACBSKQK6PWNDISW4IVMIXHLN

not attested not anchored not stored refs resolved

When AI reviews science: Can we trust the referee?

Hang Xu, Jialiang Wang, Kaichun Hu, Kui Ren, Lei Chen, Linan Yue, Min-Ling Zhang, Shimin Di, Wangze Ni, Yuchen Liu

AI peer review is vulnerable to manipulation by hidden prompts, prestige framing, and rebuttal sycophancy.

arxiv:2604.23593 v1 · 2026-04-26 · cs.AI

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{PNACBSKQK6PWNDISW4IVMIXHLN}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

Together, this taxonomy and experimental audit provide an evidence-based baseline for assessing and tracking the reliability of AI peer review and highlight concrete failure points to guide targeted, testable mitigations.

C2weakest assumption

That the causal effects observed in the four treatment-control probes on a stratified set of ICLR 2025 submissions using two specific advanced LLMs generalize to other models, conferences, and review contexts.

C3one line summary

AI peer review systems are vulnerable to prompt injections, prestige biases, assertion strength effects, and contextual poisoning, as demonstrated by a new attack taxonomy and causal experiments on real conference submissions.

References

138 extracted · 138 resolved · 15 Pith anchors

[1] Sample I. (2025). Quality of scientific papers ques- tioned as academics “overwhelmed” by the millions published. The Guardian. https://www.theguardian. com/science/2025/jul/13/quality-of-scientific-p 2025

[2] Nature 567(7748), 305 (Mar 2019) 2025 · doi:10.1038/d41586-

[3] and Bak-Coleman J 2025 · doi:10.1038/d41586-025-01839-w

[4] and Albadawy M 2024 · doi:10.1016/j.cmpbup.2024.100145

[5] Ai4research: A survey of artificial intelligence for scientific research 2025 · doi:10.48550/arxiv.2507.01903

Receipt and verification

First computed	2026-06-02T01:04:16.429379Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

7b4020c950579f668d12b7115622e75b58cf4c156f32c8336c71d5a67346771e

Aliases

arxiv: 2604.23593 · arxiv_version: 2604.23593v1 · doi: 10.48550/arxiv.2604.23593 · pith_short_12: PNACBSKQK6PW · pith_short_16: PNACBSKQK6PWNDIS · pith_short_8: PNACBSKQ

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/PNACBSKQK6PWNDISW4IVMIXHLN \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 7b4020c950579f668d12b7115622e75b58cf4c156f32c8336c71d5a67346771e

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "ae2d03cff26dee307219f3994bfd95811261ab16f0cc6e74de21e003c6d3fa73",
    "cross_cats_sorted": [],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.AI",
    "submitted_at": "2026-04-26T08:03:32Z",
    "title_canon_sha256": "04c22a538b6099863d5142de6a9fdb1fd874196d0c3d6b9c84c4b7faf635a0d8"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2604.23593",
    "kind": "arxiv",
    "version": 1
  }
}