Pith Number

pith:FCKZ6JJK

pith:2026:FCKZ6JJKVWJM35PNRHLWIGEFAZ

not attested not anchored not stored refs pending

Quantifying and Mitigating Self-Preference Bias of LLM Judges

Chuxian Qiu, Jinming Yang, Tao Zhou, Xinshan Jiao, Zheng Hu, Zhenyu Deng

LLM judges show self-preference bias uncorrelated with capability, but a multi-dimensional strategy reduces it by 31.5 percent.

arxiv:2604.22891 v4 · 2026-04-24 · cs.LG · cs.AI · cs.CL

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{FCKZ6JJKVWJM35PNRHLWIGEFAZ}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

Empirical analysis across 20 mainstream LLMs reveals that advanced capabilities are often uncorrelated, or even negatively correlated, with low SPB. To mitigate this bias, we propose a structured multi-dimensional evaluation strategy grounded in cognitive load decomposition, which reduces SPB by 31.5% on average.

C2weakest assumption

The constructed pairs of responses truly have negligible quality differences, allowing statistical separation of bias propensity from genuine discriminability without human gold standards.

C3one line summary

An automated framework using equal-quality response pairs quantifies self-preference bias in LLM judges and reduces it by 31.5% via a cognitive-load-based multi-dimensional evaluation strategy.

Formal links

2 machine-checked theorem links

Receipt and verification

First computed	2026-06-03T01:05:50.628659Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

28959f252aad92cdf5ed89d76418850669e7c6d48051135a8709401c7fa1e9f0

Aliases

arxiv: 2604.22891 · arxiv_version: 2604.22891v4 · doi: 10.48550/arxiv.2604.22891 · pith_short_12: FCKZ6JJKVWJM · pith_short_16: FCKZ6JJKVWJM35PN · pith_short_8: FCKZ6JJK

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/FCKZ6JJKVWJM35PNRHLWIGEFAZ \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 28959f252aad92cdf5ed89d76418850669e7c6d48051135a8709401c7fa1e9f0

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "0e61c9836c571c203405ee11af53a5bc9430944ca9704f8f9f98e09b091f3d38",
    "cross_cats_sorted": [
      "cs.AI",
      "cs.CL"
    ],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.LG",
    "submitted_at": "2026-04-24T09:46:22Z",
    "title_canon_sha256": "4d4d23e09121506db1381c0fccf13bf15848e72874277c48229f8414b8ea0d8d"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2604.22891",
    "kind": "arxiv",
    "version": 4
  }
}