Pith Number

pith:PYCDSKS3

pith:2023:PYCDSKS33CWPGXPQZTU5GRC7S3

not attested not anchored not stored refs pending

Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Adam Fisch, Ahmad Beirami, Alekh Agarwal, Alex D'Amour, Chirag Nagpal, Deepak Ramachandran, Dj Dvijotham, Jacob Eisenstein, Jonathan Berant, Katherine Heller, Peter Shaw, Stephen Pfohl

arxiv:2312.09244 v3 · 2023-12-14 · cs.LG

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{PYCDSKS33CWPGXPQZTU5GRC7S3}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Cited by

23 papers in Pith

Uncertainty-Aware Reward Modeling for Stable RLHF

Proxy Reward Internalization and Mechanistic Exploitation: A Learned Precursor to Reward Hacking and Its Generalization

A Unifying Lens on Reward Uncertainty in RLHF

HARVE: Hacking-Aware Reward-Head Vector Editing for Robust Reward Models

Clearer Sight, Fewer Lies: Oriented Pickup Preference Optimization for Multimodal Hallucination Mitigation

Receipt and verification

First computed	2026-07-05T08:56:15.816481Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

7e04392a5bd8acf35df0cce9d3445f96eeee1e2aaeeeb630a6bb86c4eb5037cc

Aliases

arxiv: 2312.09244 · arxiv_version: 2312.09244v3 · doi: 10.48550/arxiv.2312.09244 · pith_short_12: PYCDSKS33CWP · pith_short_16: PYCDSKS33CWPGXPQ · pith_short_8: PYCDSKS3

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/PYCDSKS33CWPGXPQZTU5GRC7S3 \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 7e04392a5bd8acf35df0cce9d3445f96eeee1e2aaeeeb630a6bb86c4eb5037cc

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "8fee102c0fabe128d0e775b1e260f9a9adf9b90e6ca601a65d0a19c9860c7d39",
    "cross_cats_sorted": [],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.LG",
    "submitted_at": "2023-12-14T18:59:04Z",
    "title_canon_sha256": "ab9920f313ad944c347bf2ed85e2598707334ad891bad6cf207e94c66f0ae623"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2312.09244",
    "kind": "arxiv",
    "version": 3
  }
}