Pith Number

pith:2NZEOYYH

pith:2024:2NZEOYYHNW56QZH6ASGBYRYHUC

not attested not anchored not stored refs pending

RRM: Robust Reward Model Training Mitigates Reward Hacking

Abe Ittycheriah, Anastasiia Makarova, Aviral Kumar, Bilal Piot, Daniel Sohn, Jeremiah Liu, Jiaming Shen, Jie Ren, Junru Wu, Lichang Chen, Mohammad Saleh, Rishabh Joshi, Tianhe Yu, Tianqi Liu, Wei Xiong, Yang Gao, Yuan Liu, Zhen Qin

arxiv:2409.13156 v2 · 2024-09-20 · cs.CL

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{2NZEOYYHNW56QZH6ASGBYRYHUC}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Cited by

6 papers in Pith

Uncertainty-Aware Reward Modeling for Stable RLHF

Proxy Reward Internalization and Mechanistic Exploitation: A Learned Precursor to Reward Hacking and Its Generalization

The Hidden Bias of Process Reward Models:PRISM for Rewarding the Right Reasoning

Factored Causal Representation Learning for Robust Reward Modeling in RLHF

Teach a Reward Model to Correct Itself: Reward Guided Adversarial Failure Discovery for Robust Reward Modeling

Receipt and verification

First computed	2026-07-05T10:21:04.427259Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

d3724763076dbbe864fe048c1c4707a09c4ec3d5a6e0bb29b0d2ce20be38cb60

Aliases

arxiv: 2409.13156 · arxiv_version: 2409.13156v2 · doi: 10.48550/arxiv.2409.13156 · pith_short_12: 2NZEOYYHNW56 · pith_short_16: 2NZEOYYHNW56QZH6 · pith_short_8: 2NZEOYYH

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/2NZEOYYHNW56QZH6ASGBYRYHUC \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: d3724763076dbbe864fe048c1c4707a09c4ec3d5a6e0bb29b0d2ce20be38cb60

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "50761d2109bcefefca08f3f152f8c6b0689ec38af078e9b1dd2b36ab879170d7",
    "cross_cats_sorted": [],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2024-09-20T01:46:07Z",
    "title_canon_sha256": "ae68b1fe4ccfcb9d210c520dd01b2a65ef27b945538f11c44135b54efbfcf063"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2409.13156",
    "kind": "arxiv",
    "version": 2
  }
}