Paper Citation Record · LEDGER

Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

As of 8 August 2026, this Paper Citation Record lists 0 of 0 outbound references and 35 inbound Pith citation observations for arXiv:2312.09244.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2312.09244 v3

Coverage vector

measured 0 of 0 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links

measured 35 of 35 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-08T06:32:00.761636+00:00

measured 35 of 35 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-07T15:09:53.635786Z

measured 1 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-08-05T02:28:24.338817Z

Reference resolution

0 of 0 outbound references displayed

verified exact0
verified fuzzy0
unresolved0
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

2
arxiv_reference, observed 2026-08-05T02:28:24.338817Z

Outbound references

No outbound reference observations are available for this paper version.

Pith citing papers

Observation f9169d0a-e4b5-4e96-b42f-62d01dd8571c · inbound

Think-RM: Enabling Long-Horizon Reasoning in Generative Reward Models cites this paper.

Think-RM: Enabling Long-Horizon Reasoning in Generative Reward Models Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-07T15:09:53.635786Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation fbe83929-5b8f-41df-8304-74f43f8010da · inbound

T2I-Eval-R1: Reinforcement Learning-Driven Reasoning for Interpretable Text-to-Image Evaluation cites this paper.

T2I-Eval-R1: Reinforcement Learning-Driven Reasoning for Interpretable Text-to-Image Evaluation Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-07T14:42:26.655153Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 39000e2c-02cc-43e3-bb45-c39de069dde3 · inbound

Token-level Accept or Reject: A Micro Alignment Approach for Large Language Models cites this paper.

Token-level Accept or Reject: A Micro Alignment Approach for Large Language Models Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-07T14:11:30.866015Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 85ccfc5b-4c63-4600-9e91-6f20604f8e6a · inbound

Learning a Pessimistic Reward Model in RLHF cites this paper.

Learning a Pessimistic Reward Model in RLHF Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 6

Resolution: unresolved; no resolver link, observed 2026-08-07T14:01:04.764200Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3caeb390-76d5-4292-abbc-7d1102c841bb · inbound

RewardAnything: Generalizable Principle-Following Reward Models cites this paper.

RewardAnything: Generalizable Principle-Following Reward Models Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 34

Resolution: unresolved; no resolver link, observed 2026-08-07T11:04:06.833906Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0b8325f5-f3fe-49db-925f-63231a3714a4 · inbound

Activation Reward Models for Few-Shot Model Alignment cites this paper.

Activation Reward Models for Few-Shot Model Alignment Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-06T21:02:42.365297Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3366544c-778c-497b-a6c4-f82a7b6c2df9 · inbound

Best-of-N through the Smoothing Lens: KL Divergence and Regret Analysis cites this paper.

Best-of-N through the Smoothing Lens: KL Divergence and Regret Analysis Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-06T19:28:54.348620Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1fb147b1-feb1-4f5e-ab42-0d89a6116e3c · inbound

Teach a Reward Model to Correct Itself: Reward Guided Adversarial Failure Discovery for Robust Reward Modeling cites this paper.

Teach a Reward Model to Correct Itself: Reward Guided Adversarial Failure Discovery for Robust Reward Modeling Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 6

Resolution: metadata mismatch; arxiv_id, observed 2026-05-19T05:17:05.782526Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 2eceda81-b0f9-4057-8cee-6fd326be5e8f · inbound

Bradley-Terry and Multi-Objective Reward Modeling Are Complementary cites this paper.

Bradley-Terry and Multi-Objective Reward Modeling Are Complementary Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-06T18:51:28.145339Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1936df79-e57c-42bc-a82a-46f309452a11 · inbound

Bridging the Gap in Vision Language Models in Identifying Unsafe Concepts Across Modalities cites this paper.

Bridging the Gap in Vision Language Models in Identifying Unsafe Concepts Across Modalities Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 19

Resolution: unresolved; no resolver link, observed 2026-08-06T17:21:35.194359Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9d88245e-cb38-489a-962f-89e09abc6976 · inbound

CoLD: Counterfactually-Guided Length Debiasing for Process Reward Models in Mathematical Reasoning cites this paper.

CoLD: Counterfactually-Guided Length Debiasing for Process Reward Models in Mathematical Reasoning Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 4

Resolution: metadata mismatch; arxiv_id, observed 2026-05-21T23:25:45.324401Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation d2b52343-ad36-4d03-85ab-8d46ab45da4f · inbound

Towards Reliable, Uncertainty-Aware Alignment cites this paper.

Towards Reliable, Uncertainty-Aware Alignment Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-06T15:40:14.278132Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d33daff4-48b0-4d19-ad27-e641f2b9cd80 · inbound

Beyond Correctness: Harmonizing Process and Outcome Rewards through RL Training cites this paper.

Beyond Correctness: Harmonizing Process and Outcome Rewards through RL Training Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 9

Resolution: metadata mismatch; arxiv_id, observed 2026-05-21T22:40:43.264520Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 2209bca8-a672-4f0c-963f-839ffae29ba4 · inbound

Factored Causal Representation Learning for Robust Reward Modeling in RLHF cites this paper.

Factored Causal Representation Learning for Robust Reward Modeling in RLHF Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 10

Resolution: metadata mismatch; arxiv_id, observed 2026-05-21T14:20:13.678202Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 53c188b7-ccb1-488a-a157-1e3c3af0e5bf · inbound

Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation cites this paper.

Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 21

Resolution: unresolved; no resolver link, observed 2026-08-03T03:04:43.502437Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d1462243-f2f0-4201-a5d6-c14fc88c3639 · inbound

Beyond Semantic Manipulation: Token-Space Attacks on Reward Models cites this paper.

Beyond Semantic Manipulation: Token-Space Attacks on Reward Models Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 4

Resolution: metadata mismatch; arxiv_id, observed 2026-05-13T20:33:16.862524Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 647e0a61-a9b6-445e-b119-5b1e41c6d209 · inbound

FUSE: Ensembling Verifiers with Zero Labeled Data cites this paper.

FUSE: Ensembling Verifiers with Zero Labeled Data Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 28

Resolution: metadata mismatch; arxiv_id, observed 2026-05-11T12:26:09.053345Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation c2445045-e6d9-4182-b38a-59f9449287e6 · inbound

How Far Are Video Models from True Multimodal Reasoning? cites this paper.

How Far Are Video Models from True Multimodal Reasoning? Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 13

Resolution: metadata mismatch; arxiv_id, observed 2026-05-11T12:51:04.216352Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 5dba58d5-f855-4cff-9966-6e019721a62f · inbound

Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring cites this paper.

Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 5

Resolution: metadata mismatch; arxiv_id, observed 2026-05-09T19:35:39.104230Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 91e20ee9-df19-4181-bc55-03a36ca23865 · inbound

Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring cites this paper.

Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 5

Resolution: metadata mismatch; arxiv_id, observed 2026-05-11T02:15:52.406421Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 595dd3df-fe53-4938-8421-b6fd18a0d474 · inbound

Power Distribution Bridges Sampling, Self-Reward RL, and Self-Distillation cites this paper.

Power Distribution Bridges Sampling, Self-Reward RL, and Self-Distillation Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 126

Resolution: metadata mismatch; arxiv_id, observed 2026-05-11T17:56:08.381340Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation b20806bf-3933-4d79-9015-e98990d3525c · inbound

Response Time Enhances Alignment with Heterogeneous Preferences cites this paper.

Response Time Enhances Alignment with Heterogeneous Preferences Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 16

Resolution: metadata mismatch; arxiv_id, observed 2026-05-11T04:45:59.518594Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation e39bdd7b-6892-43cb-a10e-85be05c618c2 · inbound

Do Less, Achieve More: Do We Need Every-Step Optimization for RL Fine-tuning of Diffusion Models? cites this paper.

Do Less, Achieve More: Do We Need Every-Step Optimization for RL Fine-tuning of Diffusion Models? Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 6

Resolution: metadata mismatch; arxiv_id, observed 2026-05-20T19:48:57.332773Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 8342c137-ccd7-40b3-9b84-bd3eec498c32 · inbound

The Human-AI Delegation-Verification Dilemma: Individual Strategies, Collective Equilibria and Sociotechnical Lock-in cites this paper.

The Human-AI Delegation-Verification Dilemma: Individual Strategies, Collective Equilibria and Sociotechnical Lock-in Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 17

Resolution: malformed identifier; arxiv_id, observed 2026-05-21T03:43:55.658050Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation cb4d80df-6aaf-4f90-8445-5265e2833eb8 · inbound

The Human-AI Delegation-Verification Dilemma: Individual Strategies, Collective Equilibria and Sociotechnical Lock-in cites this paper.

Reference 17

Resolution: malformed identifier; arxiv_id, observed 2026-06-30T17:14:56.535054Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation c03e3297-ef02-4b98-b201-5ac4c2e9b141 · inbound

Benchmarking and Improving Monitors for Out-Of-Distribution Alignment Failure in LLMs cites this paper.

Benchmarking and Improving Monitors for Out-Of-Distribution Alignment Failure in LLMs Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 11

Resolution: verified exact; arxiv_id, observed 2026-05-22T09:41:22.293334Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 683dab4d-0563-4100-970f-496a022b5b2b · inbound

Benchmarking and Improving Monitors for Out-Of-Distribution Alignment Failure in LLMs cites this paper.

Benchmarking and Improving Monitors for Out-Of-Distribution Alignment Failure in LLMs Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 71

Resolution: verified exact; arxiv_id, observed 2026-05-22T09:41:21.380125Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 76842f9b-559e-43ae-8c8d-84afba033d7b · inbound

Benchmarking and Improving Monitors for Out-Of-Distribution Alignment Failure in LLMs cites this paper.

Benchmarking and Improving Monitors for Out-Of-Distribution Alignment Failure in LLMs Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 12

Resolution: verified exact; arxiv_id, observed 2026-06-30T17:34:58.060030Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 9ca641ca-ca54-47dc-9f8b-594dc499edfa · inbound

HARVE: Hacking-Aware Reward-Head Vector Editing for Robust Reward Models cites this paper.

HARVE: Hacking-Aware Reward-Head Vector Editing for Robust Reward Models Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 24

Resolution: metadata mismatch; arxiv_id, observed 2026-07-02T01:46:26.792724Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 16667bdb-4ecb-4f3a-816d-7d452f5c4284 · inbound

A Unifying Lens on Reward Uncertainty in RLHF cites this paper.

A Unifying Lens on Reward Uncertainty in RLHF Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 5

Resolution: verified exact; arxiv_id, observed 2026-07-03T00:27:29.842084Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 45b67a8f-0ef7-4aeb-9754-43663e65f0db · inbound

Proxy Reward Internalization and Mechanistic Exploitation: A Learned Precursor to Reward Hacking and Its Generalization cites this paper.

Proxy Reward Internalization and Mechanistic Exploitation: A Learned Precursor to Reward Hacking and Its Generalization Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 87

Resolution: metadata mismatch; arxiv_id, observed 2026-07-03T01:37:30.548957Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation ce4e1c54-6b51-44d1-9195-3b9771a503d0 · inbound

Uncertainty-Aware Reward Modeling for Stable RLHF cites this paper.

Uncertainty-Aware Reward Modeling for Stable RLHF Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 7

Resolution: metadata mismatch; arxiv_id, observed 2026-07-04T03:19:30.697631Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 8b1a7393-77d3-4d5c-8dd4-545a86f3333d · inbound

Clearer Sight, Fewer Lies: Oriented Pickup Preference Optimization for Multimodal Hallucination Mitigation cites this paper.

Clearer Sight, Fewer Lies: Oriented Pickup Preference Optimization for Multimodal Hallucination Mitigation Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 12

Resolution: metadata mismatch; arxiv_id, observed 2026-06-30T06:14:18.664913Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation c52b93cc-4f9a-46e5-8677-0fc57a8e894f · inbound

Clearer Sight, Fewer Lies: Oriented Pickup Preference Optimization for Multimodal Hallucination Mitigation cites this paper.

Clearer Sight, Fewer Lies: Oriented Pickup Preference Optimization for Multimodal Hallucination Mitigation Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 12

Resolution: metadata mismatch; arxiv_id, observed 2026-07-01T07:05:28.611341Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-08T06:32:00.761636+00:00.

Observation 1589a814-1230-4d0d-a3e6-e6e0514900d8 · inbound

What do Reward Models Memorize? cites this paper.

What do Reward Models Memorize? Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking

Reference 2020

Resolution: unresolved; no resolver link, observed 2026-07-31T13:41:42.277767Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.