Paper Citation Record · LEDGER

Reward Models in Deep Reinforcement Learning: A Survey

As of 21 August 2026, this Paper Citation Record lists 36 of 36 outbound references and 6 inbound Pith citation observations for arXiv:2506.15421.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2506.15421 v1

Coverage vector

measured 36 of 36 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-15T19:38:51.086103Z

measured 42 of 42 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-21T06:32:19.484+00:00

measured 6 of 6 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-06-27T18:08:19.797150Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-07-04T13:29:50.941267Z

Reference resolution

36 of 36 outbound references displayed

verified exact3
verified fuzzy2
unresolved31
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 49a2ba99-8f48-418d-a2b3-2d472e72612c · outbound

This paper cites Vision-Language Models as a Source of Rewards.

Reward Models in Deep Reinforcement Learning: A Survey Vision-Language Models as a Source of Rewards

Reference 4

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.445678Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4b81b121-5b55-402d-b1e7-7a72d76bf2d1 · outbound

This paper cites Diversity is All You Need: Learning Skills without a Reward Function.

Reward Models in Deep Reinforcement Learning: A Survey Diversity is All You Need: Learning Skills without a Reward Function

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.461416Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f002d8c3-1e2c-423d-8e3e-a11ad8568f40 · outbound

This paper cites Quantifying Differences in Reward Functions.

Reward Models in Deep Reinforcement Learning: A Survey Quantifying Differences in Reward Functions

Reference 10

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.478051Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0bff0728-f114-4b9d-8edc-c33ad3690e8d · outbound

This paper cites Preprocessing Reward Functions for Interpretability.

Reward Models in Deep Reinforcement Learning: A Survey Preprocessing Reward Functions for Interpretability

Reference 13

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.544516Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f4a188ef-28ba-45b9-b6d5-9f3237367050 · outbound

This paper cites Regularized Inverse Reinforcement Learning.

Reward Models in Deep Reinforcement Learning: A Survey Regularized Inverse Reinforcement Learning

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.578945Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9270cc22-8d05-44ab-bba5-0b97572f10d2 · outbound

This paper cites A survey of reinforcement learning from human feedback.arXiv preprint arXiv:2312.14925, 10,.

Reward Models in Deep Reinforcement Learning: A Survey A survey of reinforcement learning from human feedback.arXiv preprint arXiv:2312.14925, 10,

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.627819Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d6570f42-3b80-4095-b1cc-01223c13b89b · outbound

This paper cites Preference Transformer: Modeling Human Preferences using Transformers for RL.

Reward Models in Deep Reinforcement Learning: A Survey Preference Transformer: Modeling Human Preferences using Transformers for RL

Reference 16

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.639257Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation dc08f9b9-f160-4338-a95e-482f090b333c · outbound

This paper cites Empowerment: A universal agent-centric measure of control.

Reward Models in Deep Reinforcement Learning: A Survey Empowerment: A universal agent-centric measure of control

Reference 18

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T19:38:52.185603Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation fb6b9eb2-7d20-4713-b7cb-c33da20962a7 · outbound

This paper cites Goal-Conditioned Reinforcement Learning: Problems and Solutions.

Reward Models in Deep Reinforcement Learning: A Survey Goal-Conditioned Reinforcement Learning: Problems and Solutions

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.662293Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5144f0ef-b109-4920-a17b-a2d97aab60bc · outbound

This paper cites Reward Modeling with Ordinal Feedback: Wisdom of the Crowd.

Reward Models in Deep Reinforcement Learning: A Survey Reward Modeling with Ordinal Feedback: Wisdom of the Crowd

Reference 21

Resolution: verified exact; local_arxiv, observed 2026-08-15T19:38:51.656840Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation a6529e8c-1975-4c4e-86d9-64d2cd39f9a1 · outbound

This paper cites ReFT: Reasoning with Reinforced Fine-Tuning.

Reward Models in Deep Reinforcement Learning: A Survey ReFT: Reasoning with Reinforced Fine-Tuning

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.679115Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4795901f-7eab-45bf-ab6e-6c5ea4aec581 · outbound

This paper cites Choreographer: Learning and Adapting Skills in Imagination.

Reward Models in Deep Reinforcement Learning: A Survey Choreographer: Learning and Adapting Skills in Imagination

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.698388Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 61063ea3-cb44-4b2e-9e8c-769bdd690a8a · outbound

This paper cites Learning to Assist Humans without Inferring Rewards.

Reward Models in Deep Reinforcement Learning: A Survey Learning to Assist Humans without Inferring Rewards

Reference 25

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.792909Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 598ec36a-b1d1-4312-86f5-6eeb6144bcfb · outbound

This paper cites METRA: Scalable Unsupervised RL with Metric-Aware Abstraction.

Reward Models in Deep Reinforcement Learning: A Survey METRA: Scalable Unsupervised RL with Metric-Aware Abstraction

Reference 26

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.841430Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b9c43ce1-7ec8-434d-99cf-8d878f4a37cb · outbound

This paper cites Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learning.

Reward Models in Deep Reinforcement Learning: A Survey Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learning

Reference 27

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.847290Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 26e7ff2b-2682-45f3-8248-ac6aaa0779a6 · outbound

This paper cites STARC: A General Framework For Quantifying Differences Between Reward Functions.

Reward Models in Deep Reinforcement Learning: A Survey STARC: A General Framework For Quantifying Differences Between Reward Functions

Reference 28

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.852504Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation fd93ee8b-7900-4168-838c-4bfeb2a8f248 · outbound

This paper cites Llama 2: Open Foundation and Fine-Tuned Chat Models.

Reward Models in Deep Reinforcement Learning: A Survey Llama 2: Open Foundation and Fine-Tuned Chat Models

Reference 29

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.858219Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b80eeca1-34af-449e-9d5f-de88fe30a17a · outbound

This paper cites Gymnasium: A Standard Interface for Reinforcement Learning Environments.

Reward Models in Deep Reinforcement Learning: A Survey Gymnasium: A Standard Interface for Reinforcement Learning Environments

Reference 30

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.863987Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 199389b0-881b-4c00-9e30-6e75e105ec4a · outbound

This paper cites Hindsight PRIORs for Reward Learning from Human Preferences.

Reward Models in Deep Reinforcement Learning: A Survey Hindsight PRIORs for Reward Learning from Human Preferences

Reference 31

Resolution: verified exact; local_arxiv, observed 2026-08-15T19:38:51.407764Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation bea44e5a-ebf5-4d1a-a72c-bffd2ba218a8 · outbound

This paper cites Text2Reward: Reward Shaping with Language Models for Reinforcement Learning.

Reward Models in Deep Reinforcement Learning: A Survey Text2Reward: Reward Shaping with Language Models for Reinforcement Learning

Reference 33

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:51.030036Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 81ce2191-54d4-409a-8c40-53578b8eaa83 · outbound

This paper cites DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Search.

Reward Models in Deep Reinforcement Learning: A Survey DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Search

Reference 34

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:51.075704Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9dbf7c04-eef8-4c1e-83be-43a1cee1455f · outbound

This paper cites Generative Verifiers: Reward Modeling as Next-Token Prediction.

Reward Models in Deep Reinforcement Learning: A Survey Generative Verifiers: Reward Modeling as Next-Token Prediction

Reference 35

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:51.080835Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 82014311-ddbe-4e41-8db7-7b212b576b57 · outbound

This paper cites Maximum entropy inverse reinforcement learning.

Reward Models in Deep Reinforcement Learning: A Survey Maximum entropy inverse reinforcement learning

Reference 36

Resolution: verified fuzzy; raw_fallback, observed 2026-08-15T19:38:52.167651Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation b88bbc5f-199d-4b32-961a-8cb31b19f2c7 · outbound

This paper cites Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discovery.

Reward Models in Deep Reinforcement Learning: A Survey Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discovery

Reference 1950

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.489300Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e4c7d4f9-2d29-479c-9d14-f6b860535c79 · outbound

This paper cites Exploration by Random Network Distillation.

Reward Models in Deep Reinforcement Learning: A Survey Exploration by Random Network Distillation

Reference 1952

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.450710Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ebb72092-549a-4dec-84b7-b23e74c8330b · outbound

This paper cites Improving Multimodal Interactive Agents with Reinforcement Learning from Human Feedback.

Reward Models in Deep Reinforcement Learning: A Survey Improving Multimodal Interactive Agents with Reinforcement Learning from Human Feedback

Reference 2004

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.410219Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation baff5cc1-6d0e-4bf8-adc7-3bcd9acfebef · outbound

This paper cites Models of human preference for learning reward functions.

Reward Models in Deep Reinforcement Learning: A Survey Models of human preference for learning reward functions

Reference 2005

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.656340Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d8b3e62f-02a8-49eb-b66c-a8dc62d1b0ee · outbound

This paper cites DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning.

Reward Models in Deep Reinforcement Learning: A Survey DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

Reference 2011

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.483634Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 875992f6-9b86-4dba-a3c3-852304b4c120 · outbound

This paper cites Learning Robust Rewards with Adversarial Inverse Reinforcement Learning.

Reward Models in Deep Reinforcement Learning: A Survey Learning Robust Rewards with Adversarial Inverse Reinforcement Learning

Reference 2016

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.473011Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d5b5aa4f-bf1a-44fe-8b45-54f78a626e19 · outbound

This paper cites Safe RLHF: Safe Reinforcement Learning from Human Feedback.

Reward Models in Deep Reinforcement Learning: A Survey Safe RLHF: Safe Reinforcement Learning from Human Feedback

Reference 2017

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.456084Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 53580f90-e529-44f7-a411-577256bbb502 · outbound

This paper cites Constitutional AI: Harmlessness from AI Feedback.

Reward Models in Deep Reinforcement Learning: A Survey Constitutional AI: Harmlessness from AI Feedback

Reference 2020

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.439486Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 64e3677c-cd60-4840-8c23-e9dacf260f04 · outbound

This paper cites Never Give Up: Learning Directed Exploration Strategies.

Reward Models in Deep Reinforcement Learning: A Survey Never Give Up: Learning Directed Exploration Strategies

Reference 2021

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.433306Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1b420d97-7414-4ab5-a1da-b8a05c7f42bc · outbound

This paper cites A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models.

Reward Models in Deep Reinforcement Learning: A Survey A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models

Reference 2022

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.467651Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1be62d4a-6672-4734-b03a-985e95e42ca8 · outbound

This paper cites Motif: Intrinsic Motivation from Artificial Intelligence Feedback.

Reward Models in Deep Reinforcement Learning: A Survey Motif: Intrinsic Motivation from Artificial Intelligence Feedback

Reference 2023

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.645727Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2fe16626-45d2-4e09-a520-a493d26b41fe · outbound

This paper cites LiPO: Listwise Preference Optimization through Learning-to-Rank.

Reward Models in Deep Reinforcement Learning: A Survey LiPO: Listwise Preference Optimization through Learning-to-Rank

Reference 2024

Resolution: unresolved; no resolver link, observed 2026-08-15T19:38:50.674235Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5f0cf442-4de2-42dd-a4a1-b515064d66f3 · outbound

This paper cites Dynamics-Aware Comparison of Learned Reward Functions.

Reward Models in Deep Reinforcement Learning: A Survey Dynamics-Aware Comparison of Learned Reward Functions

Reference 2025

Resolution: verified exact; local_arxiv, observed 2026-08-15T19:38:51.286313Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Pith citing papers

Observation 9c30b6bc-215c-4a92-8b91-d9a93b451a39 · inbound

Multi-objective Reinforcement Learning With Augmented States Requires Rewards After Deployment cites this paper.

Multi-objective Reinforcement Learning With Augmented States Requires Rewards After Deployment Reward Models in Deep Reinforcement Learning: A Survey

Reference 24

Resolution: verified exact; arxiv_id, observed 2026-05-10T09:38:41.945064Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation e53582ca-beba-4ed5-90ca-36d1efeb6910 · inbound

Occupancy Reward Shaping: Improving Credit Assignment for Offline Goal-Conditioned Reinforcement Learning cites this paper.

Occupancy Reward Shaping: Improving Credit Assignment for Offline Goal-Conditioned Reinforcement Learning Reward Models in Deep Reinforcement Learning: A Survey

Reference 23

Resolution: verified exact; arxiv_id, observed 2026-05-10T00:24:47.223690Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 79cea2e5-be26-43e8-9720-51304d27cddc · inbound

D-VLA: A High-Concurrency Distributed Asynchronous Reinforcement Learning Framework for Vision-Language-Action Models cites this paper.

D-VLA: A High-Concurrency Distributed Asynchronous Reinforcement Learning Framework for Vision-Language-Action Models Reward Models in Deep Reinforcement Learning: A Survey

Reference 27

Resolution: verified exact; arxiv_id, observed 2026-05-14T19:47:53.425028Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation ee58a84b-58e0-4b21-80ad-9659fcf303f4 · inbound

D-VLA: A High-Concurrency Distributed Asynchronous Reinforcement Learning Framework for Vision-Language-Action Models cites this paper.

D-VLA: A High-Concurrency Distributed Asynchronous Reinforcement Learning Framework for Vision-Language-Action Models Reward Models in Deep Reinforcement Learning: A Survey

Reference 27

Resolution: verified exact; arxiv_id, observed 2026-05-15T06:05:06.483419Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 6c8a3fc2-9855-4232-b2f2-9bf323ba31e0 · inbound

AudioProcessBench: Benchmark for Identifying Process Errors in Audio-Grounded Reasoning cites this paper.

AudioProcessBench: Benchmark for Identifying Process Errors in Audio-Grounded Reasoning Reward Models in Deep Reinforcement Learning: A Survey

Reference 3

Resolution: metadata mismatch; arxiv_id, observed 2026-07-02T23:37:27.332423Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation ad523e39-1735-4f45-8802-1b24dff731b1 · inbound

PortraitGen: Exemplar-Driven GRPO with Dual-Reward Guidance for Photorealistic Portrait Generation cites this paper.

PortraitGen: Exemplar-Driven GRPO with Dual-Reward Guidance for Photorealistic Portrait Generation Reward Models in Deep Reinforcement Learning: A Survey

Reference 51

Resolution: metadata mismatch; arxiv_id, observed 2026-07-04T13:29:50.942774Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.