Paper Citation Record · LEDGER

Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

As of 18 August 2026, this Paper Citation Record lists 0 of 0 outbound references and 34 inbound Pith citation observations for arXiv:2005.12729.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2005.12729 v1

Coverage vector

measured 0 of 0 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links

measured 34 of 34 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-18T06:34:40.430872+00:00

measured 34 of 34 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-16T11:13:53.530577Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-07-04T12:09:48.784830Z

Reference resolution

0 of 0 outbound references displayed

verified exact0
verified fuzzy0
unresolved0
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

No outbound reference observations are available for this paper version.

Pith citing papers

Observation d1e461ff-bee6-4969-857b-70a087d36390 · inbound

RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment cites this paper.

RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 94

Resolution: verified exact; arxiv_id, observed 2026-05-18T00:46:56.870644Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation 6320458f-5f51-4b32-b4d5-70cbfc97563a · inbound

Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents cites this paper.

Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 208

Resolution: verified exact; arxiv_id, observed 2026-05-20T09:42:04.375198Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation 976af50a-b1ce-43e3-ae7c-3b3b41d7c76b · inbound

DSTC: Direct Preference Learning with Only Self-Generated Tests and Code to Improve Code LMs cites this paper.

DSTC: Direct Preference Learning with Only Self-Generated Tests and Code to Improve Code LMs Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-12T17:05:15.103259Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation de21805e-f23c-4299-8bff-fc4190a4dbd9 · inbound

Self-reconfiguration Strategies for Space-distributed Spacecraft cites this paper.

Self-reconfiguration Strategies for Space-distributed Spacecraft Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 19

Resolution: unresolved; no resolver link, observed 2026-08-12T12:33:32.145779Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c7ca3801-c8ff-4f8e-a3be-145a2f85703a · inbound

Adam on Local Time: Addressing Nonstationarity in RL with Relative Adam Timesteps cites this paper.

Adam on Local Time: Addressing Nonstationarity in RL with Relative Adam Timesteps Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-11T05:51:10.143723Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 32c84664-64b6-4e43-9ace-e2025aed1d76 · inbound

PPO-Q: Proximal Policy Optimization with Parametrized Quantum Policies or Values cites this paper.

PPO-Q: Proximal Policy Optimization with Parametrized Quantum Policies or Values Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 65

Resolution: unresolved; no resolver link, observed 2026-08-10T20:53:52.596341Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3522272c-5d93-4a4e-b9e6-77d9c2c39948 · inbound

Evolution and The Knightian Blindspot of Machine Learning cites this paper.

Evolution and The Knightian Blindspot of Machine Learning Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 48

Resolution: unresolved; no resolver link, observed 2026-08-10T16:30:09.580796Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8161ca46-facc-4825-bec4-0d5aa31e7eed · inbound

Learning Explainable Dense Reward Shapes via Bayesian Optimization cites this paper.

Learning Explainable Dense Reward Shapes via Bayesian Optimization Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 19

Resolution: unresolved; no resolver link, observed 2026-08-16T11:13:53.530577Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 76df687c-b747-47bb-a911-6ea3f7a8c79d · inbound

Dynamic Action Interpolation: A Universal Approach for Accelerating Reinforcement Learning with Expert Guidance cites this paper.

Dynamic Action Interpolation: A Universal Approach for Accelerating Reinforcement Learning with Expert Guidance Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 32

Resolution: unresolved; no resolver link, observed 2026-08-16T10:14:54.414297Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a5693dd3-b148-488b-8ab7-57553632e80f · inbound

A Survey on Progress in LLM Alignment from the Perspective of Reward Design cites this paper.

A Survey on Progress in LLM Alignment from the Perspective of Reward Design Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 73

Resolution: unresolved; no resolver link, observed 2026-08-16T00:52:06.789472Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation dc0ec6a0-0fe7-4e1f-8560-d83ff9f57534 · inbound

Shallow Preference Signals: Large Language Model Aligns Even Better with Truncated Data? cites this paper.

Shallow Preference Signals: Large Language Model Aligns Even Better with Truncated Data? Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-07T15:15:46.288418Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 19324a54-6076-42a7-9974-d420050cd73f · inbound

Thompson Sampling in Online RLHF with General Function Approximation cites this paper.

Thompson Sampling in Online RLHF with General Function Approximation Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 12

Resolution: unresolved; no resolver link, observed 2026-08-07T12:43:47.833667Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ca9784f2-02ca-4061-ad86-be0546e474fc · inbound

On the Effect of Regularization in Policy Mirror Descent cites this paper.

On the Effect of Regularization in Policy Mirror Descent Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-06T18:18:45.097844Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b798d612-57e9-4f25-9a09-a704bf31f366 · inbound

Shared Control of Holonomic Wheelchairs through Reinforcement Learning cites this paper.

Shared Control of Holonomic Wheelchairs through Reinforcement Learning Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 30

Resolution: unresolved; no resolver link, observed 2026-08-06T15:01:56.164569Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 96838120-0c29-41d2-b247-d6365188a7d3 · inbound

Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges cites this paper.

Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 216

Resolution: unresolved; no resolver link, observed 2026-08-06T14:13:07.089075Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3cc5cde8-e828-4b8c-a869-f4ff5772a7b0 · inbound

Greener Deep Reinforcement Learning: Analysis of Energy and Carbon Efficiency Across Atari Benchmarks cites this paper.

Greener Deep Reinforcement Learning: Analysis of Energy and Carbon Efficiency Across Atari Benchmarks Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-15T16:30:00.976208Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 44e20981-aeb7-4aa2-b449-20b1aaf520ec · inbound

SERA: Soft-Verified Efficient Repository Agents cites this paper.

SERA: Soft-Verified Efficient Repository Agents Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 6

Resolution: unresolved; no resolver link, observed 2026-08-03T07:17:13.398189Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 43518c25-4e3d-4773-8eda-17dedb3c887c · inbound

Preventing Learning Stagnation in PPO by Scaling to 1 Million Parallel Environments cites this paper.

Preventing Learning Stagnation in PPO by Scaling to 1 Million Parallel Environments Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 13

Resolution: unresolved; no resolver link, observed 2026-08-02T18:45:20.537597Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c9d43959-881d-43c0-9145-44e658aec8e0 · inbound

Bounded Ratio Reinforcement Learning cites this paper.

Bounded Ratio Reinforcement Learning Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 5

Resolution: verified exact; arxiv_id, observed 2026-05-10T11:35:19.053807Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation 27c12819-cd74-4918-8b56-c044b86bbd6f · inbound

Application of Deep Reinforcement Learning to Event-Triggered Control for Networked Artificial Pancreas Systems cites this paper.

Application of Deep Reinforcement Learning to Event-Triggered Control for Networked Artificial Pancreas Systems Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 44

Resolution: metadata mismatch; arxiv_id, observed 2026-05-12T00:41:16.486953Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation 07e7e605-660a-40c8-beec-f7d10bbdfcb8 · inbound

Application of Deep Reinforcement Learning to Event-Triggered Control for Networked Artificial Pancreas Systems cites this paper.

Application of Deep Reinforcement Learning to Event-Triggered Control for Networked Artificial Pancreas Systems Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 44

Resolution: metadata mismatch; arxiv_id, observed 2026-05-19T17:47:41.626886Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation 4a49f367-01c8-4aa0-abed-8c07c071b344 · inbound

ANO: A Principled Approach to Robust Policy Optimization cites this paper.

ANO: A Principled Approach to Robust Policy Optimization Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 6

Resolution: verified exact; arxiv_id, observed 2026-05-09T05:45:23.303091Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation 52dd2441-8816-4fb8-af6a-aacbb3fc3bea · inbound

Does Synthetic Data Help? Empirical Evidence from Deep Learning Time Series Forecasters cites this paper.

Does Synthetic Data Help? Empirical Evidence from Deep Learning Time Series Forecasters Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 254

Resolution: verified exact; arxiv_id, observed 2026-05-11T18:41:12.454247Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation bb983171-0d8d-45a4-be46-b146e61802ae · inbound

Response Time Enhances Alignment with Heterogeneous Preferences cites this paper.

Response Time Enhances Alignment with Heterogeneous Preferences Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 66

Resolution: verified exact; arxiv_id, observed 2026-05-11T04:46:00.141867Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation 5cdcc69b-7f56-4ea8-828b-6657dc5f04f9 · inbound

TOPPO: Rethinking PPO for Multi-Task Reinforcement Learning with Critic Balancing cites this paper.

TOPPO: Rethinking PPO for Multi-Task Reinforcement Learning with Critic Balancing Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 4

Resolution: metadata mismatch; arxiv_id, observed 2026-05-13T01:52:05.738887Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation 34485d47-4743-4400-a09d-9c895fef8c0d · inbound

Ratio-Variance Regularized Policy Optimization cites this paper.

Ratio-Variance Regularized Policy Optimization Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 3

Resolution: metadata mismatch; arxiv_id, observed 2026-06-29T19:53:55.985787Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation f6af7c43-22eb-45ff-a1b1-9f8337fb0d18 · inbound

Personalized Observation Normalization for Federated Reinforcement Learning in Simulation Environments with Heterogeneity cites this paper.

Personalized Observation Normalization for Federated Reinforcement Learning in Simulation Environments with Heterogeneity Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 25

Resolution: unresolved; no resolver link, observed 2026-07-12T23:02:52.079919Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation cf90a442-801c-4282-9a38-7337e7bc4607 · inbound

Dynamic Multi-Pair Trading Strategy in Cryptocurrency Markets with Deep Reinforcement Learning cites this paper.

Dynamic Multi-Pair Trading Strategy in Cryptocurrency Markets with Deep Reinforcement Learning Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 13

Resolution: metadata mismatch; arxiv_id, observed 2026-07-02T06:06:41.426638Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation 316ce026-771d-4e4e-b3c9-692453b5f899 · inbound

Distribution-Agnostic Robust Trajectory Optimization via Chance-Constrained Reinforcement Learning cites this paper.

Distribution-Agnostic Robust Trajectory Optimization via Chance-Constrained Reinforcement Learning Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 34

Resolution: verified exact; arxiv_id, observed 2026-07-03T16:28:38.903131Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation c43b365a-919c-42ee-9c54-d17a1c1fdd92 · inbound

LOLLA: Deep Reinforcement Learning for Closed-Loop Link Adaptation Towards a GPU-Accelerated AI-RAN cites this paper.

LOLLA: Deep Reinforcement Learning for Closed-Loop Link Adaptation Towards a GPU-Accelerated AI-RAN Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 45

Resolution: metadata mismatch; arxiv_id, observed 2026-07-04T12:09:48.786465Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-18T06:34:40.430872+00:00.

Observation fe19167d-f4a3-4b44-9437-b41ed7f7b555 · inbound

Understanding electricity consumption behaviour through Inverse Reinforcement Learning cites this paper.

Understanding electricity consumption behaviour through Inverse Reinforcement Learning Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 59

Resolution: unresolved; no resolver link, observed 2026-07-12T04:21:03.464973Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 47e40277-b8b8-483d-a3a0-2a33dbbe64d8 · inbound

Multi-Turn On-Policy Distillation with Prefix Replay cites this paper.

Multi-Turn On-Policy Distillation with Prefix Replay Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 147

Resolution: unresolved; no resolver link, observed 2026-07-11T13:53:36.775836Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0253a4f8-8e1d-4aab-85cf-946b64d8577d · inbound

Multi-Turn On-Policy Distillation with Prefix Replay cites this paper.

Multi-Turn On-Policy Distillation with Prefix Replay Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 148

Resolution: unresolved; no resolver link, observed 2026-08-02T08:40:48.552061Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 40ce625e-4d6b-4884-b8bc-3a84d544f609 · inbound

ODYSSE: Episode-wise Policy Optimization for Personalized Agentic Reasoning cites this paper.

ODYSSE: Episode-wise Policy Optimization for Personalized Agentic Reasoning Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Reference 10

Resolution: unresolved; no resolver link, observed 2026-08-01T02:44:30.092180Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.