Paper Citation Record · LEDGER

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment

As of 7 August 2026, this Paper Citation Record lists 36 of 36 outbound references and 2 inbound Pith citation observations for arXiv:2506.12725.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2506.12725 v1

Coverage vector

measured 36 of 36 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-07T00:52:40.142142Z

measured 38 of 38 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-07T06:34:17.273281+00:00

measured 2 of 2 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-07-14T10:33:54.851493Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-07-03T10:48:02.077289Z

Reference resolution

36 of 36 outbound references displayed

verified exact1
verified fuzzy0
unresolved35
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation eea05dbb-9d5e-4c5d-96c5-eeda1ea10ebb · outbound

This paper cites Nemotron-4 340B Technical Report.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Nemotron-4 340B Technical Report

Reference 1

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:36.504800Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation be3c1356-7b0c-41fe-b551-d2a62c7d1b27 · outbound

This paper cites an unresolved cited work.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Unresolved cited work

Reference 2

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:36.583198Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e8416cc5-e960-429e-a412-41d57b747108 · outbound

This paper cites Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback

Reference 3

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:36.684782Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation fecc0cc1-0998-40c1-9b0a-4ca3c7c5e8cf · outbound

This paper cites Preference Learning Algorithms Do Not Learn Preference Rankings.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Preference Learning Algorithms Do Not Learn Preference Rankings

Reference 4

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:36.777313Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 584d3c2f-5e80-4a77-a6f5-9f8fc7961a0c · outbound

This paper cites Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:36.897991Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7e280702-b685-4bb0-9a7d-b0c727e6140d · outbound

This paper cites Training Verifiers to Solve Math Word Problems.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Training Verifiers to Solve Math Word Problems

Reference 6

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:37.016549Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e321a7a6-f36a-4d5d-b7c3-319fd6a1159b · outbound

This paper cites an unresolved cited work.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Unresolved cited work

Reference 7

Resolution: unresolved; raw_fallback, observed 2026-08-07T00:52:41.861680Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.

Observation 2626964b-c71b-4c81-8e7b-ebe65b5487c9 · outbound

This paper cites an unresolved cited work.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Unresolved cited work

Reference 8

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:37.207357Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6758a168-2c5d-403b-b989-2374ebf5ca1c · outbound

This paper cites The Llama 3 Herd of Models.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment The Llama 3 Herd of Models

Reference 9

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:37.397258Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 69614674-8c3e-4a0c-b2a7-2212109711c7 · outbound

This paper cites an unresolved cited work.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Unresolved cited work

Reference 10

Resolution: unresolved; raw_fallback, observed 2026-08-07T00:52:41.832367Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.

Observation c7658ce2-70e4-45a3-9b4b-da6d6aeb9b54 · outbound

This paper cites an unresolved cited work.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Unresolved cited work

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:37.786943Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b6e5fefe-5cb4-4e2a-927e-7b4fce63c90e · outbound

This paper cites an unresolved cited work.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Unresolved cited work

Reference 12

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:37.953089Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation dec2d96c-b540-40d5-b2fd-6ac2c13d463b · outbound

This paper cites Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedback.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedback

Reference 13

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:38.089108Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ae148c67-3aee-4709-bed9-cfe1c368eacc · outbound

This paper cites Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:38.224309Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 636ebf8b-c0cc-465f-8c3f-320acc1071a4 · outbound

This paper cites Decoupled Weight Decay Regularization.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Decoupled Weight Decay Regularization

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:38.366472Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 52dc2c5a-4574-4702-97f6-6ae8a112963a · outbound

This paper cites an unresolved cited work.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Unresolved cited work

Reference 16

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:38.506533Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1bca4ddf-b767-4fef-97ee-50634f531fc3 · outbound

This paper cites Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:38.611434Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1c8bc8ab-dfff-4b38-b740-7d91927a1c8d · outbound

This paper cites Iterative Reasoning Preference Optimization.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Iterative Reasoning Preference Optimization

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:38.682978Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0a63c277-c376-4034-8a15-bf4045abdc64 · outbound

This paper cites From $r$ to $Q^*$: Your Language Model is Secretly a Q-Function.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment From $r$ to $Q^*$: Your Language Model is Secretly a Q-Function

Reference 19

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:38.754574Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 516444c0-147a-4c9c-94d0-d65198c0f411 · outbound

This paper cites an unresolved cited work.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Unresolved cited work

Reference 20

Resolution: unresolved; raw_fallback, observed 2026-08-07T00:52:41.665312Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.

Observation f10f0bf1-7302-4892-b3a1-08e07374f506 · outbound

This paper cites an unresolved cited work.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Unresolved cited work

Reference 21

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:38.904176Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ebf9e83c-244e-42e8-87d9-acd9d69a2cdb · outbound

This paper cites Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:38.975908Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7dc98299-d56f-4f05-a29f-e51e7514e890 · outbound

This paper cites an unresolved cited work.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Unresolved cited work

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:39.043292Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 89a706e6-a3cf-4515-9476-3c59ddedfcdf · outbound

This paper cites an unresolved cited work.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Unresolved cited work

Reference 24

Resolution: unresolved; raw_fallback, observed 2026-08-07T00:52:41.472975Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.

Observation ba407102-3452-4fc6-a313-f1dc0ebbda04 · outbound

This paper cites Understanding the performance gap between online and offline alignment algorithms.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Understanding the performance gap between online and offline alignment algorithms

Reference 25

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:39.184131Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 23e75e75-04d2-441f-87b8-ad46b8f76dcb · outbound

This paper cites an unresolved cited work.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Unresolved cited work

Reference 26

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:39.270943Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7cc21a04-c493-4bc8-b9c2-a7f276b5e96f · outbound

This paper cites Self-Play Preference Optimization for Language Model Alignment.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Self-Play Preference Optimization for Language Model Alignment

Reference 27

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:39.353170Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e31260c0-e133-4221-9f08-b385be1d2d8f · outbound

This paper cites Minor DPO reject penalty to increase training robustness.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Minor DPO reject penalty to increase training robustness

Reference 28

Resolution: verified exact; local_arxiv, observed 2026-08-07T00:52:40.355270Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.

Observation dd7e1b4f-6f3d-4a44-b87c-15b8f9510814 · outbound

This paper cites an unresolved cited work.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Unresolved cited work

Reference 29

Resolution: unresolved; raw_fallback, observed 2026-08-07T00:52:41.126372Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.

Observation b041ab72-18b7-4dc3-9737-9372bbad9760 · outbound

This paper cites Qwen2.5 Technical Report.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Qwen2.5 Technical Report

Reference 30

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:39.621098Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1df16d6b-e59a-4f7a-a205-005bb957c8f0 · outbound

This paper cites an unresolved cited work.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Unresolved cited work

Reference 31

Resolution: unresolved; raw_fallback, observed 2026-08-07T00:52:40.912826Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.

Observation 03dd5b3f-78d5-44f9-a972-9f66493b4e59 · outbound

This paper cites an unresolved cited work.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Unresolved cited work

Reference 32

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:39.749756Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 42593b94-453a-41e9-a4c5-655ca96d3b21 · outbound

This paper cites an unresolved cited work.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Unresolved cited work

Reference 33

Resolution: unresolved; raw_fallback, observed 2026-08-07T00:52:40.726777Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.

Observation b94d2f5b-1367-4867-abf6-632dd9e7d655 · outbound

This paper cites Instruction-Following Evaluation for Large Language Models.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment Instruction-Following Evaluation for Large Language Models

Reference 34

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:39.972011Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3d3f3190-89d9-46ad-b0ad-c35eeb0ee1ef · outbound

This paper cites online" 'onlinestring :=.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment online" 'onlinestring :=

Reference 35

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:40.073887Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 63bc41f5-9f7f-4a0d-8fbd-12c65e1f724e · outbound

This paper cites write newline.

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment write newline

Reference 36

Resolution: unresolved; no resolver link, observed 2026-08-07T00:52:40.142142Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

Observation b6c688fd-1aff-47c1-9b42-4fce800708bd · inbound

Analyzing and Improving Fine-grained Preference Optimization in Medical LVLMs cites this paper.

Analyzing and Improving Fine-grained Preference Optimization in Medical LVLMs Rethinking DPO: The Role of Rejected Responses in Preference Misalignment

Reference 30

Resolution: verified exact; arxiv_id, observed 2026-07-03T10:48:02.079020Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-07T06:34:17.273281+00:00.

Observation 1b629639-8ed1-4636-8dfb-7d7e727e1127 · inbound

Agentic-DPO: From Imitation to Agentic Policy Optimization on Expert Trajectories cites this paper.

Agentic-DPO: From Imitation to Agentic Policy Optimization on Expert Trajectories Rethinking DPO: The Role of Rejected Responses in Preference Misalignment

Reference 8

Resolution: unresolved; no resolver link, observed 2026-07-14T10:33:54.851493Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.