Paper Citation Record · LEDGER

When Can Proxies Improve the Sample Complexity of Preference Learning?

As of 24 August 2026, this Paper Citation Record lists 43 of 43 outbound references and 1 inbound Pith citation observation for arXiv:2412.16475.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2412.16475 v1

Coverage vector

measured 43 of 43 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-11T10:41:24.133597Z

measured 44 of 44 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-23T06:30:58.430688+00:00

measured 1 of 1 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-07T15:15:46.439486Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: pith, observed 2026-08-07T15:15:46.504789Z

Reference resolution

43 of 43 outbound references displayed

verified exact1
verified fuzzy18
unresolved24
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 0697c033-2049-4b76-a444-cb22ee7d507a · outbound

This paper cites write newline.

When Can Proxies Improve the Sample Complexity of Preference Learning? write newline

Reference 1

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:23.850580Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0407d3c7-024b-4ad7-905a-9e86d08624af · outbound

This paper cites GPT-4 Technical Report.

When Can Proxies Improve the Sample Complexity of Preference Learning? GPT-4 Technical Report

Reference 2

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:23.858731Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 06aa6455-3a21-4490-8e1a-8d7a295660fc · outbound

This paper cites Accuracy of chatgpt, google bard, and microsoft bing for simplifying radiology reports.

When Can Proxies Improve the Sample Complexity of Preference Learning? Accuracy of chatgpt, google bard, and microsoft bing for simplifying radiology reports

Reference 3

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:41:25.132545Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-23T06:30:58.430688+00:00.

Observation 2f75013d-35b0-4a43-9e98-8f815536e98b · outbound

This paper cites The evolved radio and its implications for modelling the evolution of novel sensors.

When Can Proxies Improve the Sample Complexity of Preference Learning? The evolved radio and its implications for modelling the evolution of novel sensors

Reference 4

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:41:25.116027Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-23T06:30:58.430688+00:00.

Observation 34dbd208-86fa-4304-ad0c-2627754f1232 · outbound

This paper cites an unresolved cited work.

When Can Proxies Improve the Sample Complexity of Preference Learning? Unresolved cited work

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:23.884929Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 11dbaa5e-9aec-4357-a9b0-a12929b8320d · outbound

This paper cites Open problems and fundamental limitations of reinforcement learning from human feedback.

When Can Proxies Improve the Sample Complexity of Preference Learning? Open problems and fundamental limitations of reinforcement learning from human feedback

Reference 6

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:41:25.088000Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-23T06:30:58.430688+00:00.

Observation 931cb768-d394-44b8-b278-cd9433a34a84 · outbound

This paper cites MaxMin-RLHF: Alignment with Diverse Human Preferences.

When Can Proxies Improve the Sample Complexity of Preference Learning? MaxMin-RLHF: Alignment with Diverse Human Preferences

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:23.896444Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 908181f5-b5b9-488b-8ebf-c5d191ee899c · outbound

This paper cites ODIN : Disentangled reward mitigates hacking in RLHF.

When Can Proxies Improve the Sample Complexity of Preference Learning? ODIN : Disentangled reward mitigates hacking in RLHF

Reference 8

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:41:25.070443Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-23T06:30:58.430688+00:00.

Observation 2b7d8406-ef65-4f0e-844e-f51d81e2b508 · outbound

This paper cites Meta discovery: Learning to discover novel classes given very limited data.

When Can Proxies Improve the Sample Complexity of Preference Learning? Meta discovery: Learning to discover novel classes given very limited data

Reference 9

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:41:25.050708Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-23T06:30:58.430688+00:00.

Observation ff703073-f67b-47a9-99cb-6265db905465 · outbound

This paper cites Faulty reward functions in the wild, 2016.

When Can Proxies Improve the Sample Complexity of Preference Learning? Faulty reward functions in the wild, 2016

Reference 10

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:41:25.030561Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-23T06:30:58.430688+00:00.

Observation 6c9fbf7e-f935-4922-90cf-f1ef8190148f · outbound

This paper cites Reward model ensembles help mitigate overoptimization.

When Can Proxies Improve the Sample Complexity of Preference Learning? Reward model ensembles help mitigate overoptimization

Reference 11

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:41:25.010653Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-23T06:30:58.430688+00:00.

Observation 7c5f317f-021c-437f-83f9-5b2a52b7125f · outbound

This paper cites The Expertise Problem: Learning from Specialized Feedback.

When Can Proxies Improve the Sample Complexity of Preference Learning? The Expertise Problem: Learning from Specialized Feedback

Reference 12

Resolution: verified exact; local_arxiv, observed 2026-08-11T10:41:24.502247Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-23T06:30:58.430688+00:00.

Observation d231620d-24d0-4340-9d71-89640e31f792 · outbound

This paper cites Group symmetry in pac learning.

When Can Proxies Improve the Sample Complexity of Preference Learning? Group symmetry in pac learning

Reference 13

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:41:24.986038Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-23T06:30:58.430688+00:00.

Observation 38a0f1d1-5cba-438d-8bfe-99a2c9b01b88 · outbound

This paper cites Active teacher selection for reward learning.

When Can Proxies Improve the Sample Complexity of Preference Learning? Active teacher selection for reward learning

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:23.943994Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0b7e3efa-6565-4eed-8135-f71a844c3077 · outbound

This paper cites Scaling laws for reward model overoptimization.

When Can Proxies Improve the Sample Complexity of Preference Learning? Scaling laws for reward model overoptimization

Reference 15

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:41:24.967509Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-23T06:30:58.430688+00:00.

Observation b96e6e82-1da8-4353-88e4-cf4087f4ba94 · outbound

This paper cites Glass floor colleges reject top applicants, accepting only the students likely to enroll, 2001.

When Can Proxies Improve the Sample Complexity of Preference Learning? Glass floor colleges reject top applicants, accepting only the students likely to enroll, 2001

Reference 16

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:41:24.945572Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-23T06:30:58.430688+00:00.

Observation d459e898-3910-4869-9dc8-9090ce61848d · outbound

This paper cites Correcting the Mythos of KL-Regularization: Direct Alignment without Overoptimization via Chi-Squared Preference Optimization.

When Can Proxies Improve the Sample Complexity of Preference Learning? Correcting the Mythos of KL-Regularization: Direct Alignment without Overoptimization via Chi-Squared Preference Optimization

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:23.970292Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation cc13a5e3-34c2-4e43-8e03-944febf9a67d · outbound

This paper cites AI Alignment: A Comprehensive Survey.

When Can Proxies Improve the Sample Complexity of Preference Learning? AI Alignment: A Comprehensive Survey

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:23.977822Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7d735d05-947e-4bdd-870b-97b7e625be5c · outbound

This paper cites Reward (mis) design for autonomous driving.

When Can Proxies Improve the Sample Complexity of Preference Learning? Reward (mis) design for autonomous driving

Reference 19

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:41:24.782930Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-23T06:30:58.430688+00:00.

Observation 5ccb960a-0681-47fb-aa64-390ab974365a · outbound

This paper cites InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modeling.

When Can Proxies Improve the Sample Complexity of Preference Learning? InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modeling

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:23.993491Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 110d7045-3500-4309-a6a5-1db8d3a2757e · outbound

This paper cites Mohri, A.

When Can Proxies Improve the Sample Complexity of Preference Learning? Mohri, A

Reference 21

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:41:24.763858Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-23T06:30:58.430688+00:00.

Observation 6ab36fc5-0af2-4c6c-a383-e4de8b3f8637 · outbound

This paper cites Overcoming exploration in reinforcement learning with demonstrations.

When Can Proxies Improve the Sample Complexity of Preference Learning? Overcoming exploration in reinforcement learning with demonstrations

Reference 22

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:41:24.746697Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-23T06:30:58.430688+00:00.

Observation 577facfb-b71c-4d97-90df-45987cc9d9eb · outbound

This paper cites The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models.

When Can Proxies Improve the Sample Complexity of Preference Learning? The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:24.008976Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c67ff398-8c69-499d-a6a7-05cd5c492973 · outbound

This paper cites A deep reinforced model for abstractive summarization.

When Can Proxies Improve the Sample Complexity of Preference Learning? A deep reinforced model for abstractive summarization

Reference 24

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:41:24.726845Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-23T06:30:58.430688+00:00.

Observation 82bbe396-158a-46d0-b37e-a69f0ec74169 · outbound

This paper cites Learning Transferable Visual Models From Natural Language Supervision.

When Can Proxies Improve the Sample Complexity of Preference Learning? Learning Transferable Visual Models From Natural Language Supervision

Reference 25

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:24.028935Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 91d54c79-cc00-4caf-b3c8-9d69c3fcdb08 · outbound

This paper cites Direct preference optimization: Your language model is secretly a reward model.

When Can Proxies Improve the Sample Complexity of Preference Learning? Direct preference optimization: Your language model is secretly a reward model

Reference 26

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:24.034379Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7c46f08b-c16d-4d5b-b3dc-097a730d8a2d · outbound

This paper cites Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithms.

When Can Proxies Improve the Sample Complexity of Preference Learning? Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithms

Reference 27

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:24.040308Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6e7637ad-d65a-4802-ad75-b54a9395c9da · outbound

This paper cites Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations.

When Can Proxies Improve the Sample Complexity of Preference Learning? Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations

Reference 28

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:24.046339Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 85fea4fa-87cb-4d5f-b2e5-0d93ad894e22 · outbound

This paper cites Countering Reward Over-optimization in LLM with Demonstration-Guided Reinforcement Learning.

When Can Proxies Improve the Sample Complexity of Preference Learning? Countering Reward Over-optimization in LLM with Demonstration-Guided Reinforcement Learning

Reference 29

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:24.051680Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b117e539-d0b6-46b2-91d8-be09761a4c97 · outbound

This paper cites Proximal Policy Optimization Algorithms.

When Can Proxies Improve the Sample Complexity of Preference Learning? Proximal Policy Optimization Algorithms

Reference 30

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:24.057383Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 471b38f0-0d6c-4198-9d2d-de0322efe506 · outbound

This paper cites Loose lips sink ships: Mitigating length bias in reinforcement learning from human feedback.

When Can Proxies Improve the Sample Complexity of Preference Learning? Loose lips sink ships: Mitigating length bias in reinforcement learning from human feedback

Reference 31

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:41:24.696125Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-23T06:30:58.430688+00:00.

Observation 699e0a8f-d3cd-489a-b6e3-dfbdf36a16e7 · outbound

This paper cites A Long Way to Go: Investigating Length Correlations in RLHF.

When Can Proxies Improve the Sample Complexity of Preference Learning? A Long Way to Go: Investigating Length Correlations in RLHF

Reference 32

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:24.069209Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 088f894f-3ba5-4cf3-b954-7382fe9c58a0 · outbound

This paper cites Defining and characterizing reward gaming.

When Can Proxies Improve the Sample Complexity of Preference Learning? Defining and characterizing reward gaming

Reference 33

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:24.074435Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3177201a-82f7-43eb-9a78-53a765599aa0 · outbound

This paper cites Causal Confusion and Reward Misidentification in Preference-Based Reward Learning.

When Can Proxies Improve the Sample Complexity of Preference Learning? Causal Confusion and Reward Misidentification in Preference-Based Reward Learning

Reference 34

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:24.080607Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation aa9b29dd-165f-484e-b684-85273c1ce7e5 · outbound

This paper cites Relatively rational: Learning utilities and rationalities jointly from pairwise preferences.

When Can Proxies Improve the Sample Complexity of Preference Learning? Relatively rational: Learning utilities and rationalities jointly from pairwise preferences

Reference 35

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:41:24.664012Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-23T06:30:58.430688+00:00.

Observation 06152cda-8b23-4307-9751-c981a0d929f3 · outbound

This paper cites Bayesian Reward Models for LLM Alignment.

When Can Proxies Improve the Sample Complexity of Preference Learning? Bayesian Reward Models for LLM Alignment

Reference 36

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:24.092589Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3a676381-6f24-4375-a3b8-c54775f34c5d · outbound

This paper cites Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback.

When Can Proxies Improve the Sample Complexity of Preference Learning? Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback

Reference 37

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:41:24.641039Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-23T06:30:58.430688+00:00.

Observation da2098a3-63e0-4281-bc6d-d3f3f7282467 · outbound

This paper cites Larger and more instructable language models become less reliable.

When Can Proxies Improve the Sample Complexity of Preference Learning? Larger and more instructable language models become less reliable

Reference 38

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:24.103363Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b4131ff1-1cf1-4919-bf85-d5d5a7d31616 · outbound

This paper cites Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHF.

When Can Proxies Improve the Sample Complexity of Preference Learning? Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHF

Reference 39

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:24.109274Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d8c41436-e5c7-48b3-9cc7-d1cf47ff52b8 · outbound

This paper cites Consequences of misaligned ai.

When Can Proxies Improve the Sample Complexity of Preference Learning? Consequences of misaligned ai

Reference 40

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:41:24.618246Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-23T06:30:58.430688+00:00.

Observation d2848552-7265-461d-a959-0e9bcf4e8739 · outbound

This paper cites @esa (Ref.

When Can Proxies Improve the Sample Complexity of Preference Learning? @esa (Ref

Reference 41

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:24.120042Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 59d96322-98f3-4cfb-92ff-49cb8412f5a5 · outbound

This paper cites an unresolved cited work.

When Can Proxies Improve the Sample Complexity of Preference Learning? Unresolved cited work

Reference 42

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:24.126138Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d7555979-9685-4b71-b5cc-418ee489e08c · outbound

This paper cites an unresolved cited work.

When Can Proxies Improve the Sample Complexity of Preference Learning? Unresolved cited work

Reference 43

Resolution: unresolved; no resolver link, observed 2026-08-11T10:41:24.133597Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

Observation 80e08e02-5c8f-49fd-9790-1f82d1c7f942 · inbound

Shallow Preference Signals: Large Language Model Aligns Even Better with Truncated Data? cites this paper.

Shallow Preference Signals: Large Language Model Aligns Even Better with Truncated Data? When Can Proxies Improve the Sample Complexity of Preference Learning?

Reference 51

Resolution: verified exact; local_arxiv, observed 2026-08-07T15:15:46.510460Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-23T06:30:58.430688+00:00.