Paper Citation Record · LEDGER

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory

As of 9 August 2026, this Paper Citation Record lists 31 of 31 outbound references and 3 inbound Pith citation observations for arXiv:2506.12350.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2506.12350 v1

Coverage vector

measured 31 of 31 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-07T01:04:33.188878Z

measured 34 of 34 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-09T06:31:02.800959+00:00

measured 3 of 3 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-07T22:50:33.214346Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-07-04T06:49:37.712152Z

Reference resolution

31 of 31 outbound references displayed

verified exact1
verified fuzzy7
unresolved22
parse uncertain0
malformed identifier0
metadata mismatch1

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 5b11d9d1-86ef-42c9-8247-4415029ad08c · outbound

This paper cites Therefore, a preference matching distribution exists.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Therefore, a preference matching distribution exists

Reference 1

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T01:04:34.815076Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation d6efb15f-2b56-4a38-b874-2a29f44bad6b · outbound

This paper cites Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback

Reference 4

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:30.630057Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a701d99e-19f1-4c20-b602-d115cabc1f85 · outbound

This paper cites Mapping Social Choice Theory to RLHF.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Mapping Social Choice Theory to RLHF

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:30.717742Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 37cfa10b-53f2-4d10-807e-fffd6a485e01 · outbound

This paper cites Policy Optimization in RLHF: The Impact of Out-of-preference Data.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Policy Optimization in RLHF: The Impact of Out-of-preference Data

Reference 8

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:31.048119Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e1530569-83dd-4e7b-9891-ac9968612789 · outbound

This paper cites Jackpot! Alignment as a Maximal Lottery.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Jackpot! Alignment as a Maximal Lottery

Reference 10

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:31.269934Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f056459f-c02d-4a49-91fb-13dadd438e7c · outbound

This paper cites AI Alignment and Social Choice: Fundamental Limitations and Policy Implications.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory AI Alignment and Social Choice: Fundamental Limitations and Policy Implications

Reference 11

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:31.412637Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation af2407b8-2d5e-41c8-a244-5ddb9e26359a · outbound

This paper cites Nash Learning from Human Feedback.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Nash Learning from Human Feedback

Reference 12

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:31.531656Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 88259934-527d-46dc-b40b-0a6c150812d2 · outbound

This paper cites RLHF from Heterogeneous Feedback via Personalization and Preference Aggregation.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory RLHF from Heterogeneous Feedback via Personalization and Preference Aggregation

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:31.735488Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d60b0dca-4b57-46e9-82f0-746fb3ec9b80 · outbound

This paper cites Proximal Policy Optimization Algorithms.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Proximal Policy Optimization Algorithms

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:31.830315Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 699e2fc7-0744-4fe0-9bf4-189127a6bc72 · outbound

This paper cites Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF

Reference 17

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:32.028901Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 92b86db4-ffbe-4594-be5f-5f1bfa313a7d · outbound

This paper cites Understanding the performance gap between online and offline alignment algorithms.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Understanding the performance gap between online and offline alignment algorithms

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:32.093625Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f2767bef-1860-470f-b3ee-a9578e512d76 · outbound

This paper cites Gemini: A Family of Highly Capable Multimodal Models.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Gemini: A Family of Highly Capable Multimodal Models

Reference 19

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:32.208926Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e894c17b-8779-4d88-b5c3-7ae66709f369 · outbound

This paper cites LLaMA: Open and Efficient Foundation Language Models.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory LLaMA: Open and Efficient Foundation Language Models

Reference 20

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:32.286520Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c8b38568-b696-4339-b24e-92ad0dda7007 · outbound

This paper cites Zhilin Wang, Yi Dong, Jiaqi Zeng, Virginia Adams, Makesh Narsimhan Sreedhar, Daniel Egert, Olivier Delalleau, Jane Scowcroft, Neel Kant, Aidan Swope, et al.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Zhilin Wang, Yi Dong, Jiaqi Zeng, Virginia Adams, Makesh Narsimhan Sreedhar, Daniel Egert, Olivier Delalleau, Jane Scowcroft, Neel Kant, Aidan Swope, et al

Reference 21

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T01:04:35.593726Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation 5f3c4032-9f7c-458e-964b-d3b0f333e6ba · outbound

This paper cites On the Algorithmic Bias of Aligning Large Language Models with RLHF: Preference Collapse and Matching Regularization.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory On the Algorithmic Bias of Aligning Large Language Models with RLHF: Preference Collapse and Matching Regularization

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:32.454213Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7a2cbe65-6812-4d08-8b31-c3678f0730a1 · outbound

This paper cites Restoring calibration for aligned large language models: A calibration-aware fine-tuning approach.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Restoring calibration for aligned large language models: A calibration-aware fine-tuning approach

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:32.522528Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7defe16f-f701-4713-bf0f-45a0706a7933 · outbound

This paper cites Iterative preference learning from human feedback: Bridging theory and practice for rlhf under kl- constraint.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Iterative preference learning from human feedback: Bridging theory and practice for rlhf under kl- constraint

Reference 24

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T01:04:35.373688Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation b33ced5a-b656-45c1-8310-f2c9ba8bd1ee · outbound

This paper cites Asymptotics of language model alignment.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Asymptotics of language model alignment

Reference 25

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T01:04:35.202878Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation 3fcb66fe-0fc2-42c9-a6cf-fd8b5072d6a7 · outbound

This paper cites Provable Multi-Party Reinforcement Learning with Diverse Human Feedback.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Provable Multi-Party Reinforcement Learning with Diverse Human Feedback

Reference 26

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:32.828481Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c87aacea-1141-4d61-b508-6f6cba18e980 · outbound

This paper cites an unresolved cited work.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Unresolved cited work

Reference 27

Resolution: unresolved; raw_fallback, observed 2026-08-07T01:04:35.004266Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation 6ff25476-08d1-443b-8c00-679588b028d9 · outbound

This paper cites The work by Tang et al.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory The work by Tang et al

Reference 29

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T01:04:34.587683Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation 861998dd-ec03-4cda-bb5b-b6c69066a382 · outbound

This paper cites Outside of fine-tuning, model editing has emerged as a complementary strategy to modify LLM behavior across tasks [Jin et al., 2025].

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Outside of fine-tuning, model editing has emerged as a complementary strategy to modify LLM behavior across tasks [Jin et al., 2025]

Reference 30

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T01:04:34.387321Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation 7aa67290-0996-43c2-8634-c003f6d7a217 · outbound

This paper cites [2024], which constrain its robustness relative to reinforcement learning techniques like PPO.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory [2024], which constrain its robustness relative to reinforcement learning techniques like PPO

Reference 31

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T01:04:34.151963Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation cad5854e-c103-4d4f-8d73-ebc7127bc9aa · outbound

This paper cites Learn Your Reference Model for Real Good Alignment.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Learn Your Reference Model for Real Good Alignment

Reference 2006

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:30.926721Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1b4f020e-eabd-446e-9266-f8a868b6dc13 · outbound

This paper cites Learning Linear Utility Functions From Pairwise Comparison Queries.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Learning Linear Utility Functions From Pairwise Comparison Queries

Reference 2009

Resolution: metadata mismatch; local_arxiv, observed 2026-08-07T01:04:33.883031Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation 80dbad57-95b5-4a4b-a3b3-b85a5bc8976e · outbound

This paper cites Sparks of Artificial General Intelligence: Early experiments with GPT-4.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Sparks of Artificial General Intelligence: Early experiments with GPT-4

Reference 2016

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:30.242850Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5f332d57-5d8f-4179-ba1e-cbc868b75180 · outbound

This paper cites Fundamental Limits of Game-Theoretic LLM Alignment: Smith Consistency and Preference Matching.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Fundamental Limits of Game-Theoretic LLM Alignment: Smith Consistency and Preference Matching

Reference 2017

Resolution: verified exact; local_arxiv, observed 2026-08-07T01:04:33.572321Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation 0093612e-68fb-4bcc-999e-531b37b21540 · outbound

This paper cites GPT-4 Technical Report.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory GPT-4 Technical Report

Reference 2020

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:31.655495Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 074dd878-1664-4c9d-88dc-fa80b3d653da · outbound

This paper cites MaxMin-RLHF: Alignment with Diverse Human Preferences.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory MaxMin-RLHF: Alignment with Diverse Human Preferences

Reference 2023

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:30.349050Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 19322493-ba98-49d0-bc1a-954f3af1adf5 · outbound

This paper cites Dataset Reset Policy Optimization for RLHF.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Dataset Reset Policy Optimization for RLHF

Reference 2024

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:30.492218Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8a24506e-5f96-4b25-8720-621acdde4d0f · outbound

This paper cites Statistical Impossibility and Possibility of Aligning LLMs with Human Preferences: From Condorcet Paradox to Nash Equilibrium.

Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory Statistical Impossibility and Possibility of Aligning LLMs with Human Preferences: From Condorcet Paradox to Nash Equilibrium

Reference 2025

Resolution: unresolved; no resolver link, observed 2026-08-07T01:04:31.156730Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

Observation 5deb5269-7ec9-4442-b42c-f9b9117dcbea · inbound

Game Theory Meets Large Language Models: A Systematic Survey with Taxonomy and New Frontiers cites this paper.

Game Theory Meets Large Language Models: A Systematic Survey with Taxonomy and New Frontiers Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory

Reference 151

Resolution: unresolved; no resolver link, observed 2026-08-07T22:50:33.214346Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 381d7141-7902-4833-9a51-9c632f59e13b · inbound

Transitivity in Inhomogeneous Random Tournaments cites this paper.

Transitivity in Inhomogeneous Random Tournaments Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory

Reference 51

Resolution: verified exact; arxiv_id, observed 2026-07-02T01:06:24.252873Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.

Observation c78663c3-2f78-44a3-b449-3e9eaf0d3262 · inbound

AI Alignment From Social Choice Perspectives cites this paper.

AI Alignment From Social Choice Perspectives Theoretical Tensions in RLHF: Reconciling Empirical Success with Inconsistencies in Social Choice Theory

Reference 81

Resolution: verified exact; arxiv_id, observed 2026-07-04T06:49:37.714149Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-09T06:31:02.800959+00:00.