Paper Citation Record · LEDGER

Online Learning from Strategic Human Feedback in LLM Fine-Tuning

As of 22 August 2026, this Paper Citation Record lists 29 of 29 outbound references and 3 inbound Pith citation observations for arXiv:2412.16834.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2412.16834 v2

Coverage vector

measured 29 of 29 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-11T10:21:33.386906Z

measured 32 of 32 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-22T06:32:14.747728+00:00

measured 3 of 3 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-09T16:38:06.631594Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-05-23T03:57:29.547083Z

Reference resolution

29 of 29 outbound references displayed

verified exact0
verified fuzzy15
unresolved12
parse uncertain0
malformed identifier2
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 5581eb0b-618e-43bc-b1fc-f2c35210ed43 · outbound

This paper cites Open and ef- ﬁcient foundation language models,.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Open and ef- ﬁcient foundation language models,

Reference 1

Resolution: malformed identifier; no resolver link, observed 2026-08-11T10:21:33.226742Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 080bf1ba-c99c-4b07-8e54-c0fd5fbf53e7 · outbound

This paper cites Openassistant conversations-democratizing large langu age model align- ment,.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Openassistant conversations-democratizing large langu age model align- ment,

Reference 2

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:21:34.048597Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 5d2cb5f1-e569-4123-ab5e-b8eb35a4e966 · outbound

This paper cites Mechanism design for llm ﬁne-tuning with multiple reward models,.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Mechanism design for llm ﬁne-tuning with multiple reward models,

Reference 3

Resolution: unresolved; no resolver link, observed 2026-08-11T10:21:33.238520Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 659ffb1b-6c6c-4db9-88ae-034280e88d3e · outbound

This paper cites Deep reinforcement learning from human preferences,.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Deep reinforcement learning from human preferences,

Reference 4

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:21:34.026522Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 13afe7f0-dfec-4ad9-804a-edc1f1c76c0e · outbound

This paper cites Training language models to follow instructions with human feedback,.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Training language models to follow instructions with human feedback,

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-11T10:21:33.249324Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2201e4b3-4a35-4481-b6e3-61ef869d996b · outbound

This paper cites Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback

Reference 6

Resolution: malformed identifier; no resolver link, observed 2026-08-11T10:21:33.254896Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d64fcd5f-ca12-4676-be30-df721c4eb9aa · outbound

This paper cites Llama 2: Open Foundation and Fine-Tuned Chat Models.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Llama 2: Open Foundation and Fine-Tuned Chat Models

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-11T10:21:33.262991Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e9fa04d5-8af8-470f-86fe-f46f49dce922 · outbound

This paper cites Iterative preference learning from human fee dback: Bridging theory and practice for rlhf under kl-constraint,.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Iterative preference learning from human fee dback: Bridging theory and practice for rlhf under kl-constraint,

Reference 8

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:21:33.992234Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 84f3f08e-0452-40bb-8c30-f96e7e88307a · outbound

This paper cites Truthful Aggregation of LLMs with an Application to Online Advertising.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Truthful Aggregation of LLMs with an Application to Online Advertising

Reference 9

Resolution: unresolved; no resolver link, observed 2026-08-11T10:21:33.274599Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1aa31b4f-9400-49ad-8cf5-4aa5b510fff6 · outbound

This paper cites Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback

Reference 10

Resolution: unresolved; no resolver link, observed 2026-08-11T10:21:33.280224Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation fc9a329d-d042-488a-a3e7-8775e9c19de6 · outbound

This paper cites Online prediction w ith selﬁsh experts,.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Online prediction w ith selﬁsh experts,

Reference 11

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:21:33.974378Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 1ceec05b-4d48-4569-ba79-91279ec29a70 · outbound

This paper cites Direct preference optimization: Y our language mo del is secretly a reward model,.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Direct preference optimization: Y our language mo del is secretly a reward model,

Reference 12

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:21:33.951188Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation bf8c9c6e-1d9c-4cd5-856b-2f461b19f5b9 · outbound

This paper cites Exploratory Preference Optimization: Harnessing Implicit Q*-Approximation for Sample-Efficient RLHF.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Exploratory Preference Optimization: Harnessing Implicit Q*-Approximation for Sample-Efficient RLHF

Reference 13

Resolution: unresolved; no resolver link, observed 2026-08-11T10:21:33.298781Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 22f9dbe2-1194-4ba6-a6ce-6bcf58055ccf · outbound

This paper cites Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF

Reference 14

Resolution: unresolved; no resolver link, observed 2026-08-11T10:21:33.305260Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 770af17c-b057-4d2b-8d01-158888e17854 · outbound

This paper cites Self-Exploring Language Models: Active Preference Elicitation for Online Alignment.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Self-Exploring Language Models: Active Preference Elicitation for Online Alignment

Reference 15

Resolution: unresolved; no resolver link, observed 2026-08-11T10:21:33.311936Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 64b23332-c366-4c73-87c2-e633713626e4 · outbound

This paper cites OPTune: Efficient Online Preference Tuning.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning OPTune: Efficient Online Preference Tuning

Reference 16

Resolution: unresolved; no resolver link, observed 2026-08-11T10:21:33.317740Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 57010b3c-4180-449f-97b4-8e19fc0d0e30 · outbound

This paper cites Rl hf from heterogeneous feedback via personalization and preferenc e aggregation,.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Rl hf from heterogeneous feedback via personalization and preferenc e aggregation,

Reference 17

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:21:33.931200Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 0065d0fc-4288-4c1f-bc2d-dcd572a60ec3 · outbound

This paper cites Auctions with LLM Summaries.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Auctions with LLM Summaries

Reference 18

Resolution: unresolved; no resolver link, observed 2026-08-11T10:21:33.330873Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7a1be47e-1fa3-4e51-b83a-ef1b41116068 · outbound

This paper cites Co llaborative algorithms for online personalized mean estimation,.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Co llaborative algorithms for online personalized mean estimation,

Reference 19

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:21:33.913419Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 6417fda4-3096-4ec6-acf2-616faec6f01b · outbound

This paper cites Mechanism design for col- laborative normal mean estimation,.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Mechanism design for col- laborative normal mean estimation,

Reference 20

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:21:33.894481Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation eb98fa7f-171c-44ed-b445-fa521143fb8f · outbound

This paper cites Strategyproof mechanisms for group- fair obnoxious facility location problems,.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Strategyproof mechanisms for group- fair obnoxious facility location problems,

Reference 21

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:21:33.872552Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation f59b7e01-752d-4ea8-88fa-29e8daedd497 · outbound

This paper cites Positive intra-group exter nalities in facility location,.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Positive intra-group exter nalities in facility location,

Reference 22

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:21:33.854554Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 102e7f13-4d9e-4e7b-b86e-24dd6689a6e8 · outbound

This paper cites Dueling RL: Reinforcement Learning with Trajectory Preferences.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Dueling RL: Reinforcement Learning with Trajectory Preferences

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-11T10:21:33.356599Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2834b20f-dec2-43fe-9d11-f6a871fb107c · outbound

This paper cites Human-i n- the-loop: Provably efﬁcient preference-based reinforcem ent learning with general function approximation,.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Human-i n- the-loop: Provably efﬁcient preference-based reinforcem ent learning with general function approximation,

Reference 24

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:21:33.829225Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation cac1b905-4782-4861-851b-c2214bd75009 · outbound

This paper cites DPO Meets PPO: Reinforced Token Optimization for RLHF.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning DPO Meets PPO: Reinforced Token Optimization for RLHF

Reference 25

Resolution: unresolved; no resolver link, observed 2026-08-11T10:21:33.365940Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8a179afb-9f1c-45ea-8f62-8831a933f466 · outbound

This paper cites E fﬁcient com- petitions and online learning with strategic forecasters,.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning E fﬁcient com- petitions and online learning with strategic forecasters,

Reference 26

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:21:33.810635Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation e64388f0-e0a5-4448-a6ad-1238f6fb6213 · outbound

This paper cites No-regret and incentive-compatible online learning,.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning No-regret and incentive-compatible online learning,

Reference 27

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:21:33.786016Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation f2b7db93-697f-4409-934e-07d819ea4fd0 · outbound

This paper cites trlx: A framework for large s cale rein- forcement learning from human feedback,.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning trlx: A framework for large s cale rein- forcement learning from human feedback,

Reference 28

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:21:33.765555Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation d03a433a-d317-4c59-b926-19421549c927 · outbound

This paper cites Tl; dr: M ining reddit to learn automatic summarization,.

Online Learning from Strategic Human Feedback in LLM Fine-Tuning Tl; dr: M ining reddit to learn automatic summarization,

Reference 29

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T10:21:33.747141Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Pith citing papers

Observation e4c26c1d-ba06-405a-9602-c300644a5c25 · inbound

The Battling Influencers Game: Nash Equilibria Structure of a Potential Game and Implications to Value Alignment cites this paper.

The Battling Influencers Game: Nash Equilibria Structure of a Potential Game and Implications to Value Alignment Online Learning from Strategic Human Feedback in LLM Fine-Tuning

Reference 8

Resolution: unresolved; no resolver link, observed 2026-08-09T16:38:06.631594Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e86a58fa-c6b6-4bea-968b-51222439eda9 · inbound

How Humans Help LLMs: Assessing and Incentivizing Human Preference Annotators cites this paper.

How Humans Help LLMs: Assessing and Incentivizing Human Preference Annotators Online Learning from Strategic Human Feedback in LLM Fine-Tuning

Reference 43

Resolution: verified exact; arxiv_id, observed 2026-05-23T03:57:29.550406Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation a8d5d594-0c3c-4105-980d-49db6e563d74 · inbound

Incentivizing High-Quality Human Annotations with Golden Questions cites this paper.

Incentivizing High-Quality Human Annotations with Golden Questions Online Learning from Strategic Human Feedback in LLM Fine-Tuning

Reference 19

Resolution: verified exact; arxiv_id, observed 2026-05-19T13:42:19.382745Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.