Matching upper and lower bounds on DPO policy optimality gap are derived that depend on a single design-dependent information matrix linking pair selection to estimation error and suboptimality.
Optimal design for reward modeling in rlhf
4 Pith papers cite this work. Polarity classification is still indexing.
4
Pith papers citing it
years
2026 4representative citing papers
In a Gaussian single-index model, neural reward models recover the hidden direction for β1 above an O(1) threshold and provide tilted-policy value-gap bounds for label-weighted and surrogate-weighted exponential fits.
GCSL reframes LLM fine-tuning as supervised pursuit of quality thresholds using natural-language goals, outperforming SFT and DPO on toxicity, code, and recommendation tasks.
A statistical survey of RLHF for LLM alignment that connects preference learning and policy optimization to models like Bradley-Terry-Luce while reviewing methods, extensions, and open challenges.
citing papers explorer
-
Which Pairs to Compare for LLM Post-Training?
Matching upper and lower bounds on DPO policy optimality gap are derived that depend on a single design-dependent information matrix linking pair selection to estimation error and suboptimality.
-
How Neural Reward Models Learn Features for Policy Optimization: A Single-Index Analysis
In a Gaussian single-index model, neural reward models recover the hidden direction for β1 above an O(1) threshold and provide tilted-policy value-gap bounds for label-weighted and surrogate-weighted exponential fits.
-
Goal-Conditioned Supervised Learning for LLM Fine-Tuning
GCSL reframes LLM fine-tuning as supervised pursuit of quality thresholds using natural-language goals, outperforming SFT and DPO on toxicity, code, and recommendation tasks.
-
Reinforcement Learning from Human Feedback: A Statistical Perspective
A statistical survey of RLHF for LLM alignment that connects preference learning and policy optimization to models like Bradley-Terry-Luce while reviewing methods, extensions, and open challenges.