Applying importance weighting to reward model training to correct for policy distribution shift in RLHF improves final policy quality without new labels.
Fernando Hernandez-Garcia, Qingfeng Lan, Parash Rahman, A
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Off-Policy Corrected Reward Modeling for Reinforcement Learning from Human Feedback
Applying importance weighting to reward model training to correct for policy distribution shift in RLHF improves final policy quality without new labels.