Semi-online DPO, which syncs the generation model every few update steps, performs nearly as well as fully online DPO and GRPO, while strongly beating offline DPO.
Preference learning algorithms do not learn preference rankings
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Bridging Offline and Online Reinforcement Learning for LLMs
Semi-online DPO, which syncs the generation model every few update steps, performs nearly as well as fully online DPO and GRPO, while strongly beating offline DPO.