SGPO uses one shared model to refine its own responses and then optimize with DPO on those self-generated preference pairs, outperforming DPO and SPIN on AlpacaEval 2.0 and Arena-Hard without external preference labels.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
SGPO: Self-Generated Preference Optimization based on Self-Improver
SGPO uses one shared model to refine its own responses and then optimize with DPO on those self-generated preference pairs, outperforming DPO and SPIN on AlpacaEval 2.0 and Arena-Hard without external preference labels.