SFT on curated data is a lower bound on a sparse-reward RL objective, and an importance-weighted variant, iw-SFT, tightens the bound and beats plain SFT on AIME 2024 and GPQA.
Efficient iterative policy optimization
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We tackle the issue of finding a good policy when the number of policy updates is limited. This is done by approximating the expected policy reward as a sequence of concave lower bounds which can be efficiently maximized, drastically reducing the number of policy updates required to achieve good performance. We also extend existing methods to negative rewards, enabling the use of control variates.
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Supervised Fine Tuning on Curated Data is Reinforcement Learning (and can be improved)
SFT on curated data is a lower bound on a sparse-reward RL objective, and an importance-weighted variant, iw-SFT, tightens the bound and beats plain SFT on AIME 2024 and GPQA.