Transferring a weak model’s RL-induced log-ratio policy shift on a strong student’s own rollouts raises AIME accuracy more cheaply than imitating the weak teacher or running matched-step RL on the student.
arXiv preprint arXiv:2505.16142 , year =
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
years
2026 3representative citing papers
Dynamic Rollout Editing reduces overthinking in RL-trained LLMs by editing post-answer continuations in successful rollouts and preferring the edited versions within GRPO groups.
MOPD improves on-policy distillation by using peer successes and failures from multiple rollouts to construct more informative teacher signals, yielding consistent gains over baselines on reasoning benchmarks.
citing papers explorer
-
Weak-to-Strong Generalization via Direct On-Policy Distillation
Transferring a weak model’s RL-induced log-ratio policy shift on a strong student’s own rollouts raises AIME accuracy more cheaply than imitating the weak teacher or running matched-step RL on the student.
-
Dynamic Rollout Editing for Reducing Overthinking in RL-Trained Reasoning Models
Dynamic Rollout Editing reduces overthinking in RL-trained LLMs by editing post-answer continuations in successful rollouts and preferring the edited versions within GRPO groups.
-
Multi-Rollout On-Policy Distillation via Peer Successes and Failures
MOPD improves on-policy distillation by using peer successes and failures from multiple rollouts to construct more informative teacher signals, yielding consistent gains over baselines on reasoning benchmarks.