NFPO augments the PPO surrogate with N-step forward traces to bridge local approximations and exact policy gradients, delivering tighter policy-improvement bounds and improved results on reasoning benchmarks.
E y„µ « Rpyq Tÿ t“1 ` ρt ´1 ˘ ´ Γt`1 ´Γ pNq t`1 ¯ff “E y„µ « Rpyq Tÿ t“1 ` ρt ´1 ˘ ΓpNq t`1 pQmt`1 ´1q ff ďξ Tÿ t“1 Eµ
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Multi-Step Likelihood-Ratio Correction for Reinforcement Learning with Verifiable Rewards
NFPO augments the PPO surrogate with N-step forward traces to bridge local approximations and exact policy gradients, delivering tighter policy-improvement bounds and improved results on reasoning benchmarks.