Reusing past trajectories with a power-mean-corrected importance-weighting estimator gives policy-gradient methods a sample complexity of O~(epsilon^{-1}) in the full-reuse regime.
Stochastic Recursive Momentum for Policy Gradient Methods
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
In this paper, we propose a novel algorithm named STOchastic Recursive Momentum for Policy Gradient (STORM-PG), which operates a SARAH-type stochastic recursive variance-reduced policy gradient in an exponential moving average fashion. STORM-PG enjoys a provably sharp $O(1/\epsilon^3)$ sample complexity bound for STORM-PG, matching the best-known convergence rate for policy gradient algorithm. In the mean time, STORM-PG avoids the alternations between large batches and small batches which persists in comparable variance-reduced policy gradient methods, allowing considerably simpler parameter tuning. Numerical experiments depicts the superiority of our algorithm over comparative policy gradient algorithms.
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Reusing Trajectories in Policy Gradients Enables Fast Convergence
Reusing past trajectories with a power-mean-corrected importance-weighting estimator gives policy-gradient methods a sample complexity of O~(epsilon^{-1}) in the full-reuse regime.