MB-SARAH-RBB uses a random Barzilai-Borwein step size to accelerate mini-batch SARAH, with a linear convergence proof and improved complexity for strongly convex objectives.
Stochastic Recursive Gradient Algorithm for Nonconvex Optimization
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
In this paper, we study and analyze the mini-batch version of StochAstic Recursive grAdient algoritHm (SARAH), a method employing the stochastic recursive gradient, for solving empirical loss minimization for the case of nonconvex losses. We provide a sublinear convergence rate (to stationary points) for general nonconvex functions and a linear convergence rate for gradient dominated functions, both of which have some advantages compared to other modern stochastic gradient algorithms for nonconvex losses.
fields
cs.LG 1years
2019 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Accelerating Mini-batch SARAH by Step Size Rules
MB-SARAH-RBB uses a random Barzilai-Borwein step size to accelerate mini-batch SARAH, with a linear convergence proof and improved complexity for strongly convex objectives.