SVRG and SARAH converge with a weighted averaging scheme driven by estimate sequences, and when combined with Barzilai-Borwein step sizes and an adaptive inner-loop rule they become almost tune-free in numerical tests.
After a simple derivation, one can have the convergence rate λs = 2ηsL 2−ηsL + 2(1 +ηsL) ( 1− 2ηsL 1 +κ )m
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2019 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Almost Tune-Free Variance Reduction
SVRG and SARAH converge with a weighted averaging scheme driven by estimate sequences, and when combined with Barzilai-Borwein step sizes and an adaptive inner-loop rule they become almost tune-free in numerical tests.