Pith. sign in

REVIEW 1 cited by

On the Ineffectiveness of Variance Reduced Optimization for Deep Learning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1812.04529 v2 pith:CFXXX62N submitted 2018-12-11 cs.LG stat.ML

Aaron Defazio , Léon Bottou This is my paper · ORCID
classification cs.LGstat.ML
keywords optimizationapplicationdeepvarianceapplicabilityapproachesduringencountered
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

The application of stochastic variance reduction to optimization has shown remarkable recent theoretical and practical success. The applicability of these techniques to the hard non-convex optimization problems encountered during training of modern deep neural networks is an open problem. We show that naive application of the SVRG technique and related approaches fail, and explore why.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Towards Better Generalization: BP-SVRG in Training Deep Neural Networks

    stat.ML 2019-08 conditional novelty 6.0 of 10

    A sign-flipped SVRG variant called BP-SVRG adds stochastic-gradient noise instead of cancelling it, and empirically generalizes better than standard SVRG and often better than SGD on CIFAR and SVHN image classifiers.

Pith tools