A sign-flipped SVRG variant called BP-SVRG adds stochastic-gradient noise instead of cancelling it, and empirically generalizes better than standard SVRG and often better than SGD on CIFAR and SVHN image classifiers.
Data-Dependent Path Normalization in Neural Networks
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We propose a unified framework for neural net normalization, regularization and optimization, which includes Path-SGD and Batch-Normalization and interpolates between them across two different dimensions. Through this framework we investigate issue of invariance of the optimization, data dependence and the connection with natural gradients.
fields
stat.ML 1years
2019 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Towards Better Generalization: BP-SVRG in Training Deep Neural Networks
A sign-flipped SVRG variant called BP-SVRG adds stochastic-gradient noise instead of cancelling it, and empirically generalizes better than standard SVRG and often better than SGD on CIFAR and SVHN image classifiers.