REVIEW 2 cited by
On the Computational Efficiency of Training Neural Networks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
It is well-known that neural networks are computationally hard to train. On the other hand, in practice, modern day neural networks are trained efficiently using SGD and a variety of tricks that include different activation functions (e.g. ReLU), over-specification (i.e., train networks which are larger than needed), and regularization. In this paper we revisit the computational complexity of training neural networks from a modern perspective. We provide both positive and negative results, some of them yield new provably efficient and practical algorithms for training certain types of neural networks.
Forward citations
Cited by 2 Pith papers
-
Landscape analysis for shallow neural networks: Complete classification of critical points for cubic activation and affine target functions
For cubic-activation shallow networks with affine targets, the squared-loss landscape has no local maxima; every critical point is a global minimizer, a rigid non-global local minimum, or a saddle, and zero loss is ac...
-
The learnability scaling of quantum states: restricted Boltzmann machines
To reproduce the ground-state energy of a one-dimensional transverse-field Ising chain near its critical point, a restricted Boltzmann machine needs a number of weights that grows as the square of the number of qubits...
Discussion (0). Continue with ORCID to comment.