REVIEW 5 cited by
Gradient Descent Finds Global Minima of Deep Neural Networks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Gradient Descent Finds Global Minima of Deep Neural Networks
read the original abstract
Gradient descent finds a global minimum in training deep neural networks despite the objective function being non-convex. The current paper proves gradient descent achieves zero training loss in polynomial time for a deep over-parameterized neural network with residual connections (ResNet). Our analysis relies on the particular structure of the Gram matrix induced by the neural network architecture. This structure allows us to show the Gram matrix is stable throughout the training process and this stability implies the global optimality of the gradient descent algorithm. We further extend our analysis to deep residual convolutional neural networks and obtain a similar convergence result.
Forward citations
Cited by 5 Pith papers
-
Limitations of Lazy Training of Two-layers Neural Networks
For quadratic targets in d dimensions, two-layer quadratic networks achieve lower risk when fully trained than in random features or neural tangent regimes if hidden units < d.
-
A Geometric Characterization of the Stationary Plateau for Two-Layer Neural Networks
A geometric classification of stationary points on neuron-splitting plateaus in two-layer NN loss landscapes using the inner Hessian.
-
Estimating Dense-Packed Zone Height in Liquid-Liquid Separation: A Physics-Informed Neural Network Approach
A PINN pretrained on mechanistic synthetic data and fine-tuned experimentally is deployed in an EKF-style filter to estimate separator phase heights from flow rates alone.
-
Convergence rates for gradient descent in the training of overparameterized artificial neural networks with piecewise affine activation
Batch gradient descent achieves linear convergence to zero MSE with high probability for sufficiently wide shallow NNs with non-affine piecewise affine activations and distinct inputs.
-
Two-block vs. Multi-block ADMM: An empirical evaluation of convergence
Empirical study finds multi-block ADMM outperforms two-block ADMM on optimization and prediction in multi-task learning across all tested datasets and dual step sizes.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.