REVIEW 1 cited by
Lecture Notes on Linear Neural Networks: A Tale of Optimization and Generalization in Deep Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
These notes are based on a lecture delivered by NC on March 2021, as part of an advanced course in Princeton University on the mathematical understanding of deep learning. They present a theory (developed by NC, NR and collaborators) of linear neural networks -- a fundamental model in the study of optimization and generalization in deep learning. Practical applications born from the presented theory are also discussed. The theory is based on mathematical tools that are dynamical in nature. It showcases the potential of such tools to push the envelope of our understanding of optimization and generalization in deep learning. The text assumes familiarity with the basics of statistical learning theory. Exercises (without solutions) are included.
Forward citations
Cited by 1 Pith paper
-
Gradient Flow Equations for Deep Linear Neural Networks: A Survey from a Network Perspective
The adjacency-matrix reformulation of deep-linear gradient flow reveals a quotient-space structure of the loss landscape, but the proof that arcs between critical points determine stable and unstable manifolds is incomplete.
Discussion (0). Continue with ORCID to comment.