Understanding over-parameterized deep networks by geometrization
read the original abstract
A complete understanding of the widely used over-parameterized deep networks is a key step for AI. In this work we try to give a geometric picture of over-parameterized deep networks using our geometrization scheme. We show that the Riemannian geometry of network complexity plays a key role in understanding the basic properties of over-parameterizaed deep networks, including the generalization, convergence and parameter sensitivity. We also point out deep networks share lots of similarities with quantum computation systems. This can be regarded as a strong support of our proposal that geometrization is not only the bible for physics, it is also the key idea to understand deep learning systems.
This paper has not been read by Pith yet.
Forward citations
Cited by 1 Pith paper
-
Deep network as memory space: complexity, generalization, disentangled representation and interpretability
Deep networks are framed as memory spaces whose complexity is defined by a Fisher metric, with the least action principle linking this complexity to generalization and disentanglement for better interpretability.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.