Derives layer-wise recursions for finite-width tensors under orthogonal initialization that reproduce the observed large-depth stability of nonlinear networks.
Roberts, Sho Yaida, and Boris Hanin
5 Pith papers cite this work. Polarity classification is still indexing.
5
Pith papers citing it
representative citing papers
Derives exact correlation statistics for nonlinear RNNs in the large-N limit with Gaussian quenched disorder using path integrals, generalizing linear results and adding 1/N corrections.
A high-level outline is given for a unified theory that reduces learning to a small set of ideas from dynamical systems, geometry, and physics via definitions of solvable problems and parametrized methods.
The work tests perturbative viability of single-layer neural networks for local QFTs at finite neuron number N in phi^4 theory, finding UV-cutoff-sensitive O(1/N) corrections with weak convergence and proposing a modification for better scaling.
Neural-network ensembles match closed Gaussian systems but lack the open-system non-Hermitian generator and continuous spectrum required by nuclear optical models, yielding a structural negative on applicability.
citing papers explorer
-
Statistics of correlations in nonlinear recurrent neural networks
Derives exact correlation statistics for nonlinear RNNs in the large-N limit with Gaussian quenched disorder using path integrals, generalizing linear results and adding 1/N corrections.