Derives layer-wise recursions for finite-width tensors under orthogonal initialization that reproduce the observed large-depth stability of nonlinear networks.
Roberts, Sho Yaida, and Boris Hanin
6 Pith papers cite this work. Polarity classification is still indexing.
6
Pith papers citing it
representative citing papers
A replica path-integral calculation yields analytic large-N formulas for covariance statistics and participation dimension in nonlinear recurrent neural networks with quenched Gaussian disorder, confirmed by simulation.
A high-level outline is given for a unified theory that reduces learning to a small set of ideas from dynamical systems, geometry, and physics via definitions of solvable problems and parametrized methods.
The work tests perturbative viability of single-layer neural networks for local QFTs at finite neuron number N in phi^4 theory, finding UV-cutoff-sensitive O(1/N) corrections with weak convergence and proposing a modification for better scaling.
Neural-network ensembles match closed Gaussian systems but lack the open-system non-Hermitian generator and continuous spectrum required by nuclear optical models, yielding a structural negative on applicability.
Lecture notes develop semiclassical methods to compute large-n scaling dimensions of composite operators in CFTs, recovering known results in free theory and deriving one-loop corrections at the Wilson-Fisher fixed point.
citing papers explorer
-
Criticality and Saturation in Orthogonal Neural Networks
Derives layer-wise recursions for finite-width tensors under orthogonal initialization that reproduce the observed large-depth stability of nonlinear networks.
-
Statistics of correlations in nonlinear recurrent neural networks
A replica path-integral calculation yields analytic large-N formulas for covariance statistics and participation dimension in nonlinear recurrent neural networks with quenched Gaussian disorder, confirmed by simulation.
-
Man, Machine, and Mathematics
A high-level outline is given for a unified theory that reduces learning to a small set of ideas from dynamical systems, geometry, and physics via definitions of solvable problems and parametrized methods.
-
Viability of perturbative expansion for quantum field theories on neurons
The work tests perturbative viability of single-layer neural networks for local QFTs at finite neuron number N in phi^4 theory, finding UV-cutoff-sensitive O(1/N) corrections with weak convergence and proposing a modification for better scaling.
-
Integrating Out, Twice:The Open-System Case That Neural-Network Ensemble Theory Is Missing
Neural-network ensembles match closed Gaussian systems but lack the open-system non-Hermitian generator and continuous spectrum required by nuclear optical models, yielding a structural negative on applicability.
-
Lectures on Semiclassical Methods for Composite Operators
Lecture notes develop semiclassical methods to compute large-n scaling dimensions of composite operators in CFTs, recovering known results in free theory and deriving one-loop corrections at the Wilson-Fisher fixed point.