REVIEW 5 cited by
A Correspondence Between Random Neural Networks and Statistical Field Theory
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
A number of recent papers have provided evidence that practical design questions about neural networks may be tackled theoretically by studying the behavior of random networks. However, until now the tools available for analyzing random neural networks have been relatively ad-hoc. In this work, we show that the distribution of pre-activations in random neural networks can be exactly mapped onto lattice models in statistical physics. We argue that several previous investigations of stochastic networks actually studied a particular factorial approximation to the full lattice model. For random linear networks and random rectified linear networks we show that the corresponding lattice models in the wide network limit may be systematically approximated by a Gaussian distribution with covariance between the layers of the network. In each case, the approximate distribution can be diagonalized by Fourier transformation. We show that this approximation accurately describes the results of numerical simulations of wide random neural networks. Finally, we demonstrate that in each case the large scale behavior of the random networks can be approximated by an effective field theory.
Forward citations
Cited by 5 Pith papers
-
Spontaneous symmetry breaking and Goldstone modes for deep information propagation
Equivariant neural networks support Goldstone-like modes enabling coherent information propagation across depth and recurrent iterations.
-
Critical Organization of Deep Neural Networks, and p-Adic Statistical Field Theories
A p-adic integral-equation formulation of deep networks is shown to have a unique hidden state under a contraction condition; the claimed thermodynamic limit and infinite-state bifurcation are not proven.
-
Criticality analysis of nuclear binding energy neural networks
On a two-input nuclear binding energy network, the paper validates ANNFT predictions for variance, kurtosis, and an optimal depth-to-width ratio r*=0.034 under SGD, while adaptive optimizers obscure criticality.
-
Time-multiplexed layer reuse for physical neural networks
ReLaX-Net cycles a small set of fixed weight matrices to deepen physical neural networks, but controlled experiments show a single repeated large layer is the best use of a fixed parameter budget.
-
Bulk-boundary decomposition of neural networks
The paper reframes SGD training of deep networks as a local Lagrangian with data confined to the boundaries, but the advertised energy continuity equation is absent from the body.
Discussion (0). Continue with ORCID to comment.