Output-output correlations in finite Bayesian one-hidden-layer networks follow the kernel shape renormalization order parameter, with readout weight overlap equal to Q*_ab/λ1.
Spring-block theory of feature learning in deep neural networks
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Feature-learning deep nets progressively collapse data to a regular low-dimensional geometry. How this emerges from the collective action of nonlinearity, noise, learning rate, and other factors, has eluded first-principles theories built from microscopic neuronal dynamics. We exhibit a noise-nonlinearity phase diagram that identifies regimes where shallow or deep layers learn more effectively and propose a macroscopic mechanical theory that reproduces the diagram and links feature learning across layers to generalization.
citation-role summary
background 1
citation-polarity summary
fields
cond-mat.dis-nn 1years
2024 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
Kernel shape renormalization explains output-output correlations in finite Bayesian one-hidden-layer networks
Output-output correlations in finite Bayesian one-hidden-layer networks follow the kernel shape renormalization order parameter, with readout weight overlap equal to Q*_ab/λ1.