REVIEW 3 cited by
Stochastic weight matrix dynamics during learning and Dyson Brownian motion
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We demonstrate that the update of weight matrices in learning algorithms can be described in the framework of Dyson Brownian motion, thereby inheriting many features of random matrix theory. We relate the level of stochasticity to the ratio of the learning rate and the mini-batch size, providing more robust evidence to a previously conjectured scaling relationship. We discuss universal and non-universal features in the resulting Coulomb gas distribution and identify the Wigner surmise and Wigner semicircle explicitly in a teacher-student model and in the (near-)solvable case of the Gaussian restricted Boltzmann machine.
Forward citations
Cited by 3 Pith papers
-
Stochastic Quantization as Optimal Control
Stochastic quantization is re-expressed as finite-time optimal control, in which a learned Doob force plus exact path weights reach the Gibbs measure without waiting for equilibrium.
-
Random Matrix Theory for Stochastic Gradient Descent
SGD weight-matrix eigenvalue fluctuations follow random matrix predictions, with variance proportional to learning rate divided by batch size, the linear scaling rule.
-
Physics-Driven Learning for Inverse Problems in Quantum Chromodynamics
A perspective article reviewing physics-driven machine learning for inverse problems in QCD, without introducing new data, derivations, or quantitative results.
Discussion (0). Continue with ORCID to comment.