In a sequential d,n→∞ then m→∞ limit with dm/n fixed, the spectrum of a deep linear Gaussian network's feature covariance converges to the free log-normal law, whose T-transform solves a Burgers equation.
arXiv preprint arXiv:1805.00915 , year=
9 Pith papers cite this work. Polarity classification is still indexing.
9
Pith papers citing it
citation-role summary
background 1
citation-polarity summary
roles
background 1polarities
background 1representative citing papers
Finite-width shallow networks remain within poly(d) m^{-min(1,c/6)} of their mean-field limit uniformly in time when mean-field excess loss decays as t^{-c} under standard regularity and an integral condition on the loss.
Establishes O(N^{-1/2}) convergence for simultaneous MDA and O(N^{-2/3}) for alternating MDA to mixed Nash equilibria in mean-field convex-concave min-max problems via dual-space Bregman analysis.
Establishes convergence guarantees for overparameterized 2-layer ReLU networks in flow matching, generalization bounds for the velocity-field objective, and Wasserstein guarantees for generated samples, using multi-task representation learning bounds.
A unified framework for exponential tilting in diffusion and flow models that includes bias-variance decompositions showing finite gradient variance for some methods, norm bounds on adjoint ODEs, and adapted losses with new Crooks and Jarzynski identities.
SPIN lets weak LLMs become strong by self-generating training data from previous model versions and training to prefer human-annotated responses over its own outputs, outperforming DPO even with extra GPT-4 data on benchmarks.
Proves mean-field limit and propagation of chaos for 1D particle systems with singular absorption interactions using Girsanov transforms, tightness, and analysis of the nonlinear Fokker-Planck equation.
Presents an active-sampling method that approximates the weight subspace from Hessian finite differences, recovers the rank-1 tensors by robust nonlinear programming, and attributes layers with gradient descent, yielding stable recovery under a-posteriori verifiable conditions.
At critical learning rate 1/N, the rescaled correlation of SGD in single-index models is claimed to converge to a mean-reverting Ornstein-Uhlenbeck process, but the diffusion coefficients are internally inconsistent.
citing papers explorer
-
Geometric Dyson Brownian Motions and the Free Log-Normal Limit for a Non-Square Gaussian Matrix Product
In a sequential d,n→∞ then m→∞ limit with dm/n fixed, the spectrum of a deep linear Gaussian network's feature covariance converges to the free log-normal law, whose T-transform solves a Burgers equation.
-
Uniform-in-Time Weak Propagation-of-Chaos in Shallow Neural Networks
Finite-width shallow networks remain within poly(d) m^{-min(1,c/6)} of their mean-field limit uniformly in time when mean-field excess loss decays as t^{-c} under standard regularity and an integral condition on the loss.
-
Mirror Descent-Ascent for mean-field min-max problems
Establishes O(N^{-1/2}) convergence for simultaneous MDA and O(N^{-2/3}) for alternating MDA to mixed Nash equilibria in mean-field convex-concave min-max problems via dual-space Bregman analysis.
-
A Theory on Flow Matching with Neural Networks
Establishes convergence guarantees for overparameterized 2-layer ReLU networks in flow matching, generalization bounds for the velocity-field objective, and Wasserstein guarantees for generated samples, using multi-task representation learning bounds.
-
A unified perspective on fine-tuning and sampling with diffusion and flow models
A unified framework for exponential tilting in diffusion and flow models that includes bias-variance decompositions showing finite gradient variance for some methods, norm bounds on adjoint ODEs, and adapted losses with new Crooks and Jarzynski identities.
-
Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models
SPIN lets weak LLMs become strong by self-generating training data from previous model versions and training to prefer human-annotated responses over its own outputs, outperforming DPO even with extra GPT-4 data on benchmarks.
-
Mean-field limit of particle systems with absorption
Proves mean-field limit and propagation of chaos for 1D particle systems with singular absorption interactions using Girsanov transforms, tightness, and analysis of the nonlinear Fokker-Planck equation.
-
Robust and Resource Efficient Identification of Two Hidden Layer Neural Networks
Presents an active-sampling method that approximates the weight subspace from Hessian finite differences, recovers the rank-1 tensors by robust nonlinear programming, and attributes layers with gradient descent, yielding stable recovery under a-posteriori verifiable conditions.
-
Limit Theorems for Stochastic Gradient Descent in High-Dimensional Single-Layer Networks
At critical learning rate 1/N, the rescaled correlation of SGD in single-index models is claimed to converge to a mean-reverting Ornstein-Uhlenbeck process, but the diffusion coefficients are internally inconsistent.