Pith. sign in

REVIEW 2 major objections 7 minor 2 cited by

The geometry of the deep linear network

T0 review · 2 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Deep linear network training is exactly Riemannian gradient descent combined with a Boltzmann entropy that selects among degenerate minimizers.

desk verdict A valuable expository synthesis of deep linear network geometry, but its central thermodynamic theorems are deferred to an unavailable companion paper, so the manuscript is not self-contained. read the letter →

arxiv 2411.09004 v1 pith:ZWRBBPI5 submitted 2024-11-13 cs.NE math.DSmath.PRnlin.AO

Govind Menon This is my paper · ORCID
classification cs.NEmath.DSmath.PRnlin.AO MSC 68T0753C2060H1058J65
keywords deeplinearnetworkgradientflowRiemanniansubmersionbalancedmanifoldBoltzmannentropyLangevinequationmatrixcompletionimplicitregularization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper gives a unified, expository account of training dynamics in the deep linear network (DLN): a deep network whose layers are matrices and whose output is their product. It claims that the gradient flow of any loss is constrained to invariant G-balanced varieties, and that on the balanced manifold the end-to-end matrix evolves by Riemannian gradient descent with an explicitly computable metric. The article then attaches a thermodynamics to this geometry: the volume of the group of weight factors giving the same end-to-end matrix defines a Boltzmann entropy with a closed Vandermonde formula, and stochastic gradient descent is modeled by a Riemannian Langevin equation whose free energy gradient is explicit. A sympathetic reader would care because this turns the vague notion of implicit regularization for degenerate losses, such as matrix completion, into an exact selection principle: entropy, not just the loss, governs which low-rank limit is reached.

What carries the argument

The machinery is a chain of geometric identifications. The G-balanced varieties are the invariant sets of the gradient flow; the SVD parametrization $W_p = Q_p \Lambda Q_{p-1}^T$, with $\Lambda=\Sigma^{1/N}$, exhibits each balanced manifold as orbits of the orthogonal group and makes tangent directions explicit. The metric $g_N$ is carried from the Frobenius inner product upstairs by Riemannian submersion, and its inverse operator $A_{N,W}$ is diagonalized by the singular-vector matrices $u_k v_l^T$ with eigenvalue $(\sigma_k^2-\sigma_l^2)/(\sigma_k^{2/N}-\sigma_l^{2/N})$ for $k\neq l$ and $N \sigma_k^{2-2/N}$ on the diagonal blocks. The entropy $S(W)=\log \mathrm{vol}(O_W)$ is the logarithm of the group-orbit volume, computed from the pullback metric through an orthonormal basis built from Chebyshev polynomials; its closed form is a ratio of Vandermonde determinants. The stochastic side is carried by Brownian motion on the quotient manifold and an explicit Itô SDE whose drift contains the mean-curvature correction that becomes Coulomb repulsion in singular-value coordinates.

What would settle it

Run the balanced gradient flow with $E(W)=\frac12((W_{11}-1)^2+(W_{22}-1)^2)$ for $d=2$ from random full-rank balanced initial conditions and compare the empirical low-rank limits with the minimizers of $F_\beta(W)=E(W)-\beta^{-1}S(W)$. If the selected limit is not among the free-energy minimizers as $\beta$ and $N$ vary, or if rank does not drop, the entropic selection principle fails.

Watch

Extended reading notes

Core claim

The central discovery is that overparametrization in a DLN does not merely add redundant coordinates; it organizes phase space into invariant algebraic varieties $M_G$ defined by the quadratic balance equations $W_{p+1}^T W_{p+1} = W_p W_p^T - G_p$, with the case $G=0$ giving balanced manifolds $M_r$ of rank $r$. On $M_r$ the map from weights to the end-to-end matrix $W$ is a Riemannian submersion, so the flow of $W$ is exactly the Riemannian gradient flow of $E$ with respect to the metric $g_N(Z,Z)=\mathrm{Tr}(Z^T A_{N,W}^{-1} Z)$, where $A_{N,W}$ is a positive operator diagonalized in singular-vector coordinates by explicit eigenvalues. The same submersion structure makes the volume of a group orbit $O_W = \varphi^{-1}(W) \cap M$ computable in closed form as $c_d^{N-1} \sqrt{\mathrm{van}(\Sigma^2)/\mathrm{van}(\Sigma^{2/N})}$, and this volume defines the Boltzmann entropy $S(W)=\log \mathrm{vol}(O_W)$. With Brownian motion on $(M_d,g_N)$, these ingredients produce the Riemannian Langevin equation for $W$, whose drift is $-\mathrm{grad}_{g_N}F_\beta$ with $F_\beta = E - \beta^{-1} S$; the entropy gradient is expressed through a diagonal matrix $\Sigma'$ whose off-diagonal terms act like Coulombic repulsion between singular values. This gives a complete thermodynamic description of training in the full-rank, distinct-singular-value regime.

Load-bearing premise

The exact entropy formula, the metric identification, and the Riemannian Langevin equation are proved on the full-rank balanced manifold with distinct singular values; the rank-deficient limit that training actually approaches is not covered by these proofs.

Editorial extensions

If this is right

  • On balanced manifolds, the end-to-end matrix follows a Riemannian gradient flow, so overparametrization does not change the energy-decay identity $dE/dt = -\|\mathrm{grad}_{g_N}E\|^2$; convergence to critical points happens at a rate set by the geometry, not by the number of parameters.
  • For degenerate losses, minimizers of the free energy $F_\beta = E - \beta^{-1} S$ supply an entropic selection principle; the paper conjectures that the minimizing set is a single point $A_\beta$ and that training with round-off noise selects its $\beta\to\infty$ limit.
  • Stochastic noise in the gauge directions is not a nuisance but the microscopic source of entropy: the Riemannian Langevin equation shows that tangential fluctuations produce a deterministic motion by mean curvature of the group orbits, exactly as in Dyson Brownian motion.
  • The explicit formulas extend to an infinite-depth limit $N\to\infty$, where the metric and entropy survive through the operator $A_{\infty,W}=\int_0^1 (WW^T)^{1-s} Z (W^T W)^s ds$ even though the upstairs group-orbit picture disappears.
  • The Bures-Wasserstein geometry of positive definite matrices arises as the $N=2$, $W_2=W_1^T$ specialization, connecting DLN training to optimal transport and to known gradient flows on matrix manifolds.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that if the thermodynamic description extends past full rank, implicit regularization is an entropic force: the volume of weight symmetries biases training toward macrostates with larger orbit volume, and this bias should be visible in the distribution of low-rank limits even before any explicit penalty is added.
  • One testable extension the paper does not run is to compare the empirical limits of balanced gradient flow with tiny gauge noise against the minimizers of $F_\beta$ for the $d=2$ matrix-completion energy $E(W)=\frac12((W_{11}-1)^2+(W_{22}-1)^2)$; the theory predicts that the selected rank-one matrix $\begin{pmatrix}1&a\\1/a&1\end{pmatrix}$ is the free-energy minimizer for each $\beta$ and $N$.
  • The anisotropic splitting of noise into gauge and observable directions predicts a concrete experimental signature: adding isotropic weight noise should change the selected low-rank limit in a computable way, whereas noiseless gradient flow should approach the entropic selection only through round-off errors.
  • The Simons-cone connection suggests that the rank-deficient balanced variety may carry singular minimal-surface structure; one could test whether tangential noise on low-rank varieties drives them toward the balanced manifold by motion by curvature, which would explain rank collapse as a geometric phenomenon rather than a pure optimization effect.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. This expository paper develops a geometric theory of training dynamics in deep linear networks (DLNs). The proofs in Section 7 establish Theorems 1–3: the phase space M_d^N is foliated by invariant G-balanced varieties; on balanced manifolds the end-to-end matrix evolves by the Riemannian gradient flow of the loss E with respect to the exactly computable metric g_N in (4.16), diagonalized explicitly in Lemma 1. The paper then constructs a thermodynamic picture: the fiber of the projection phi over W in the balanced manifold is an O_d^{N-1} orbit whose volume is claimed to be the Vandermonde-type formula (6.2) (Theorem 10); the metric g_N arises by Riemannian submersion (Theorem 13); an explicit orthonormal basis for the balanced manifold is given in (8.20)–(8.24) (Theorem 14); and a Riemannian Langevin equation with explicit free-energy gradient (12.12)–(12.13) is proposed as a stochastic extension, modeled on Dyson Brownian motion (Section 11). Theorems 10, 13, 14, and 18 are attributed to the author's unpublished companion manuscript [34] and are presented only as sketches.

Significance. If correct, this framework is a remarkably complete exact description of DLN training: an invariant foliation, a parameter-free Riemannian metric, entropy as the logarithm of a group-orbit volume, and an explicit noise model with a Coulomb-like repulsion term (12.8). The proofs that are actually present (Section 7, Lemmas 1–2) are direct, reproducible matrix computations, and the paper is candid about its modeling axiom (the RLE is phenomenological, Section 10.1) and about the rank-deficient gap (Section 14.4). I independently verified several central formulas: (7.13)–(7.14) follow from a geometric sum; (7.15) is the correct reciprocal metric; the interior (N−1)×(N−1) blocks of the pullback metric (8.10) have determinant 2^{N−1}(σ_k^2−σ_l^2)/(σ_k^{2/N}−σ_l^{2/N}), which yields (6.2); and (12.13) matches the g_N-gradient of −β^{−1}S computed from (6.2). The main weakness is verifiability: the headline quantitative claims are deferred to an unpublished, unlocatable companion [34], so the manuscript cannot currently serve as a standalone rigorous reference.

major comments (2)
  1. [§6.2, §8, §8.3, §12.2; ref. [34]] The paper's central new quantitative claims—Theorem 10 (entropy formula (6.2)), Theorem 14 (orthonormal basis (8.20)–(8.24)), and Theorem 18 (Brownian motion SDE (12.10)), together with the projected RLE (12.12)–(12.13)—are quoted from the author's companion manuscript [34], which has no venue or preprint identifier. The manuscript itself locates the omissions: Section 8 calls Theorem 14 'the technical core of [34]' and Section 12.2 says 'Then we show in [34] that…'. The determinant evaluation converting the block-tridiagonal metric (8.9)–(8.10) into (6.2), the Chebyshev-polynomial diagonalization behind (8.20)–(8.24), and the Itô/generator calculation behind (12.10) are not supplied. Because the entropy formula and the RLE are advertised in the abstract as the paper's contribution, this is a load-bearing verification gap rather than a presentation detail. I verified that the deferred claims are consistent with what is shown: the interior (N−1)×(N−1) blocks of (8.10) have determinant 2^{N−1}(σ_k^2−σ_l^2)/(σ_k^{2/N}−σ_l^{2/N}), which yields (6.2), and (12.13) agrees with the g_N-gradient of the entropy from (6.2). The concern is therefore completeness, not evident error. Please include the proofs (an appendix or supplement would suffice) or update [34] to a publicly verifiable form with results keyed to specific statements.
  2. [§5.3, §6.1, §13.3–§13.4, §14.1, §14.4; abstract] The entropy and RLE results are established only on the full-rank stratum with distinct singular values, where the SVD parametrization z is a local bijection (Section 5.3), and Section 14.4 explicitly concedes that singularities at repeated and repeated-zero singular values are not understood. Yet the motivating phenomena—rank drops during training (Section 13.3), convergence to low-rank energy minimizers (Section 14.1), and the suggested 'entropic origin for implicit regularization' (Sections 1 and 13.4)—live in the excluded rank-deficient regime, and the entropic selection principle (6.9) is explicitly conjectural (Section 6.3). I credit the paper for flagging the limitation in Sections 6.1 and 14.4; however, the abstract and introduction claim 'exact formulas for a Boltzmann entropy' and a thermodynamic framework without this scoping caveat. Please state in the abstract and introduction that the exact entropy and RLE formulas hold on the full-rank, distinct-singular-value stratum and that the rank-deficient and repeated-spectrum cases are open.
minor comments (7)
  1. [§6.2, Theorem 10] The theorem as stated ('For each W ∈ (M_d, g_N)') omits the hypothesis that the singular values of W be distinct; formula (6.2) and the parametrization (5.15)–(5.17) degenerate at repeated singular values (cf. Section 5.3). State the hypothesis explicitly.
  2. [§4.2, Eq. (4.8)] The range '1 ≤ p ≤ n' should be '1 ≤ p ≤ N − 1'; the integer n is not defined in this section.
  3. [§10.1, §12.1, abstract] The title 'RLE for stochastic gradient descent' and the abstract's phrase 'stochastic gradient descent of free energy' should be qualified: Section 10.1 states that the RLE is a phenomenological model that is 'not stochastic gradient descent (SGD) corresponding to batch processing'. Suggest rewording to avoid the implication of equivalence to practical SGD.
  4. [§14.1, Eqs. (14.3)–(14.4)] The symbol 'a ∈ R\{0}' is printed after the limiting matrix (14.4), but it parameterizes the family of rank-one matrices (14.3); move it to (14.3).
  5. [Ref. [34]] Reference [34] should be updated to include a preprint identifier or a 'to appear' venue; as printed it cannot be located, which compounds the verification gap described in Major Comment 1.
  6. [§3.3, §4.2, Eq. (4.20)] The symbol M_r is used both for the rank-r leaf of the balanced variety (Section 3.3) and for the rank-r matrix manifold (4.20); the relationship is explained, but the double use is a readability hazard.
  7. [§6.2, Eq. (6.3)] The phrase 'van(A) denoted' should read 'van(A) denotes'.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the balanced-variety and Riemannian-gradient results are derived from the group-action geometry; the entropy/RLE theorems are deferred to a same-author companion, which is a verification gap rather than a circular reduction.

full rationale

The derivations actually carried out in the manuscript are self-contained matrix computations: Theorem 1 (invariance of G-balanced varieties), Theorem 2 (reduced flow on balanced manifolds), Lemma 1 (spectral decomposition of A_{N,W}), and the explicit diagonal metric (7.15). Equation (4.15) is identified with a Riemannian gradient flow under the metric defined in (4.16); this is a definitional identification via the duality (4.18)-(4.19), not a fitted or circular step. The genuinely novel thermodynamic objects are not derived in this paper: Theorem 10 (entropy formula (6.2)), Theorem 14 (orthonormal basis (8.20)-(8.24)), and Theorem 18 (Brownian motion SDE (12.10)) are quoted from the same-author companion [34], with the paper stating 'This is the technical core of [34]' and 'Then we show in [34] that...'. That is an omitted-proof/verification gap: if [34] is unavailable or contains an error, the central formulas lack support in this manuscript. However, deferring a theorem to a companion paper is not equivalent to making the result identical to its inputs. No parameter is fitted to data, no prediction is a renamed fit, and no uniqueness theorem is imported to exclude alternatives. Section 14.4 honestly states that repeated and zero singular values are not understood; that is a scope limitation, not circularity. The self-citation is heavy and load-bearing, but the manuscript does not exhibit the equation-level reduction needed to call the derivation circular.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

No data were fitted; the only hand-set quantities are beta and kappa in the stochastic extension. The main formulas are derived from the Riemannian submersion geometry. The paper introduces no new physical entities. Its core premise is that group-orbit volume is a Boltzmann entropy and that RLE models round-off noise, an interpretation the authors flag as phenomenological.

free parameters (2)
  • inverse temperature beta
    Chosen by hand in F_beta = E - beta^{-1}S and in the RLE; controls the strength of microscopic fluctuations. Not fitted to data.
  • anisotropy parameter kappa
    Chosen by hand in the RLE to weight noise in the gauge group relative to observable directions. Not fitted to data.
assumptions (3)
  • standard math Existence and uniqueness of ODE and Ito SDE solutions, and Brownian motion on Riemannian manifolds.
    Used implicitly for gradient flows (4.7) and RLE (12.3, 12.11); no proof given.
  • domain assumption Theorems 10, 13, 14, and 18 from Menon and Yu are accepted as established results.
    The paper states these theorems with sketches and explicitly defers details: 'Further details may be found in [34]' (Section 8.3) and 'we show in [34]' (Section 12.2).
  • ad hoc to paper The Riemannian Langevin equation is a faithful model for noise in the null directions of the DLN.
    Section 10.1 says the RLE is a phenomenological model that 'loosely corresponds to how noise due to round-off errors may interact' and 'is not stochastic gradient descent'. This assumption supports the thermodynamic interpretation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The geometry of the deep linear network." pith.science (2026). https://pith.science/paper/ZWRBBPI5

@misc{pith2026241109004,
  author       = {Pith},
  title        = {Pith review of: The geometry of the deep linear network},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZWRBBPI5}},
  note         = {Machine review of arXiv:2411.09004}
}
read the original abstract

This article provides an expository account of training dynamics in the Deep Linear Network (DLN) from the perspective of the geometric theory of dynamical systems. Rigorous results by several authors are unified into a thermodynamic framework for deep learning. The analysis begins with a characterization of the invariant manifolds and Riemannian geometry in the DLN. This is followed by exact formulas for a Boltzmann entropy, as well as stochastic gradient descent of free energy using a Riemannian Langevin Equation. Several links between the DLN and other areas of mathematics are discussed, along with some open questions.

Figures

Figures reproduced from arXiv: 2411.09004 by the authors.

Figure 9.1
Figure 9.1. The foliation of MN d by the balanced varieties MG may be visualized as a foliation by conic sections (see equa￾tion (5.10)). ℳ N d W [PITH_FULL_IMAGE:figures/full_fig_p022_9_1.png] view at source ↗
Figure 9.2
Figure 9.2. In comparison with [PITH_FULL_IMAGE:figures/full_fig_p022_9_2.png] view at source ↗
Figure 9.3
Figure 9.3. Motion by (minus one half) curvature arising from tangential Brownian fluctuations as discussed in Section 11.2. ℳ N d d d ϕ W(t) W(t) [PITH_FULL_IMAGE:figures/full_fig_p023_9_3.png] view at source ↗
Figures from the paper (2 more)
Figure 9.4
Figure 9.4. Figure 9.4: Riemannian submersion ϕ : M → Md and the dy￾namics upstairs and downstairs. In this image, we illustrate the RLE with stochastic dynamics upstairs and deterministic gradient descent of free energy downstairs. See equations (12.6) and (12.7). and it takes the followin…
Figure 9.5
Figure 9.5. Figure 9.5: A rank-one variety within the zero energy set for matrix completion. See equations (14.3)–(14.4). The correspondence between the RLE for the DLN and Dyson Brownian motion is a powerful tool. It provides a way to use ideas from random matrix theory to analyze training…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse

    cs.LG 2026-07 accept novelty 7.0 of 10

    Noisy SGD in the mean-field regime forces wide multivariate ReLU networks to an effective width of at most 2P-1, yielding a continuous piecewise-affine predictor whose hyperplanes are non-redundant with respect to the...

  2. The Effect of Depth on the Expressivity of Deep Linear State-Space Models

    cs.LG 2025-06 conditional novelty 7.0 of 10

    Deeper linear state-space models can match the expressive power of shallower wider ones, and under norm constraints they do so with exponentially smaller parameter norms.

Reference graph

Works this paper leans on

40 extracted references · 34 canonical work pages · cited by 2 Pith papers

  1. [34]

    Menon and T

    G. Menon and T. Yu , An entropy formula for the deep linear network , 2024

    work page 2024

  2. [1]

    Arora, S

    R. Arora, S. Arora, J. Bruna, N. Cohen, S. Du, R. Ge, S. Gunasekar, C. Jin, J. Lee, T. Ma, et al. , Theory of deep learning , 2020

    work page 2020

  3. [2]

    Arora, N

    S. Arora, N. Cohen, N. Golowich, and W. Hu, A convergence analysis of gradient descent for deep linear neural networks, in 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, OpenReview.net, 2019

    work page 2019

  4. [3]

    Arora, N

    S. Arora, N. Cohen, and E. Hazan , On the optimization of deep networks: Implicit accel- eration by overparameterization , in International Conference on Machine Learning, PMLR, 2018, pp. 244–253

    work page 2018

  5. [4]

    Arora, N

    S. Arora, N. Cohen, W. Hu, and Y. Luo , Implicit regularization in deep matrix factoriza- tion, in Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, 2019, pp. 7411–7422

    work page 2019

  6. [5]

    B. Bah, H. Rauhut, U. Terstiege, and M. Westdickenberg , Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizers , Information and Inference: A Journal of the IMA, 11 (2022), pp. 307–353

    2022

  7. [6]

    J. Baik, P. Deift, and T. Suidan , Combinatorics and random matrix theory , vol. 172 of Graduate Studies in Mathematics, American Mathematical Society, Providence, RI, 2016

    work page 2016

  8. [7]

    Baldi and K

    P. Baldi and K. Hornik, Neural networks and principal component analysis: Learning from examples without local minima , Neural networks, 2 (1989), pp. 53–58

    work page 1989

Show all 40 references
  1. [8]

    P. F. Baldi and K. Hornik , Learning in linear neural networks: A survey , IEEE Transac- tions on neural networks, 6 (1995), pp. 837–858

    1995

  2. [9]

    E. A. Barbaˇ sin and N. N. Krasovski˘ i , On stability of motion in the large , Doklady Akad. Nauk SSSR (N.S.), 86 (1952), pp. 453–456

    1952

  3. [10]

    Bhatia, T

    R. Bhatia, T. Jain, and Y. Lim , On the Bures–Wasserstein distance between positive defi- nite matrices, Expositiones Mathematicae, 37 (2019), pp. 165–191

    2019

  4. [11]

    A. M. Bloch, R. W. Brockett, and T. S. Ratiu , Completely integrable gradient flows , Communications in Mathematical Physics, 147 (1992), pp. 57–74

    1992

  5. [12]

    Br ´echet, K

    P. Br ´echet, K. Papagiannouli, J. An, and G. Mont ´ufar, Critical points and conver- gence analysis of generative deep linear networks trained with Bures-Wasserstein loss , in International Conference on Machine Learning, PMLR, 2023, pp. 3106–3147

    2023

  6. [13]

    H. T. M. Chu, S. Ghosh, C. T. Lam, and S. S. Mukherjee , Implicit regularization via spectral neural networks and non-linear matrix sensing , arXiv:2402.17595, (2024)

    2024 arXiv

  7. [14]

    Cohen, G

    N. Cohen, G. Menon, and Z. Veraszto , Deep linear networks for matrix completion—an infinite depth limit , SIAM Journal on Applied Dynamical Systems, 22 (2023), pp. 3208–3232. THE GEOMETRY OF THE DEEP LINEAR NETWORK 37

    2023

  8. [15]

    DeVore, B

    R. DeVore, B. Hanin, and G. Petrova , Neural network approximation , Acta Numerica, 30 (2021), pp. 327–444

    2021

  9. [16]

    C. C. J. Domin ´e, N. Anguita, A. M. Proca, L. Braun, D. Kunin, P. A. M. Mediano, and A. M. Saxe , From lazy to rich: Exact learning dynamics in deep linear networks , arXiv:2409.14623, (2024)

    2024 arXiv

  10. [17]

    S. S. Du, W. Hu, and J. D. Lee , Algorithmic regularization in learning deep homogeneous models: Layers are automatically balanced , Advances in Neural Information Processing Sys- tems, 31 (2018)

    2018

  11. [18]

    R. Ge, C. Jin, and Y. Zheng, No spurious local minima in nonconvex low rank problems: A unified geometric analysis , in International Conference on Machine Learning, PMLR, 2017, pp. 1233–1242

    2017

  12. [19]

    Gunasekar, J

    S. Gunasekar, J. D. Lee, D. Soudry, and N. Srebro , Implicit bias of gradient descent on linear convolutional networks , Advances in Neural Information Processing Systems, 31 (2018)

    2018

  13. [20]

    Gunasekar, B

    S. Gunasekar, B. E. Woodworth, S. Bhojanapalli, B. Neyshabur, and N. Srebro , Implicit regularization in matrix factorization , in Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Be...

    2017

  14. [21]

    E. P. Hsu , Stochastic analysis on manifolds , vol. 38 of Graduate Studies in Mathematics, American Mathematical Society, Providence, RI, 2002

    2002

  15. [22]

    Huang, D

    C.-P. Huang, D. Inauen, and G. Menon , Motion by mean curvature and Dyson Brownian motion, Electronic Communications in Probability, 28 (2023), pp. 1–10

    2023

  16. [23]

    Ikeda and S

    N. Ikeda and S. W atanabe, Stochastic differential equations and diffusion processes, vol. 24 of North-Holland Mathematical Library, North-Holland Publishing Co., Amsterdam; Kodan- sha, Ltd., Tokyo, second ed., 1989

    1989

  17. [24]

    Inauen and G

    D. Inauen and G. Menon , Stochastic Nash evolution , arXiv preprint arXiv:2312.06541, (2023)

    2023 arXiv

  18. [25]

    K. Kohn, T. Merkh, G. Mont ´ufar, and M. Trager , Geometry of linear convolutional networks, SIAM J. Appl. Algebra Geom., 6 (2022), pp. 368–406

    2022

  19. [26]

    K. Kohn, G. Mont´ufar, V. Shahverdi, and M. Trager, Function space and critical points of linear convolutional networks , SIAM J. Appl. Algebra Geom., 8 (2024), pp. 333–362

    2024

  20. [27]

    L ojasiewicz, Sur les trajectoires du gradient d’une fonction analytique , in Geometry seminars, 1982–1983 (Bologna, 1982/1983), Univ

    S. L ojasiewicz, Sur les trajectoires du gradient d’une fonction analytique , in Geometry seminars, 1982–1983 (Bologna, 1982/1983), Univ. Stud. Bologna, Bologna, 1984, pp. 115– 117

    1982

  21. [28]

    Lapedes and R

    A. Lapedes and R. F arber , How neural nets work , in Evolution, learning and cognition, World Sci. Publ., Teaneck, NJ, 1988, pp. 331–346

    1988

  22. [29]

    J. P. LaSalle , Stability theory and invariance principles , in Dynamical systems (Proc. In- ternat. Sympos., Brown Univ., Providence, R.I., 1974), Vol. I, Academic Press, New York- London, 1976, pp. 211–222

    1974

  23. [30]

    J. M. Lee, Introduction to Riemannian manifolds , vol. 176 of Graduate Texts in Mathemat- ics, Springer, Cham, second ed., 2018

    2018

  24. [31]

    Marcotte, R

    S. Marcotte, R. Gribonval, and G. Peyr ´e, Abide by the law and follow the flow: Con- servation laws for gradient flows , Advances in Neural Information Processing Systems, 36 (2024)

    2024

  25. [32]

    , Keep the momentum: Conservation laws beyond euclidean gradient flows , arXiv preprint arXiv:2405.12888, (2024)

    2024 arXiv

  26. [33]

    Menon, Pattern theory: old and new , Lecture notes, Brown University, June 2023

    G. Menon, Pattern theory: old and new , Lecture notes, Brown University, June 2023

    2023

  27. [35]

    Milnor, Morse theory, vol

    J. Milnor, Morse theory, vol. No. 51 of Annals of Mathematics Studies, Princeton University Press, Princeton, NJ, 1963. Based on lecture notes by M. Spivak and R. Wells

    1963

  28. [36]

    G. M. Nguegnang, H. Rauhut, and U. Terstiege , Convergence of gradient descent for learning linear neural networks , 2021

    2021

  29. [37]

    D. A. Roberts, S. Yaida, and B. Hanin , The principles of deep learning theory , vol. 46, Cambridge University Press, Cambridge, MA, USA, 2022

    2022

  30. [38]

    A. M. Saxe, J. L. McClelland, and S. Ganguli, Exact solutions to the nonlinear dynamics of learning in deep linear neural networks , arXiv preprint arXiv:1312.6120, (2013)

    2013 arXiv

  31. [39]

    11537–11546

    , A mathematical theory of semantic development in deep neural networks, Proceedings of the National Academy of Sciences, 116 (2019), pp. 11537–11546. 38 GOVIND MENON

    2019

  32. [40]

    Simon , Theorems on regularity and singularity of energy minimizing maps , Lectures in Mathematics ETH Z¨ urich, Birkh¨ auser Verlag, Basel, 1996

    L. Simon , Theorems on regularity and singularity of energy minimizing maps , Lectures in Mathematics ETH Z¨ urich, Birkh¨ auser Verlag, Basel, 1996. Based on lecture notes by Norbert Hungerb¨ uhler. Division of Applied Mathematics, Brown University, 182 George St., Providence...

    1996

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.