Pith. sign in

REVIEW 4 major objections 4 minor 36 references

Random weights of DNNs and emergence of fixed points

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read In square DNNs, the tail of random weights controls the fixed-point landscape.

desk verdict Interesting but unproven: heavy-tail fixed point multiplicity is plausible, yet the depth-dependence claim rests on a schematic and grid-counting assumptions. read the letter →

arxiv 2501.04182 v2 pith:3V7NMXKE submitted 2025-01-07 cs.LG cs.AIcs.NAmath.NA

L. Berlyand , O. Krupchytskyi , V. Slavin This is my paper · ORCID
classification cs.LGcs.AIcs.NAmath.NA MSC 68T0737C25
keywords fixedpointsrandomweightsheavy-taileddistributionslight-tailedautoencodernetworkscontractionmappingbasinsofattractiondeepneural
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper studies fully-connected feedforward networks whose input and output dimensions are equal (autoencoder-type maps), asking how many stable fixed points such a map has when its weight matrices are drawn i.i.d. at random. The central claim is that the tail of the weight distribution, not just the architecture or the training procedure, decides the fixed-point set: Gaussian (light-tailed) initialization produces a single stable fixed point whose basin of attraction is the whole domain, while Cauchy (heavy-tailed) initialization produces several stable fixed points whose basins partition the domain. The number of fixed points $Q(L)$ depends on depth $L$ non-monotonically, first growing and then falling back to one. The paper further reports that in trained DNNs with standard initialization, multiple fixed points form at positions coinciding with the 'true' images of the training set, with each basin containing its training set. This matters because it connects the widely observed heavy-tailed statistics of trained weights to a concrete, observable property of the network map: how many distinct inputs it can stably recognise.

What carries the argument

The central object is the layer map $\Phi_l(x)=\phi(W_l x+b_l)$ and its composition $\Phi=\Phi_{L-1}\circ\cdots\circ\Phi_0$; fixed points and basins are computed numerically by iterating $x_{m+1}=\Phi(x_m)$ from grid points with step $\delta=0.05$, convergence tolerance $\epsilon=10^{-5}$ and at most $N_0=50$ steps. The argument's load-bearing mechanism is the Banach contraction mapping: for light-tailed weights the layer normalization $\sigma=N^{-\beta}$ with $\beta\geq\beta_{cr}\approx 1/2$ keeps the single-layer Lipschitz constant $g_0<1$, so $g=g_0^L<1$ and the composed map has a unique attracting fixed point. For heavy-tailed weights the contraction condition fails in parts of the domain, allowing several stable fixed points whose basins of attraction are precisely the regions where the iteration converges. The critical exponent $\beta_{cr}$ and the exponential composition law $g=g_0^L$ are what carry the depth dependence.

What would settle it

Repeat the Cauchy-initialized depth-$L=5$ and $L=20$ experiments with a finer grid (e.g., $\delta=0.005$) and with random starting points off the grid; if the count $Q$ changes materially, or if $L=20$ yields more than one stable fixed point from off-grid initializations, then the reported non-monotone $Q(L)$ is a grid artifact. A cleaner test is to run iteration (5) from several thousand uniformly random points, count distinct limits, and compare the result with the grid-based $Q$.

Watch

Extended reading notes

Core claim

On the paper's own terms: for $\Phi(x)=x_L$ built from layers $x_{l+1}=\phi(W_l x_l+b_l)$ with $n_0=n_L$, and with i.i.d. entries in $W_l,b_l$, the tail of the entry distribution determines the number and stability of fixed points. For Gaussian entries normalized with variance $\sigma_l^2=(n_l)^{-1}$, numerical iteration from a grid on $\Omega=[-1,1]^2$ finds exactly one stable fixed point, $x^*=0$, with the whole grid converging to it; the map satisfies the Banach contraction criterion $g<1$ when the variance exponent $\beta$ in $\sigma=N^{-\beta}$ is at least $\beta_{cr}\approx 1/2$ (for odd activations), and $g=g_0^L$ grows or decays exponentially with depth. For Cauchy entries with the same normalization, depths $L=2,3,5$ yield $Q=2,3,5$ stable fixed points, each with a basin of attraction partitioning $\Omega$, whereas $L=20$ returns $Q=1$; hence $Q(L)$ is non-monotone. In trained DNNs with Gaussian initialization and a two-dimensional toy photo-encoding task, fixed points after training coincide numerically with the centers of the training-set circles, and each basin contains the corresponding training set. The paper treats these as numerical evidence and a computer-assisted contraction proof rather than a rigorous theorem.

Load-bearing premise

The reported fixed-point counts assume that every true fixed point has a basin of attraction large enough to contain at least one grid point (step $\delta=0.05$) and that iteration (5) converges within $N_0=50$ steps to tolerance $\epsilon=10^{-5}$; if a basin is smaller than a grid cell, that fixed point is missed and the non-monotone $Q(L)$ curve could be an artifact of discretization.

Editorial extensions

If this is right

  • With standard Gaussian initialization, an untrained autoencoder-type network generically has one stable fixed point, so it cannot by itself distinguish multiple 'true' inputs; recognition requires training.
  • When weights acquire heavy-tailed statistics — as training is claimed to do — the network generically develops multiple stable fixed points whose basins partition the input domain, giving a dynamical account of how distinct learned patterns are stored.
  • The non-monotone dependence of $Q(L)$ on depth means that for a fixed width there is an intermediate depth range that maximizes the number of attractors; architecture choice can therefore tune how many patterns a random network can hold.
  • In trained networks, fixed points coincide with training-set centers and each basin contains its training set, so the set of attracting fixed points can serve as a stopping or cross-validation diagnostic.
  • The contraction parameter $g$ follows $g=g_0^L$, so depth composes the single-layer contraction factor; changing the variance exponent $\beta$ near $\beta_{cr}$ should sharply switch the network between one and many fixed points.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the reported $Q(L)$ counts only attractors whose basins contain at least one grid point with $\delta=0.05$; refining the grid should reveal smaller basins if they exist, so the true fixed-point count is likely at least the reported $Q$, and the non-monotone peak could be sharper or shifted.
  • Beyond the paper: the mechanism behind non-monotone $Q(L)$ may be a bifurcation as the effective Jacobian norm of the random composition crosses 1; one could test this by computing the largest Lyapunov exponent of the weight matrices as a function of $L$ and checking whether the peak in $Q$ coincides with the transition.
  • Beyond the paper: because the trained-DNN experiment uses two-dimensional toy data and HardTanh activation, the same 'fixed points coincide with class centers' finding could be tested on real autoencoders with higher-dimensional images, checking whether the number of stable fixed points matches the number of classes.
  • Beyond the paper: the critical normalization $\beta_{cr}\approx 1/2$ for odd activations matches the standard scaling used in practical initialization; measuring $g$ at finite width near $\beta=1/2$ could predict where $Q$ jumps from 1 to multiple fixed points without extensive grid searches.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper studies fixed points of fully connected feedforward networks with equal input and output dimensions. It reports numerical experiments claiming that Gaussian i.i.d. weight initialization gives a single stable fixed point for a range of architectures, while Cauchy initialization gives multiple stable fixed points whose basins partition the input domain, and that the number of fixed points Q(L) depends non-monotonically on depth L. A toy trained autoencoder is used to show that fixed points coincide with training centers, and a contraction-mapping analysis for Gaussian weights identifies beta_cr approximately 1/2 for Tanh and approximately 1/4 for Sigmoid.

Significance. If correct, the tail behavior of random weights would be an important determinant of the fixed-point landscape, connecting heavy-tailed self-regularization to a concrete dynamical-systems phenomenon. The paper makes clear falsifiable predictions and provides an explicit numerical protocol, and the CLT scaling argument for beta_cr is a useful heuristic. However, the central non-monotone Q(L) claim and the trained-network verification need substantially stronger evidence before the conclusions can be accepted.

major comments (4)
  1. [Section III and Fig. 2] The non-monotone Q(L) claim is not supported by the reported data. Fig. 2 is explicitly labeled 'schematic illustration', no table of Q values is given, no number of random realizations or variance is reported, and Q is only described as a mode. The grid counting rule in Eqs. (4)-(5) silently assumes every true fixed point has a basin containing at least one grid point and that iteration (5) converges within N0=50 steps; for non-contractive Cauchy maps this can miss small-basin fixed points, misclassify slow orbits, or confuse limit cycles with fixed points. A resolution study and ensemble statistics are needed before the depth dependence can be claimed.
  2. [Section IV, Eq. (3)] The trained-DNN experiment does not independently verify the heavy-tail mechanism. The loss in Eq. (3) explicitly drives Phi(x) toward the chosen centers x*_k for all training inputs, including x*_k itself, so the appearance of fixed points near x*_k is partly built into the training objective. To support the claimed analogy with untrained heavy-tailed DNNs, the authors should either measure the empirical weight distribution after training and compare its tail behavior with untrained Cauchy DNNs, or design a training objective that does not directly target the fixed points.
  3. [Section V, Eq. (8)] The statement that for Cauchy distributions there is 'full correspondence between the areas of contraction of Phi (i.e., the areas where g<1) and basins' is inconsistent with the definition of g in Eq. (8) as the global maximum of the ratio over all pairs of grid points. A single global constant cannot localize regions where contraction holds; the authors need to define and compute a local or restricted contraction constant if they wish to make claims about areas of contraction.
  4. [Section V, heading] Calling Section V a 'computer-assisted proof' is an overstatement. The contraction check is performed on a finite grid, and the beta_cr argument relies on a CLT variance scaling rather than a rigorous bound valid for all weight matrices and architectures. The text should be rephrased as numerical evidence and should explicitly state the finite-sample, finite-grid nature of the verification.
minor comments (4)
  1. [Fig. 2] The figure contains garbled placeholder text such as '/s48 /s53' and is described as 'schematic'; it should be replaced by a data plot with error bars or a table of the underlying Q values.
  2. [Section III] The 'weak similarity effect' is invoked to justify the claim that the activation function does not change the number of fixed points, but the term is not defined or explained in this manuscript; at minimum a precise statement or a more specific reference is needed.
  3. [Captions and references] There are several typos and inconsistencies, including 'employer' for 'employee' in the Fig. 3 caption, 'N711 4' in the acknowledgments, and 'IEEE Acess' in reference 22; the manuscripts should be carefully proofread.
  4. [Reproducibility] The manuscript does not state whether the code or the full numerical data will be made available; for a paper whose claims are entirely numerical, a data-availability statement or a link to code is strongly recommended.

Circularity Check

2 steps flagged · score 6.0 of 10

Trained-DNN verification reduces to the loss objective; activation universality rests on a self-citation; the untrained heavy/light-tail comparison is self-contained.

  1. fitted input called prediction [Section IV, around Eq. (3) and Fig. 3]
    "The loss function is chosen in the form (3). After training we search for FPs of the DNN. Similar to the case of untrained DNNs we run the iterative process (5) for each starting point x1 of the set xj,l ⊂ Ω (see (4)). If the process converges, then the corresponding FP is marked as ∗ (see lower part of Fig. 3). Numerically we see that the positions of FPs coincide with 'true' photos x∗ k."

    Loss (3) is L(α)=Σ_k Σ_{x∈T_k} ||Φ(x,α)−x∗_k||^2. Since x∗_k ∈ T_k, every minimizer drives Φ(x∗_k) toward x∗_k. Reporting that the trained DNN's fixed points coincide with x∗_k is therefore a restatement of the training objective, not an emergent phenomenon. The abstract presents this as verification ('then verified by considering DNNs in which the heavy tail distributions arise in training'), but the positions of the FPs are imposed by the choice of loss, so the coincidence is by construction rather than an independent confirmation of the distribution-driven mechanism.

  2. self citation load bearing [Section III, paragraph after Fig. 1(d)]
    "Due to the 'weak similarity' effect 30 the choice of activation function ϕ does not change the number of FPs."

    Reference [30] is Pastur & Slavin (2023), co-authored by V. Slavin, a co-author of this paper. This citation is the sole support for the statement that FP counts are independent of the activation function, a claim that extends the numerical grids of Fig. 1 to 'broad types of architectures'. The present paper provides no independent proof or simulation for arbitrary ϕ regarding FP counts; the breadth claim thus rests on the authors' own prior work being accepted as an external theorem rather than on evidence supplied here.

full rationale

The untrained-DNN part (Sections III and V) is not circular: fixed points are defined by iterating Eq. (5) on the grid (4), and the light-tail/heavy-tail contrast is an empirical observation with explicit parameters (δ=0.05, ε=10^-5, N0=50); no parameter is fitted to the target number of FPs. The contraction parameter g in Eq. (8) is also a direct computation, so that portion has independent content. Two issues raise the score. First, Section IV's trained-DNN 'verification' is circular in the fitted-input-called-prediction pattern: the MSE loss (3) is minimized over training sets T_k that contain the target centers x∗_k, so reporting FPs at x∗_k after training restates the training objective rather than providing an emergent check. Second, the claim that activation function choice does not change FP counts is delegated to reference [30], a co-authored paper (Pastur & Slavin), without independent evidence here; this is load-bearing for the 'broad types of architectures' generality. I also note in-scope limitations that are not circularity: the Q(L) curve is explicitly labelled 'schematic' in Fig. 2, and the grid-based counting rule can miss basins smaller than δ=0.05; these are numerical-support concerns, not definitional circularity. The central untrained light-versus-heavy-tail observation could survive even if the cited universality were removed, so the score is 6 rather than higher.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The core light-tail result has a CLT-based analytical backbone, but the heavy-tail and depth-dependence findings rest on hand-chosen distribution scales, grid resolution, convergence thresholds, and a self-cited activation-independence assumption. No code or data is provided, so these parameters cannot be audited independently.

free parameters (4)
  • beta_cr for Sigmoid activation = approximately 1/4
    Critical scaling exponent separating contraction from non-contraction for a single layer with Sigmoid, read off numerically from Fig. 4b; used to argue activation choice changes the contraction threshold.
  • Grid step delta = 0.05
    Hand-chosen mesh size for enumerating fixed points over [-1,1]^2 in Eq. (4); the reported number of fixed points depends on this resolution.
  • Convergence tolerance epsilon and iteration cap N0 = epsilon=10^-5, N0=50
    Hand-chosen stopping criteria for iteration (5); slow-converging orbits may be declared fixed points or non-converged depending on these values.
  • Cauchy scale gamma_l = gamma_l = n_l^-1
    Scale of the heavy-tailed weight distribution chosen to model Heavy-Tailed Self-Regularization; the specific value affects the number and location of fixed points.
assumptions (5)
  • standard math Banach fixed-point theorem applies when the numerical contraction parameter g < 1
    Invoked in Sections III and V to infer a unique stable fixed point from numerical estimates of g on a finite grid.
  • domain assumption Trained DNN weights develop heavy-tailed empirical spectral distributions (Heavy-Tailed Self-Regularization)
    Imported from refs [27,30] in Section II; used to justify replacing trained DNNs with Cauchy-initialized untrained DNNs.
  • domain assumption Weak similarity effect: activation function does not change the number of fixed points
    Cited from ref [30], by one of the present authors, in Section III; used to generalize FP counts across activations without testing in this paper.
  • domain assumption Odd activation functions do not affect beta_cr because they leave distribution tails unchanged
    Stated in Section V; supports the claim that beta_cr=1/2 holds for Tanh-like activations.
  • standard math Central Limit Theorem for sums of i.i.d. random matrix entries
    Used in Section V to derive the sigma ~ N^-1/2 scaling and the light-tail contraction threshold.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Random weights of DNNs and emergence of fixed points." pith.science (2026). https://pith.science/paper/3V7NMXKE

@misc{pith2026250104182,
  author       = {Pith},
  title        = {Pith review of: Random weights of DNNs and emergence of fixed points},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3V7NMXKE}},
  note         = {Machine review of arXiv:2501.04182}
}
abstract

This paper is concerned with a special class of deep neural networks (DNNs) where the input and the output vectors have the same dimension. Such DNNs are widely used in applications, e.g., autoencoders. The training of such networks can be characterized by their fixed points (FPs). We are concerned with the dependence of the FPs number and their stability on the distribution of randomly initialized DNNs' weight matrices. Specifically, we consider the i.i.d. random weights with heavy and light-tail distributions. Our objectives are twofold. First, the dependence of FPs number and stability of FPs on the type of the distribution tail. Second, the dependence of the number of FPs on the DNNs' architecture. We perform extensive simulations and show that for light tails (e.g., Gaussian), which are typically used for initialization, a single stable FP exists for broad types of architectures. In contrast, for heavy tail distributions (e.g., Cauchy), which typically appear in trained DNNs, a number of FPs emerge. We further observe that these FPs are stable attractors and their basins of attraction partition the domain of input vectors. Finally, we observe an intriguing non-monotone dependence of the number of fixed points $Q(L)$ on the DNNs' depth $L$. The above results were first obtained for untrained DNNs with two types of distributions at initialization and then verified by considering DNNs in which the heavy tail distributions arise in training.

Figures

Figures reproduced from arXiv: 2501.04182 by the authors.

Figure 1
Figure 1. FIG. 1. (a) Normal distribution of the matrix elements and bi [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. FIG. 2. Schematic illustration of the dependence of FP numbe [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. FIG. 3. The simplified model of employer’s photo coding/deco [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: FIG. 4. The dependence of contraction mapping parameter [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: FIG. 5. The dependence of the contraction mapping parameter [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 29 canonical work pages

  1. [1]

    Y. Yan, S. Yang, Y. Wang, J. Zhao, F. Shen, Review Neural Networks about Image Transformation Based on IGC Learning Framework with Annotated Information, arXiv:2206.10155v1 (2022)

    work page Pith review arXiv 2022

  2. [2]

    S. Kaji, S. Kida, Overview of image-to-image translation by use of deep neural networks: denoising, super-resolution, modality conversion, and reconstruction in medical imaging, arXiv:1905.08603 (2019)

    work page Pith review arXiv 2019

  3. [3]

    W. Hong, T. Chen, M. Lu, S. Pu and Z. Ma, Efficient Neural Image Decoding via Fixed-Point Inference, in IEEE Transactions on Circuits and Systems for Video Technology, 31 (9) 3618-3630, (2021)

    work page 2021

  4. [4]

    C. Mou, Q. Wang, J. Zhang, Deep Generalized Unfolding Networks for Image Restoration, arXiv:2204.13348 (2022)

    work page Pith review arXiv 2022

  5. [5]

    Ferster and K

    D. Ferster and K. D. Miller, Neural mechanisms of orientation selectivity in the visual cortex. Annual review of neuroscience, 23 (1), 441-471 (2000)

    work page 2000

  6. [6]

    Ozeki, I

    H. Ozeki, I. M. Finn, E. S. Schaffer, K. D. Miller, and D. Ferster. Inhibitory stabilization of the cortical network underlies visual surround suppression. Neuron, 62 (4), 578–592, 2009

    work page 2009

  7. [7]

    D. B. Rubin, S. D. Van Hooser, and K. D. Miller. The stabilized supralinear network: a unifying circuit motif underlying multi-input integration in sensory cortex. Neuron, 85 (2), 402– 417, 2015

    work page 2015

  8. [8]

    Ebsch and R

    C. Ebsch and R. Rosenbaum. Imbalanced amplification: A mechanism of amplification and suppression from local imbalance of excitation and inhibition in cortical circuits. PLoS computational biology, 14 (3), e1006048, 2018

    work page 2018

Show all 36 references
  1. [9]

    Curto, J

    C. Curto, J. Geneson, and K. Morrison, Fixed points of competitive threshold-linear networks. Neural computation, 31 (1), 94–155, 2019

    2019

  2. [10]

    Hopfield, Neural networks and physical systems with emergent collective computational abilities, Proceedings of the National Academy of Sciences 79 (8), 2554 (1982)

    J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, Proceedings of the National Academy of Sciences 79 (8), 2554 (1982)

    1982

  3. [11]

    J. Hopfield, Neurons with graded response have collective computational properties like those of two-state neurons., Proceedings of the National Academy of Sciences 81 (10), 3088 (1984)

    1984

  4. [12]

    Krotov, J

    D. Krotov, J. Hopfield, Large Associative Memory Problem in Neurobiology and Machine Learning, arXiv:2008.06996 (2021)

    2021 arXiv

  5. [13]

    Kimura, K

    T. Kimura, K. Kato, Analysis of Discrete Modern Hopfield Networks in Open Quantum System, arXiv:2411.02883 (2024)

    2024 arXiv

  6. [14]

    Berlyand, P.-E

    L. Berlyand, P.-E. Jabin, Mathematics of Deep Learning: An Introduction, Walter de Gruyter GmbH & Co KG, 132 pages (2023)

    2023

  7. [15]

    Kingma, M

    D.P. Kingma, M. Welling, An Introduction to Variational Autoencoders, arXiv:1906.02691v3 (2019)

    2019 arXiv

  8. [16]

    J. Wang, R. Cao, N.J. Brandmeir, et al. Face identity coding in the deep neural network and primate brain. Commun Biol 5 , 611 (2022)

    2022

  9. [17]

    Piotrowski, R.L.G

    T.J. Piotrowski, R.L.G. Cavalcante, M. Gabor, Fixed points of nonnegative neural networks, Journal of Machine Learning Research, 25 (139) 1-40, arXiv:2106.16239v9 (2024)

    2024 arXiv

  10. [18]

    Buduma, Fundamentals of Deep Learning, O'Reilly Media, Inc., 2-nd edition, 387 p, 2017

    N. Buduma, Fundamentals of Deep Learning, O'Reilly Media, Inc., 2-nd edition, 387 p, 2017

    2017

  11. [19]

    Bahri, J

    Y. Bahri, J. Kadmon, J. Pennington, S. Schoenholz, J. Sohl-Dickstein, and S. Ganguli, Statistical mechanics of deep learning, Annual Review of Condensed Matter Physics, 11 501--528 (2020)

    2020

  12. [20]

    Gallicchio and S

    C. Gallicchio and S. Scardapane, Deep randomized neural networks, in Recent Trends in Learning From Data. Studies in Computational Intelligence, 896 , eds. L. Oneto, N. Navarin, A. Sperduti and D. Anguita (Springer, Heidelberg, 2020)

    2020

  13. [21]

    Giryes, G

    R. Giryes, G. Sapiro and A. M. Bronstein, Deep neural networks with random Gaussian weights: A universal classification strategy?. IEEE Trans. Signal Processes 64 3444 (2016)

    2016

  14. [22]

    Z. Ling, X. He and R. C. Qiu, Spectrum concentration in deep residual learning: a free probability approach, IEEE Acess 7 105212, arxiv:1807.11697 (2019)

    2019 arXiv

  15. [23]

    A. G. de G. Matthews, J. Hron, M. Rowland, R. E. Turner,and Z. Ghahramani. Gaussian process behaviour in wide deep neural networks, Int. Conf. on Learn. Represent, arxiv:1804.1127100952 (2018)

    2018 arXiv

  16. [24]

    Yang, Tensor programs III: neural matrix laws, arxiv:2009.10685v1 (2020)

    G. Yang, Tensor programs III: neural matrix laws, arxiv:2009.10685v1 (2020)

    2020 arXiv

  17. [25]

    Marchenko, L

    V. Marchenko, L. Pastur, The eigenvalue distribution in some ensembles of random matrices, Math. USSR Sbornik 1 , 457 (1967)

    1967

  18. [26]

    Berlyand, E

    L. Berlyand, E. Sandier, Y. Shmalo, and L. Zhang, Enhancing Accuracy in Deep Learning Using Random Matrix Theory, Journal of Machine Learning. 3(4) 347-412, (2024)

    2024

  19. [27]

    Martin and M.W

    C.H. Martin and M.W. Mahoney, Implicit self-regularization in deep neural networks: evidence from random matrix theory and implications for learning, Journal of Machine Learning Research 22 , 1-73 (2021)

    2021

  20. [28]

    Pennington, S

    J. Pennington, S. S. Schoenholz, S. Ganguli, The Emergence of Spectral Universality in Deep Networks. arXiv:1802.09979v1 (2018)

    2018 arXiv

  21. [29]

    Belrose, A

    N. Belrose, A. Scherlis, Understanding gradient descent through the training Jacobian, arXiv:2412.07003v2 (2024)

    2024 arXiv

  22. [30]

    Pastur, V

    L. Pastur, V. Slavin, On random matrices arising in deep neural networks: General iid case, Random Matrices: Theory and Applications 12 (01), 2250046 (2023)

    2023

  23. [31]

    Hoffman, D.A

    J. Hoffman, D.A. Roberts, Sh. Yaida, Robust Learning with Jacobian Regularization, arXiv:1908.02729v1 (2019)

    2019 arXiv

  24. [32]

    Shmalo, J

    Y. Shmalo, J. Jenkins, O. Krupchytskyi, Deep Learning Weight Pruning with RMT-SVD: Increasing Accuracy and Reducing Overfitting, arXiv:2303.08986v1 (2023)

    2023 arXiv

  25. [33]

    Shcherbina, On Universality of Local Edge Regime for the Deformed Gaussian Unitary Ensemble, J

    T. Shcherbina, On Universality of Local Edge Regime for the Deformed Gaussian Unitary Ensemble, J. Stat. Phys. 143 , 455 (2011)

    2011

  26. [34]

    Russo, Deformed Cauchy random matrix ensembles and large N phase transitions

    J.G. Russo, Deformed Cauchy random matrix ensembles and large N phase transitions. J. High Energ. Phys, 14 , 1 2020, (2020)

    2020

  27. [35]

    Hisakado, T

    M. Hisakado, T. Kaneko, Deformation of Marchenko-Pastur distribution for the correlated time series, arXiv:2305.12632v2 (2024)

    2024 arXiv

  28. [36]

    Jachymski, I

    J. Jachymski, I. Jóźwik, M. Terepeta, The Banach Fixed Point Theorem: selected topics from its hundred-year history. Rev. Real Acad. Cienc. Exactas Fis. Nat. Ser. A-Mat. 118 , 140 (2024)

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.