Pith. sign in

REVIEW 4 major objections 5 minor 25 references

Entrywise application of non-linear functions on orthogonally invariant matrices

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper shows that applying a centered nonlinear function entrywise to an orthogonally invariant random matrix leaves the same asymptotic spectrum as a linear combination of the matrix with an independent GOE.

desk verdict A worthwhile structural generalization of Gaussian equivalence to orthogonally invariant matrices, with a solid centered-polynomial core but an abstract that oversells non-polynomial and non-centered cases. read the letter →

arxiv 2412.06943 v1 pith:DXIG6PSU submitted 2024-12-09 math.SP cond-mat.stat-mechcs.LGmath.OA

Roland Speicher , Alexander Wendel This is my paper · ORCID
classification math.SPcond-mat.stat-mechcs.LGmath.OA MSC 60B2046L54
keywords randommatricesorthogonalinvarianceGaussianequivalenceprinciplefreecumulantsentrywisenonlinearfunctionsReLUGOEeigenvaluedistribution
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that for a symmetric random matrix ensemble whose joint entry distribution is unchanged by conjugation with any orthogonal matrix, the entrywise application of a nonlinear function has a surprisingly simple spectral effect: in the large-size limit it is equivalent to a linear combination of the original matrix and an independent Gaussian orthogonal ensemble (GOE). The precise statement is that, for a centered nonlinearity $f$, the matrix $Y_N$ with off-diagonal entries $y_{ij} = N^{-1/2} f(\sqrt{N} x_{ij})$ and zero diagonal has the same asymptotic eigenvalue distribution as $\theta_1 X_N + \theta_2 Z_N$, where $Z_N$ is a standard GOE and the two coefficients are computed from $f$ and from one Gaussian test variable of variance $\kappa_2$, the second free cumulant of the limiting spectrum of $X_N$. The same equivalence is proved for multivariate functions applied entrywise to several matrices, including matrices that are correlated with each other. The paper would matter because it reduces a nonlinear, high-dimensional transformation to a linear model whose spectrum is known, and it explains why such reductions appear across machine-learning settings.

What carries the argument

The engine of the proof is the combinatorial relation between classical cumulants of the matrix entries and free cumulants of the limiting eigenvalue distribution, processed through the Leonov-Shiryaev formula for cumulants of products. Orthogonal invariance forces almost all entry cumulants of $X_N$ to vanish; the only ones surviving asymptotically have a cyclic index structure, and their scaled limits are the free cumulants $\kappa_n$. When the nonlinearity is inserted, Leonov-Shiryaev reorganizes the expanded moments, and the leading cyclic cumulants of $Y_N$ come from one full cycle of the original entries together with Gaussian pairings of the remaining factors. That pairing count evaluates to $\mathbb{E}[f'(g)]^n$ per full cycle, which is the mechanism that produces the linear replacement model.

What would settle it

Take $X_N$ to be a single GOE and apply the centered cubic $f(x)=x^3-3x$ entrywise. The theorem predicts $\theta_1=0$ and that $Y_N$ converges to a semicircle with second moment $\mathbb{E}[f(g)^2]=6$; simulating large $N$ and comparing the empirical spectrum to that semicircle would falsify the equivalence if a systematic deviation beyond finite-size fluctuations appeared. A second check targets the centeredness assumption directly: applying the uncentered ReLU to any of the paper's ensembles should produce an eigenvalue of order $\sqrt{N}$ outside the bulk, while the bulk alone should still match the linear model.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is a cumulant-level equality. For an orthogonally invariant symmetric ensemble $X_N$ with a limit distribution of all orders and a centered polynomial $f$, the asymptotic free cumulants $\kappa_n^f$ of the entrywise-transformed matrix $Y_N$ are $\kappa_1^f=0$, $\kappa_2^f = \mathbb{E}[f(g)^2]-\mathbb{E}[f(g)]^2$, and $\kappa_n^f = \kappa_n \,\mathbb{E}[f'(g)]^n$ for $n\ge 3$, where $g$ is Gaussian with mean zero and variance $\kappa_2$. These are exactly the free cumulants of $\theta_1 X_N + \theta_2 Z_N$ with $\theta_1=\mathbb{E}[f'(g)]$ and $\theta_2 = \sqrt{\mathbb{E}[f(g)^2]-\mathbb{E}[f(g)]^2-\kappa_2 \mathbb{E}[f'(g)]^2}$, so the nonlinear operation never creates spectral structure beyond a rescaling of $X_N$ plus an independent GOE. In the multivariate case the formula replaces $f'$ by partial derivatives and $\kappa_n$ by mixed free cumulants, with the second-order mixed cumulants entering the coefficient of the GOE term.

Load-bearing premise

The load-bearing premise is that the nonlinear function is centered, so the transformed off-diagonal entries have mean zero; the paper's own max-function example shows that without this condition an extra large eigenvalue appears and the full spectral claim fails.

Editorial extensions

If this is right

  • For every centered polynomial nonlinearity, the bulk spectrum of the transformed matrix is computable as the free convolution of a rescaled copy of the original spectrum with a semicircle.
  • If $\theta_1 = \mathbb{E}[f'(g)] = 0$, which happens for even functions, the original matrix's spectral shape disappears entirely and $Y_N$ becomes asymptotically semicircular.
  • For multivariate functions, the asymptotic effect of functions such as $\max(x_1,x_2)$ on two ensembles is the same as $\theta_1 X_N^{(1)} + \theta_2 X_N^{(2)} + \theta Z_N$, with coefficients given by the Gaussian expectations of the corresponding partial derivatives.
  • The equivalence also holds when the input matrices are correlated, as long as the whole tuple is jointly orthogonally invariant; the mixed free cumulants then determine the coefficients.
  • Uncentered functions are a genuine boundary: the bulk still follows the linear model, but the nonzero mean produces an additional outlier eigenvalue, so the centeredness assumption is not a formality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extrapolation the paper leaves open is that not only the eigenvalue distribution but also finer spectral quantities, such as eigenvector overlaps or linear-statistics fluctuations, of $Y_N$ are inherited from the linear model $\theta_1 X_N + \theta_2 Z_N$; this would be a testable conjecture, not a proven consequence.
  • Because the theorem is proved only for polynomials while the numerical examples use ReLU and max, a quantitative finite-$N$ check of the rate of convergence for non-polynomial functions would indicate whether the polynomial restriction is a proof artifact or a genuine boundary.
  • In applications where entrywise activations are applied to random matrices with nonzero mean, subtracting the mean before the nonlinearity should remove the outlier and restore the clean Gaussian-equivalence description; this is an actionable design rule suggested by the paper's outlier remark.
  • The result suggests a general heuristic: rotational invariance, not the particular entry distribution, is what makes nonlinear preprocessing 'linearize' in the spectrum; ensembles that are only approximately invariant may show the same equivalence up to corrections controlled by the violation of invariance.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies the entrywise application of nonlinear functions to symmetric orthogonally invariant random matrices, proposing a Gaussian equivalence principle: the asymptotic eigenvalue distribution of Y_N (defined by y_ij = N^{-1/2} f(N^{1/2} x_ij) off the diagonal, zero on the diagonal) should equal that of a linear combination theta_1 X_N + theta_2 Z_N, where Z_N is an independent GOE. The main theorems (Theorems 4, 12, and 14) derive the free cumulants of Y_N in the case where f is a polynomial and Assumption 2 (centeredness of f with respect to the off-diagonal entry distribution) holds, and the corollaries give explicit linear models. Numerical examples for the ReLU function (Section 5) and the max function (Sections 7 and 8) are presented as evidence for the general principle. The abstract claims the result for general non-linear functions, including multivariate and correlated cases, but the proofs are restricted to centered polynomials.

Significance. If fully established, the result would provide a broad Gaussian equivalence principle for orthogonally invariant ensembles, unifying and extending existing results for Wigner-type and kernel random matrices, and would give explicit coefficients computable from the activation function and the second free cumulant. The use of classical cumulants and the Leonov--Shiryaev formula to pass from entrywise nonlinearities to free cumulants is conceptually appealing and yields clean formulas that agree with the bulk numerical spectra shown in the paper. However, as it stands, the advertised general claim is not proven: the theorems require f to be a centered polynomial, while the showcased ReLU and max functions are neither centered nor polynomial, and the proof of the crucial subcycle/disconnected-cycle cancellations is informal. The paper does not supply machine-checked proofs or reproducible code, so the numerical agreement should be regarded as illustrative evidence only. With the claims properly calibrated, the paper would make a useful contribution.

major comments (4)
  1. [Abstract and Section 3 (Assumption 2)] The abstract states that a Gaussian equivalence principle holds for entrywise application of non-linear functions in general, but Theorem 4 is proved only for polynomials satisfying Assumption 2. The examples in Section 5 (ReLU) and Section 7 (max) are not centered; their Gaussian means are sqrt(5/(2 pi)) and 1/sqrt(pi), respectively. Consequently, the numerical agreement in Figures 1 and 3 cannot be taken as evidence for the stated general theorem; at most it supports a bulk-limit statement. The authors should either restrict the abstract and theorem statements to the centered polynomial case or provide the missing argument for general f, and the outlier phenomenon should be acknowledged in Section 5 just as it is in Section 7.
  2. [Section 3, continuation of the proof after Definition 8] The treatment of subcycles and disconnected cycles is not rigorous. The proof relies on phrases such as "Careful analysis also reveals" and "It is clear that this construction can lead to partitions pi which are crossing" without giving the counting of contributing multi-indices, and the estimate with the parameters d and d_j is introduced informally. The later conclusion that crossing partitions contribute O(N^{-1}) is asserted rather than proven. Since the free moment-cumulant formula for Y_N depends on exactly these cancellations, this is a load-bearing gap in the proof of Theorem 4.
  3. [Corollary 11 and Assumption 2] Corollary 11 is stated as an equality of asymptotic eigenvalue distributions, but under Assumption 2 it is at most a statement about the bulk spectrum. For a non-centered f, the matrix Y_N contains a rank-one term of size mu N^{-1/2} up to diagonal corrections, producing an eigenvalue of order sqrt(N) that is not present in theta_1 X_N + theta_2 Z_N; the proof of Theorem 4 uses Assumption 2 to eliminate c_1(y_ij) before the Leonov--Shiryaev expansion. The manuscript should explicitly state that Corollary 11 and the related multivariate corollaries hold only for centered f, and it should describe the outlier separately, as it does in the max example.
  4. [Theorem 12 and proof of Corollary 13] Theorem 12 is stated without the polynomial restriction, but the proof of Corollary 13 begins "Let us assume that our function f is a (multivariate) polynomial." This mismatch between statement and proof leaves the theorem unproved for arbitrary f. Similarly, the proof of Theorem 14 treats only the full-cycle case and refers to the univariate argument for multiple cycles without addressing the correlated-matrix complications in detail. The statements should be aligned with what is actually proven.
minor comments (5)
  1. [Section 4] The name "Cauchy-Schwartz" should be "Cauchy-Schwarz."
  2. [Section 5] "It is feasible that the statements ... are also true" should be "It is plausible that ... are also true."
  3. [Proposition 3 proof] There is a typo: "cyclity" should be "cyclicity."
  4. [Section 2] The word "similiar" should be "similar."
  5. [Section 7] The sentence about the eigenvalue of order 40 would be more informative if it gave the predicted scaling, e.g., sqrt(N) times the Gaussian mean of max(g1,g2).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the Gaussian equivalence coefficients θ1 and θ2 are computed in closed form from f and the free cumulants of X_N, then compared with independent simulations; the derivation never fits or redefines the target.

full rationale

The derivation is self-contained and non-circular. Theorem 4 (with Proposition 3) computes the limiting free cumulants κ^f_n of Y_N directly from the cyclic classical cumulants of X_N via the Leonov–Shiryaev formula; the paper provides its own proof and an appendix proof of the orthogonal cumulant-scaling statement (Proposition 1) rather than relying on an unverified self-citation. Corollary 11 then defines the linear model Y_hat_N = θ1 X_N + θ2 Z_N with θ1 = E[f'(g)] and θ2 selected so that the free cumulants match, and proves equality by a one-line calculation using asymptotic freeness of Z_N from X_N. These parameters are closed-form functions of f and κ2, not fitted to Y_N; the numerical ReLU and max comparisons use independently simulated matrices, so the prediction is not forced by construction. The non-centeredness of ReLU and max is a substantive scope limitation: the proof of κ^f_1 = 0 relies on Assumption 2, and the outlier eigenvalue noted for max lies outside the theorem's bulk statement. That affects correctness, generality, and outlier claims, but it is not circularity because the theorem does not assume the conclusion. The self-citations ([MS17], [NS06], [CMSS07]) are standard background or are accompanied by proofs, and no uniqueness theorem from the authors' prior work is invoked to forbid alternatives. No circular step can be exhibited.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The derivation is honest: the only genuinely new input is the class of orthogonally invariant ensembles with all-order limits and the polynomial/centeredness restriction. All other ingredients are standard free probability and cumulant tools. No parameters are fitted to the target spectra; theta1 and theta2 are functions of f and the known kappa2. The main uncharged assumption is the unproved extension beyond polynomials and centered functions, which the paper itself flags as future work.

assumptions (6)
  • domain assumption X_N is a symmetric orthogonally invariant random matrix ensemble with a limit distribution of all orders (all limits in Eq. (1) exist).
    This defines the input class and is the basis for Proposition 1, which controls the order of entry cumulants. It is assumed rather than proven.
  • domain assumption The joint distribution of the l-tuple (X_N^(1),...,X_N^(l)) is orthogonally invariant in the correlated multivariate case.
    Section 8 requires invariance of the tuple to define mixed free cumulants kappa_n^{(r1,...,rn)} and to control mixed entry cumulants.
  • ad hoc to paper f is a polynomial (Theorems 4, 12, 14) and is centered with respect to the off-diagonal entry distribution (Assumption 2).
    The cumulant proof uses multilinearity and first-cumulant vanishing. The abstract and the ReLU/max examples go beyond these restrictions, so the general claim is not covered.
  • standard math Classical cumulant machinery: Leonov-Shiryaev formula (Theorem 17), Moebius inversion, and the free moment-cumulant formula.
    Used throughout to pass from cumulants of entries to moments of traces; stated in the appendix or cited to [NS06, MS17].
  • standard math Independent GOE Z_N is asymptotically free from the orthogonally invariant ensemble X_N (and from each other in the multivariate linear models).
    Corollary 11 proof adds the free cumulants of theta1 X_N and theta2 Z_N; this standard asymptotic freeness fact is invoked without a dedicated reference.
  • standard math The Weingarten calculus for Haar orthogonal matrices gives the Wick-type formula used in Appendix B.
    Used to connect averaged entry moments to macroscopic correlation moments; cited to [Col03, CS06].

how reviews work

0 comments
Cite this review

Pith. "Pith review of Entrywise application of non-linear functions on orthogonally invariant matrices." pith.science (2026). https://pith.science/paper/DXIG6PSU

@misc{pith2026241206943,
  author       = {Pith},
  title        = {Pith review of: Entrywise application of non-linear functions on orthogonally invariant matrices},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DXIG6PSU}},
  note         = {Machine review of arXiv:2412.06943}
}
read the original abstract

In this article, we investigate how the entrywise application of a non-linear function to symmetric orthogonally invariant random matrix ensembles alters the spectral distribution. We treat also the multivariate case where we apply multivariate functions to entries of several orthogonally invariant matrices; where even correlations between the matrices are allowed. We find that in all those cases a Gaussian equivalence principle holds, that is, the asymptotic effect of the non-linear function is the same as taking a linear combination of the involved matrices and an additional independent GOE. The ReLU-function in the case of one matrix and the max-function in the case of two matrices provide illustrative examples.

Figures

Figures reproduced from arXiv: 2412.06943 by the authors.

Figure 1
Figure 1. The effect of entrywise application of ReLU on XN from Equation (4) and comparison to the Gaussian equivalent YˆN from Equation (5); N = 5000 By the same kind of arguments as before we get then the following description of the asymptotic free cumulants of YN . Theorem 12. We have the following relation between the free cumulants κ (1) n , . . . , κ (l) n of the asymptotic eigenvalue distributions of X (1) N , . . . … view at source ↗
Figure 2
Figure 2. The effect of entrywise application of max on X (1) N and X (2) N from Equation (7); N = 5000 it will become important to deal with this when one considers questions of strong convergence and outliers. This eigenvalue comes from the non-vanishing mean of the entries of the matrix YN .) In [PITH_FULL_IMAGE:figures/full_fig_p019_2.png] view at source ↗
Figure 3
Figure 3. Superposition of the eigenvalue distributions of the non-linear matrix YN and of its Gaussian equivalent YˆN from Equa￾tion (8); N = 5000 cumulants in the l matrices, κ (r1,...,rn) n := lim N→∞ N n−1 cn(x (r1) i1i2 , x (r2) i2i3 , . . . , x (rn) ini1 ). Of those mixed free cumulants only the second ones will be relevant for the effect of the non-linearities. Theorem 14. We have the following relation between the fre… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Superposition of the eigenvalues of the non-linear ma￾trix YN , given as the max of the correlated matrices X (1) N and X (2) N from Equation (9), and of its Gaussian equivalent YˆN from Equa￾tion (10); N = 5000 c2(h (r) i , h(s) j ) = δijκ (r,s) 2 . Then, we will obta…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 24 canonical work pages

  1. [1]

    118, Cambridge university press, 2010

    Greg W Anderson, Alice Guionnet, and Ofer Zeitouni, An introduction to random matrices, no. 118, Cambridge university press, 2010

    work page 2010

  2. [2]

    Denis Bernard and Ludwig Hruza, Structured random matrices and cyclic cumulants: A free probability approach, Random Matrices: Theory and Applications (2024)

    work page 2024

  3. [3]

    Lucas Benigni and Sandrine P \'e ch \'e , Eigenvalue distribution of some nonlinear models of random matrices, Electronic Journal of Probability 26 (2021), 1--37

    work page 2021

  4. [4]

    Capitaine and M

    M. Capitaine and M. Casalis, Cumulants for random matrices as convolutions on the symmetric group, ii, Journal of Theoretical Probability 20 (2007), no. 3, 505--533

    work page 2007

  5. [5]

    Beno \^ t Collins, Sho Matsumoto, and Jonathan Novak, The W eingarten calculus , Notices of the American Mathematical Society 69 (2022), no. 5

    work page 2022

  6. [6]

    III : Higher order freeness and free cumulants , Documenta Mathematica 12 (2007), 1--70

    Beno \^ t Collins, James A Mingo, Piotr \'S niady, and Roland Speicher, Second order freeness and fluctuations of random matrices. III : Higher order freeness and free cumulants , Documenta Mathematica 12 (2007), 1--70

    work page 2007

  7. [7]

    17, 953--982

    Beno \^ t Collins, Moments and cumulants of polynomial random variables on unitary groups, the I tzykson- Z uber integral, and free probability , International Mathematics Research Notices 2003 (2003), no. 17, 953--982

    work page 2003

  8. [8]

    3, 773--795

    Beno \^ t Collins and Piotr \'S niady, Integration with respect to the H aar measure on unitary, orthogonal and symplectic group , Communications in Mathematical Physics 264 (2006), no. 3, 773--795

    work page 2006

Show all 25 references
  1. [9]

    04, 1350010

    Xiuyuan Cheng and Amit Singer, The spectrum of random inner-product kernel matrices, Random Matrices: Theory and Applications 2 (2013), no. 04, 1350010

    2013

  2. [10]

    Noureddine El Karoui, The spectrum of kernel random matrices, Ann. Statist. 38 (2010), no. 1, 1--50

    2010

  3. [11]

    Michele Fava, Jorge Kurchan, and Silvia Pappalardi, Designs via free probability, arXiv preprint arXiv:2308.06200 (2023)

    2023 arXiv

  4. [12]

    Zhou Fan and Andrea Montanari, The spectral norm of random inner-product kernel matrices, Probability Theory and Related Fields 173 (2019), 27--85

    2019

  5. [13]

    Zhou Fan and Zhichao Wang, Spectra of the conjugate kernel and neural tangent kernel for linear-width neural networks, Advances in neural information processing systems 33 (2020), 7710--7721

    2020

  6. [14]

    426--471

    Sebastian Goldt, Bruno Loureiro, Galen Reeves, Florent Krzakala, Marc M \'e zard, and Lenka Zdeborov \'a , The G aussian equivalence of generative models for learning with shallow neural networks , Mathematical and Scientific Machine Learning, PMLR, 2022, pp. 426--471

    2022

  7. [15]

    Siddharth Jindal and Pavan Hosur, Generalized free cumulants for quantum chaotic systems, Journal of High Energy Physics 2024 (2024), no. 9, 66

    2024

  8. [16]

    2, 1190--1248

    Cosme Louart, Zhenyu Liao, and Romain Couillet, A random matrix approach to neural networks, The Annals of Applied Probability 28 (2018), no. 2, 1190--1248

    2018

  9. [17]

    3, 319--329

    Valerii P Leonov and Albert Nikolaevich Shiryaev, On a method of calculation of semi-invariants, Theory of Probability & its applications 4 (1959), no. 3, 319--329

    1959

  10. [18]

    11, 113301

    Antoine Maillard, Laura Foini, Alejandro Lage Castellanos, Florent Krzakala, Marc Mézard, and Lenka Zdeborová, High-temperature expansions and message passing algorithms, Journal of Statistical Mechanics: Theory and Experiment 2019 (2019), no. 11, 113301

    2019

  11. [19]

    James A Mingo and Mihai Popa, Real second order freeness and H aar orthogonal matrices , Journal of Mathematical Physics 54 (2013), no. 5

    2013

  12. [20]

    James A Mingo and Roland Speicher, Free probability and random matrices, Fields Institute Monographs (2017)

    2017

  13. [21]

    13, Cambridge University Press, 2006

    Alexandru Nica and Roland Speicher, Lectures on the combinatorics of free probability, vol. 13, Cambridge University Press, 2006

    2006

  14. [22]

    17, 170603

    Silvia Pappalardi, Laura Foini, and Jorge Kurchan, Eigenstate thermalization hypothesis and free probability, Physical Review Letters 129 (2022), no. 17, 170603

    2022

  15. [23]

    Vanessa Piccolo and Dominik Schr \"o der, Analysis of one-hidden-layer neural networks via the resolvent method, Advances in Neural Information Processing Systems 34 (2021), 5225--5235

    2021

  16. [24]

    Jeffrey Pennington and Pratik Worah, Nonlinear random matrix theory for deep learning, Advances in neural information processing systems 30 (2017)

    2017

  17. [25]

    T heory of M \"o bius functions , Probability Theory and Related Fields 2 (1964), no

    Gian-Carlo Rota, On the foundations of combinatorial theory I . T heory of M \"o bius functions , Probability Theory and Related Fields 2 (1964), no. 4, 340--368

    1964

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.