Pith. sign in

REVIEW 3 major objections 9 minor 43 references

Quantitative Gaussian-Process limits of Tensor Programs

T0 review · 3 major / 9 minor · reviewed 2026-07-08 · glm-5.2

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Finite-width neural nets converge to Gaussian limits at rate 1/√n

desk verdict Solid quantitative CLT for tensor programs with weight sharing; the basic Netsor proof is sound, the Netsor K extension has an honest but real limitation. read the letter →

arxiv 2607.06290 v1 pith:R67V5SKE submitted 2026-07-07 cs.LG math.PRstat.ML

Andrea Agazzi , Eloy Mosig García , Dario Trevisan This is my paper
classification cs.LGmath.PRstat.ML
keywords tensorprogramsGaussianprocesslimitWassersteindistanceneuralnetworkwidthcentraltheoremattentionmechanismweightsharingNetsor
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proves that randomly initialized neural networks of essentially any architecture — feed-forward, recurrent, and attention-based — approach their infinite-width Gaussian-process limits at an explicit, quantifiable rate. The central object is the Netsor program, a formal language that expresses a neural network's forward pass as a sequence of matrix multiplications and nonlinearities. The authors show that for any such program with Lipschitz activation functions, the Wasserstein distance between the finite-width network's outputs and the idealized infinite-width Gaussian process is bounded by a constant times the sum of 1/√m over all hidden layer widths m. The proof works by constructing an explicit line-by-line coupling: at each matrix multiplication step, both the finite and infinite executions are conditionally Gaussian, and the authors couple them using shared random noise, bounding the error through a law of large numbers for empirical kernels. A key extension to Netsor K programs adds scalar kernel variables and parametric nonlinearities, covering attention mechanisms like softmax and layer normalization, though under a non-degeneracy assumption on the Gaussian variables that must hold in the infinite-width limit.

What carries the argument

Netsor programs (a formal algebraic language for neural network forward passes); line-by-line inductive coupling via shared Gaussian noise; generalized Schur complements and Greville's formula for conditional Gaussian laws; non-degeneracy reduction to eliminate redundant matrix multiplication lines; Netsor K extension adding scalar kernel variables for attention mechanisms

What would settle it

Construct an attention-based Netsor K program whose G-vars are degenerate in the infinite-width execution, demonstrating that the non-degeneracy assumption is not automatically satisfied and that the convergence bound cannot be applied without verifying it.

Watch

Extended reading notes

Core claim

The paper's central result is a quantitative central limit theorem for tensor programs: the Wasserstein-p distance between a finite-width network's execution and its infinite-width Gaussian-process limit is bounded by c × Σ 1/√m (summed over hidden widths), where c depends only on structural constants. This extends prior quantitative CLTs — which covered only feed-forward networks with independent weights per layer — to architectures with weight sharing (RNNs, multi-input evaluation) and, via the Netsor K extension, to attention layers. The mechanism is a line-by-line inductive coupling that exploits the conditional Gaussian structure of matrix multiplication outputs in both finite and无限-宽度实

Load-bearing premise

For the extension to attention architectures (Netsor K programs, Theorem 2.4), the authors require that the Gaussian variables produced by matrix multiplications are jointly non-degenerate in the infinite-width limit. This property must be verified case-by-case and could fail for certain weight-sharing patterns or input configurations; if it fails, the theorem does not apply.

Editorial extensions

If this is right

  • Practitioners can use the explicit 1/√m bound to estimate how wide a network must be for the Gaussian-process approximation to be accurate within a given tolerance, for architectures beyond simple MLPs.
  • The framework provides a principled way to certify when the NNGP or NTK approximation is valid for recurrent and attention-based architectures at finite width, rather than only asymptotically.
  • The line-by-line coupling technique could be extended to backward passes and gradient computations, potentially yielding quantitative convergence rates for NTK-based training dynamics in architectures with weight sharing.
  • The Netsor K extension with its non-degeneracy assumption identifies a precise mathematical condition that determines which attention configurations admit clean Gaussian-process limits and which do not.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The non-degeneracy assumption for Netsor K programs suggests that certain attention configurations — particularly those with specific weight-sharing patterns or degenerate input geometries — may exhibit convergence to non-Gaussian limits or slower rates, which would be architecturally significant.
  • The gap between the theoretical O(1/√n) bound and the numerically observed slopes (ranging from -0.58 to -0.92) hints that the true convergence rate may be architecture-dependent and potentially sharper than the theorem guarantees, especially for feed-forward networks where O(1/n) rates are known.
  • Since the constant c depends on structural parameters like the number of program lines L, the bound may become loose for very deep networks, suggesting a regime where depth and width interact nontrivially in determining Gaussian approximation quality.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 9 minor

Summary. This paper establishes quantitative Wasserstein-distance convergence rates (of order sum of inverse square roots of hidden widths) between finite-width and infinite-width executions of Netsor tensor programs. The main result (Theorem 1.2) covers architectures with weight sharing (RNNs, multi-input evaluation) that prior feed-forward quantitative CLTs could not handle, via a line-by-line inductive coupling argument. An extension to Netsor K programs (Theorem 2.4) covers attention-type layers under a non-degeneracy assumption on G-vars. The proof uses structure lemmas for conditional Gaussian laws, a non-degeneracy reduction, and a kernel LLN (Corollary 1.3) that serves both as a consequence and an inductive ingredient.

Significance. The paper makes a genuine contribution by extending quantitative Gaussian-process convergence bounds beyond feed-forward architectures to the tensor-program framework, which naturally accommodates weight sharing. The line-by-line induction correctly handles the apparent circularity between Theorem 1.2 and Corollary 1.3 by using the theorem only for truncated programs at each inductive step. The structure lemmas (3.5, 3.6) cleanly characterize conditional laws in both finite and infinite-width executions. The Netsor K extension and the explicit acknowledgment of the non-degeneracy limitation (Remark 2.5) are commendable. The numerical experiments in Section 5, while not theorem-level for non-MLP architectures, provide reasonable empirical support.

major comments (3)
  1. Proposition 3.11, proof of (ii)=>(iii): The argument that Phi's zero set being open requires non-degeneracy of ALL previous G-vars jointly, not just marginally. The proof states: 'Since the (g_bar^(j)) are non-degenerate, the set {z : Phi(z) != 0} has null Lebesgue measure, and by continuity of Phi it is an open set, hence empty.' This step requires that the joint law of (g_bar^(j))_{j=1,...,k'} be non-degenerate (i.e., supported on all of R^{k'}). The non-degeneracy assumption in Lemma 3.9 is stated conditionally ('conditionally upon F_{l-1}'), but the proof of Proposition 3.11(ii)=>(iii) appears to use marginal non-degeneracy of the joint vector. In programs with weight sharing, the parents h^(i) can depend on previously generated G-vars from the same A-var W, so joint non-degeneracy of the G-vars is not automatic from conditional non-degeneracy at each step. The authors should clarify
  2. Proposition 3.11, proof of (ii)=>(iii): The lifting from infinite-width to finite-width linear identities is the load-bearing step for the non-degeneracy reduction (Lemma 3.9), which in turn is needed for differentiability of the maps involving pseudoinverses and matrix square roots in Section 4.2 ('Covariance stability'). The proof argues that since Phi(g_bar^(j)) = 0 a.s. (infinite-width) and the zero set is open, Phi vanishes identically, hence Phi(g^(j)) = 0 a.s. in finite-width too. This works when the coefficients alpha_i are deterministic constants, which they are (computed from infinite-width Gram data). However, the argument that the zero set {z : Phi(z) != 0} is open AND has null Lebesgue measure requires that the joint distribution of (g_bar^(j)) has full support. The proof should explicitly verify that the non-degeneracy hypothesis (conditional non-degeneracy at each MatMul)
  3. Theorem 2.4 (Netsor K extension): The non-degeneracy assumption is stated as 'its G-vars are jointly non-degenerate Gaussians in the infinite-width execution.' Remark 2.5 honestly acknowledges that the reduction does not lift because 'a scalar kernel variable is deterministic in infinite-width but random at finite width, so a linear relation that holds identically in the infinite-width execution need not hold identically in the finite-width execution.' This means Theorem 2.4's coverage of attention architectures is conditional on a property that must be verified case-by-case. The paper should provide at least one concrete example (e.g., the single-head attention of Program 5) where this non-degeneracy is verified, or clarify that the attention example is illustrative rather than covered by the theorem. As stated, it is unclear whether any practical attention configuration satisfies the假设
minor comments (9)
  1. Section 1.1: 'A depth-M Multilayer Perceptron' — the hyphenation is inconsistent with later usage.
  2. Equation (1.2): The notation W^{(ell)}_{ij} ~ iid N(0, n_ell^{-1}) should specify that i ranges over {1,...,n_{ell+1}} and j over {1,...,n_ell} for clarity.
  3. Section 2.2, Example 2.2: The LayerNorm epsilon is mentioned but the specific value or range is not specified. Clarifying that epsilon > 0 is a structural constant would help.
  4. Figure 6 caption: 'Schematic program graph for a single-head attention layer' — the figure shows dependencies for a fixed query token i and key/value token j, but the caption does not mention this restriction.
  5. Section 4.2, MatMul case: The text states 'We first dispose of the case m_r in n_in' and then later repeats 'We first consider the case m_r in n_in.' This appears to be duplicated text from a draft revision.
  6. Section 5.2: The reference to 'Theorem 1 in [FG15]' for the SW_1 empirical measure convergence rate should specify the exact statement or page, as the result in that paper is for W_p, not SW_1 specifically.
  7. Table 1: The RNN slope (-0.579) is close to -0.5, and the text acknowledges slopes 'strictly below -1/2.' A brief discussion of why the RNN slope is notably shallower than the MLP slopes would be informative.
  8. References: Several arXiv preprints are cited with future dates (2026). Ensure these are the correct final versions or update to published references where available.
  9. Notation: The use of both 'hidd' subscript and 'hidd(r)' for truncated programs could be made more consistent; a brief remark defining n_{hidd(r)} explicitly as the hidden widths appearing in lines 1,...,r would help.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive report. The referee identifies a genuine gap in the proof of Proposition 3.11 (ii)=>(iii) concerning the distinction between marginal and joint non-degeneracy of G-vars, and raises an important question about whether the non-degeneracy assumption in Theorem 2.4 is ever satisfied for practical attention architectures. We address both points below. The first requires a revision to clarify the proof; the second requires either a verification for a concrete example or a recharacterization of the attention example's status.

read point-by-point responses

  1. Referee: Proposition 3.11, proof of (ii)=>(iii): The argument that Phi's zero set being open requires non-degeneracy of ALL previous G-vars jointly, not just marginally. The proof states: 'Since the (g_bar^(j)) are non-degenerate, the set {z : Phi(z) != 0} has null Lebesgue measure, and by continuity of Phi it is an open set, hence empty.' This step requires that the joint law of (g_bar^(j))_{j=1,...,k'} be non-degenerate (i.e., supported on all of R^{k'}). The non-degeneracy assumption in Lemma 3.9 is stated conditionally ('conditionally upon F_{l-1}'), but the proof of Proposition 3.11(ii)=>(iii) appears to use marginal non-degeneracy of the joint vector. In programs with weight sharing, the parents h^(i) can depend on previously generated G-vars from the same A-var W, so joint non-degeneracy of the G-vars is not automatic from conditional non-degeneracy at each step.

    Authors: The referee is correct that the proof of Proposition 3.11(ii)=>(iii) requires joint non-degeneracy of the vector (g_bar^(j))_{j=1,...,k'}, not merely marginal non-degeneracy of each component. The current wording 'Since the (g_bar^(j)) are non-degenerate' is ambiguous and should read 'Since the joint law of (g_bar^(j))_{j=1,...,k'} is non-degenerate.' We will revise the proof to explicitly state this requirement. Regarding the concern that joint non-degeneracy is not automatic from conditional non-degeneracy at each step: this is also correct in general. However, in the context of Lemma 3.9, the reduction proceeds sequentially and the non-degeneracy hypothesis of Lemma 3.9 is precisely that every MatMul line has non-degenerate conditional Gaussian output in the infinite-width execution. We will add a remark clarifying that this sequential conditional non-degeneracy, combined with the inductive structure of the reduction (where previously processed G-vars are already non-degenerate), ensures that the joint law of the relevant G-vars at each step is non-degenerate. If this implication does not hold in full generality for weight-sharing programs, we will state the joint non-degeneracy as an explicit hypothesis of Proposition 3.11 and verify it for the architectures considered in Section 2.1. revision: partial

  2. Referee: Proposition 3.11, proof of (ii)=>(iii): The lifting from infinite-width to finite-width linear identities is the load-bearing step for the non-degeneracy reduction (Lemma 3.9), which in turn is needed for differentiability of the maps involving pseudoinverses and matrix square roots in Section 4.2 ('Covariance stability'). The proof argues that since Phi(g_bar^(j)) = 0 a.s. (infinite-width) and the zero set is open, Phi vanishes identically, hence Phi(g^(j)) = 0 a.s. in finite-width too. This works when the coefficients alpha_i are deterministic constants, which they are (computed from infinite-width Gram data). However, the argument that the zero set {z : Phi(z) != 0} is open AND has null Lebesgue measure requires that the joint distribution of (g_bar^(j)) has full support. The proof should explicitly verify that the non-degeneracy hypothesis (conditional non-degeneracy at each MatMul)

    Authors: The referee correctly identifies that the lifting argument requires full support of the joint law of (g_bar^(j))_{j=1,...,k'}, which is equivalent to joint non-degeneracy (positive definiteness of the joint covariance matrix). The current proof does not explicitly verify this, and as noted in our response to the first comment, we will revise the proof to either (a) show that the sequential conditional non-degeneracy hypothesis of Lemma 3.9 implies joint non-degeneracy of the relevant G-vars at each step of the reduction, or (b) state joint non-degeneracy as an explicit hypothesis. We agree that the coefficients alpha_i are deterministic (computed from infinite-width Gram data), so the core lifting argument is sound once full support is established. We will add the missing verification and clarify the logical structure of the proof to make the dependency on joint non-degeneracy explicit. revision: partial

  3. Referee: Theorem 2.4 (Netsor K extension): The non-degeneracy assumption is stated as 'its G-vars are jointly non-degenerate Gaussians in the infinite-width execution.' Remark 2.5 honestly acknowledges that the reduction does not lift because 'a scalar kernel variable is deterministic in infinite-width but random at finite width, so a linear relation that holds identically in the infinite-width execution need not hold identically in the finite-width execution.' This means Theorem 2.4's coverage of attention architectures is conditional on a property that must be verified case-by-case. The paper should provide at least one concrete example (e.g., the single-head attention of Program 5) where this non-degeneracy is verified, or clarify that the attention example is illustrative rather than covered by the theorem. As stated, it is unclear whether any practical attention configuration satisfies the假设

    Authors: We agree that the paper should either verify the non-degeneracy assumption for at least one concrete attention configuration or explicitly state that the attention example is illustrative rather than covered by Theorem 2.4. Upon reflection, we believe the non-degeneracy assumption can be verified for the single-head attention of Program 5 under generic conditions on the input (specifically, when the input tokens are in general position so that the projected query/key vectors have a non-degenerate joint Gaussian law in the infinite-width execution). We will add a proposition or remark verifying this for Program 5 with appropriate hypotheses on the input. If the verification turns out to be more subtle than expected for the full attention layer (particularly due to the softmax nonlinearity creating dependencies), we will instead recharacterize the attention example as illustrative and state clearly that Theorem 2.4's coverage of attention is conditional on a case-by-case verification of non-degeneracy. Either way, the current ambiguity in the manuscript will be resolved. revision: partial

Circularity Check

0 steps flagged · score 1.0 of 10

No circularity found; the kernel LLN / Theorem 1.2 mutual dependence is a valid induction, explicitly and correctly handled

full rationale

The paper's main result (Theorem 1.2) is proved by line-by-line induction. The only potential circularity concern is that Corollary 1.3 (Kernel LLN) is both a consequence of Theorem 1.2 and an ingredient in its proof. The authors explicitly address this in Section 4: they prove the kernel estimate in conditional form ('if the conclusion of Theorem 1.2 holds for the truncated program up to line r-1, then the corresponding kernel LLN holds for all H-vars generated up to line r-1'), use it in the induction step from r-1 to r, and derive the unconditional Corollary 1.3 only after the induction completes. This is a standard, valid inductive technique with a trivial base case (r=0, deterministic inputs). The non-degeneracy reduction (Proposition 3.11) uses deterministic coefficients and a support argument for non-degenerate Gaussians, not circular reasoning. Self-citations to [BT22], [Tre23], and [MAT26] (all involving co-author Trevisan) provide context and comparison (e.g., recovering prior bounds as special cases), but are not load-bearing premises for the proof. The proof is self-contained, using standard tools (Gaussian conditioning, Greville's formula, Rosenthal's inequality). Score 1 reflects the presence of contextual self-citations that are not load-bearing.

Assumptions & free parameters 0 free parameters · 6 assumptions · 1 invented entities

The paper introduces no free parameters or fitted constants. The bound constant c depends only on structural constants (input dimensions, activations, p, program length). The Netsor K language is a formal construct with demonstrated coverage of attention layers. All background mathematical results (Rosenthal, Greville, Gaussian conditioning) are standard.

assumptions (6)
  • domain assumption All nonlinearities in NonLin lines are Lipschitz continuous (Theorem 1.2); for Netsor K, nonlinearities in NonLin+ satisfy local Lipschitz estimate with linear growth (condition 2.5).
    Stated as a hypothesis of Theorem 1.2 and Theorem 2.4. This excludes unbounded activations and is essential for the stability estimates in the induction step (§4.2, NonLin case).
  • domain assumption G-vars are jointly non-degenerate Gaussians in the infinite-width execution (Theorem 2.4 for Netsor K).
    Stated as a hypothesis of Theorem 2.4. Required because the non-degeneracy reduction (Lemma 3.9) does not extend to Netsor K (Remark 2.5). Must be verified case-by-case for each architecture.
  • domain assumption Weight matrices W are initialized with i.i.d. Gaussian entries of variance 1/m (finite-width execution rule, §1.1).
    Standard NNGP scaling. The proof uses Gaussian conditioning formulas (3.7)-(3.9) that rely on this specific distribution. Non-Gaussian weights would require different treatment.
  • domain assumption Input dimensions n_in are kept fixed as hidden widths diverge (§1.1, structural constants).
    The constant c depends on input dimensions but not on hidden widths. If input dimensions also grow, the analysis would need modification.
  • standard math Rosenthal's inequality (3.6) for moment bounds on sums of independent centered random variables.
    Used in the proof of Corollary 1.3 (§4.1) to control the sampling term W_p(K̄_n, K). Standard result from probability theory [Ros70].
  • standard math Greville's formula (3.3) for pseudoinverse of rank-one updates.
    Used in the proof of Lemma 3.5 to establish the structure of conditional means. Standard matrix algebra result [Gre60; UK97].
invented entities (1)
  • Netsor K language independent evidence
    purpose: Restricted extension of Netsor with scalar kernel variables (C-vars) and scalar-parametric nonlinearities, designed to cover attention layers while maintaining tractable quantitative analysis.
    The language is a formal construct, not a physical entity. Its utility is demonstrated through Example 2.2 (single-head attention). The restriction to Kernel + Scal + NonLin+ operations (vs. Yang's general Moment) is justified by the analytic requirements (Remark 2.1). Falsifiable in the sense that if the local Lipschitz condition (2.5) fails for a practical attention variant, the theorem does not apply.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quantitative Gaussian-Process limits of Tensor Programs." pith.science (2026). https://pith.science/paper/R67V5SKE

@misc{pith2026260706290,
  author       = {Pith},
  title        = {Pith review of: Quantitative Gaussian-Process limits of Tensor Programs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R67V5SKE}},
  note         = {Machine review of arXiv:2607.06290}
}
read the original abstract

We study the infinite-width Gaussian-process limit of random neural networks through the lens of tensor programs, and we provide a quantitative convergence theory in Wasserstein distance. Our main result gives explicit finite-width error bounds, of order inverse square-root of the widths between finite-network executions and their Gaussian-process limits. The framework is architecture-agnostic and covers feed-forward models together with weight-sharing schemes relevant for recurrent and transformer-type architectures.

Figures

Figures reproduced from arXiv: 2607.06290 by the authors.

Figure 1
Figure 1. Program graph of a shallow fully connected network evaluated on a single input. The input dimension is d ∈ nin, the hidden width is n ∈ nhidd, and the output dimension is dout ∈ nout. Program execution. For a given choice of inputs X , the execution of a program consists in the realization of its H-vars h (j) (including its G-vars g (j) ) as vector-valued random variables, with a given distribution on a common proba… view at source ↗
Figure 2
Figure 2. Program graph of a shallow fully connected network. The dimen￾sion n is a hidden width because it is produced by the first MatMul line and later contracted by the readout MatMul line. The input dimension d and output dimension dout are structural. Deep fully connected network [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Program graph of a deep feed-forward network with activated output. The internal widths are n1, . . . , nM−1. The terminal output di￾mension dout is structural, even if the final program line is obtained by a componentwise nonlinearity. opposed to the two previous examples. In particular, the results for fully connected ar￾chitectures are no longer applicable in this case. Under the Netsor formalism, Theorem 1.2 gua… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Time-unrolled recurrent architecture. The same input weight Wx is reused across time, and Wh maps the previous hidden state h1 into the next preactivation g3. The postactivation is defined as h2 = ψ(g2, g3). The recurrent dimension n is an internal hidden width. Two-in…
Figure 5
Figure 5. Figure 5: A shallow network evaluated on two inputs. The same A-vars are reused for the two inputs, represented by repeated edge labels W(0) and W(1). The width n is hidden and is common to both branches. NetsorK programs. A NetsorK program is a Netsor program, augmented with C￾…
Figure 6
Figure 6. Figure 6: Schematic program graph for a single-head attention layer in NetsorK. Solid arrows are MatMul operations labelled by A-vars, dotted arrows indicate kernel/scalar operations, and dashed arrows indicate scalar￾parametric nonlinearities. The width n is produced by the pro…
Figure 7
Figure 7. Figure 7: Sliced Wasserstein-1 (SW1) distances between the empirical out￾put laws (N = 5000 ensemble samples) and the law of the limiting Gaussian process activations across four architectures, plotted in log-log scale. Architecture SW1 slope Shallow MLP −0.721 Deep MLP L = 4 −0…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 43 canonical work pages

  1. [1]

    Probability Theory and Related Fields , volume=

    Quantitative CLTs in deep neural networks , author=. Probability Theory and Related Fields , volume=. 2025 , publisher=

    work page 2025

  2. [2]

    , year =

    Basteri, Andrea and Trevisan, Dario , title =. , year =

    work page

  3. [3]

    arXiv preprint arXiv:2601.21539 , year=

    Wide neural networks with general weights: convergence rate and explicit dependence on the hyper-parameters , author=. arXiv preprint arXiv:2601.21539 , year=

    work page arXiv

  4. [4]

    2026 , journal=

    Universality in Deep Neural Networks: An approach via the Lindeberg exchange principle , author=. 2026 , journal=

    work page 2026

  5. [5]

    Wide neural networks of any depth evolve as linear models under gradient descent * , volume=

    Lee, Jaehoon and Xiao, Lechao and Schoenholz, Samuel S and Bahri, Yasaman and Novak, Roman and Sohl-Dickstein, Jascha and Pennington, Jeffrey , year=. Wide neural networks of any depth evolve as linear models under gradient descent * , volume=. Journal of Statistical Mechanics: Theory and Experiment , publisher=. doi:10.1088/1742-5468/abc62b , number=

    work page doi:10.1088/1742-5468/abc62b

  6. [6]

    2023 , eprint=

    Wide Deep Neural Networks with Gaussian Weights are Very Close to Gaussian Processes , author=. 2023 , eprint=

    work page 2023

  7. [7]

    Optimal transport -- Old and new , volume =

    Villani, Cédric , year =. Optimal transport -- Old and new , volume =

    work page

  8. [8]

    2020 , eprint=

    Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation , author=. 2020 , eprint=

    work page 2020

Show all 43 references
  1. [9]

    Communications in Mathematical Physics , volume=

    Trained quantum neural networks are gaussian processes , author=. Communications in Mathematical Physics , volume=. 2025 , publisher=

    2025

  2. [10]

    Melchor Hernandez et al

    Quantitative Convergence of Trained Quantum Neural Networks to a Gaussian Process: A. Melchor Hernandez et al. , author=. Annales Henri Poincar. 2025 , organization=

    2025

  3. [11]

    2021 , eprint=

    Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes , author=. 2021 , eprint=

    2021

  4. [12]

    2020 , eprint=

    Tensor Programs II: Neural Tangent Kernel for Any Architecture , author=. 2020 , eprint=

    2020

  5. [13]

    2020 , eprint=

    Tensor Programs III: Neural Matrix Laws , author=. 2020 , eprint=

    2020

  6. [14]

    2020 , eprint=

    Neural Tangent Kernel: Convergence and Generalization in Neural Networks , author=. 2020 , eprint=

    2020

  7. [15]

    and Hu, Wei and Li, Zhiyuan and Salakhutdinov, Ruslan and Wang, Ruosong , title =

    Arora, Sanjeev and Du, Simon S. and Hu, Wei and Li, Zhiyuan and Salakhutdinov, Ruslan and Wang, Ruosong , title =. Proceedings of the 33rd International Conference on Neural Information Processing Systems , articleno =. 2019 , publisher =

    2019

  8. [16]

    Neal, R. M. , biburl =

  9. [17]

    and Hinton, Geoffrey , year =

    LeCun, Yann and Bengio, Y. and Hinton, Geoffrey , year =. Deep Learning , volume =. Nature , doi =

  10. [18]

    Mathematical Proceedings of the Cambridge Philosophical Society , author=

    A generalized inverse for matrices , volume=. Mathematical Proceedings of the Cambridge Philosophical Society , author=. 1955 , pages=. doi:10.1017/S0305004100030401 , number=

    1955 doi

  11. [19]

    Golub and Charles F

    Gene H. Golub and Charles F. Van Loan , title =. 2013 , publisher =

    2013

  12. [20]

    Zhang, Fuzhen , biburl =. The. doi:10.1007/b105056 , interhash =

    doi

  13. [21]

    , biburl =

    Muirhead, Robb J. , biburl =

  14. [22]

    and Nagar, Daya K

    Gupta, Arjun K. and Nagar, Daya K. , title =. 1999 , address =

    1999

  15. [23]

    T. N. E. Greville , journal =. Some Applications of the Pseudoinverse of a Matrix , urldate =

  16. [24]

    , title=

    Rosenthal, Haskell P. , title=. Israel Journal of Mathematics , year=. doi:10.1007/BF02771562 , url=

    doi

  17. [25]

    Udwadia, F. E. and Kalaba, R. E. , title=. Journal of Optimization Theory and Applications , year=. doi:10.1023/A:1022699317381 , url=

    doi

  18. [26]

    2026 , eprint=

    Infinite-Width Limit of a Single Attention Layer: Analysis via Tensor Programs , author=. 2026 , eprint=

    2026

  19. [27]

    and Hanin, B

    Favaro, S. and Hanin, B. and Marinucci, D. and Nourdin, I. and Peccati, G. , title=. Probability Theory and Related Fields , year=. doi:10.1007/s00440-025-01360-1 , url=

    doi

  20. [28]

    Probability Theory and Related Fields , year=

    Fournier, Nicolas and Guillin, Arnaud , title=. Probability Theory and Related Fields , year=. doi:10.1007/s00440-014-0583-7 , url=

    doi

  21. [29]

    Proceedings of the 38th International Conference on Machine Learning , pages =

    Tensor Programs IV: Feature Learning in Infinite-Width Neural Networks , author =. Proceedings of the 38th International Conference on Machine Learning , pages =. 2021 , editor =

    2021

  22. [30]

    Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics , pages =

    Infinitely deep neural networks as diffusion processes , author =. Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics , pages =. 2020 , editor =

    2020

  23. [31]

    Journal of Machine Learning Research , year =

    Soufiane Hayou , title =. Journal of Machine Learning Research , year =

  24. [32]

    2026 , eprint=

    Stochastic Scaling Limits and Synchronization by Noise in Deep Transformer Models , author=. 2026 , eprint=

    2026

  25. [33]

    Proceedings of Thirty Fourth Conference on Learning Theory , pages =

    Non-asymptotic approximations of neural networks by Gaussian processes , author =. Proceedings of Thirty Fourth Conference on Learning Theory , pages =. 2021 , editor =

    2021

  26. [34]

    Modern Stochastics: Theory and Applications , volume =

    Valentina Cammarota and Domenico Marinucci and Michele Salvi and Stefano Vigogna , title =. Modern Stochastics: Theory and Applications , volume =. 2024 , pages =. doi:10.15559/23-VMSTA238 , issn =

    2024 doi

  27. [35]

    2025 , eprint=

    Non-asymptotic approximations of Gaussian neural networks via second-order Poincar\'e inequalities , author=. 2025 , eprint=

    2025

  28. [36]

    2026 , eprint=

    Homogenized Transformers , author=. 2026 , eprint=

    2026

  29. [37]

    2026 , eprint=

    Quantitative convergence of trained single layer neural networks to Gaussian processes , author=. 2026 , eprint=

    2026

  30. [38]

    2023 , eprint=

    A Quantitative Functional Central Limit Theorem for Shallow Neural Networks , author=. 2023 , eprint=

    2023

  31. [39]

    2020 , eprint=

    Infinite attention: NNGP and NTK for deep attention networks , author=. 2020 , eprint=

    2020

  32. [40]

    2018 , eprint=

    Gaussian Process Behaviour in Wide Deep Neural Networks , author=. 2018 , eprint=

    2018

  33. [41]

    Morrison , title =

    Jack Sherman and Winifred J. Morrison , title =. The Annals of Mathematical Statistics , number =. 1950 , doi =

    1950

  34. [42]

    Schur Complements and Matrix Inequalities: Operator-Theoretic Approach

    Ando, Tsuyoshi. Schur Complements and Matrix Inequalities: Operator-Theoretic Approach. The Schur Complement and Its Applications. 2005. doi:10.1007/0-387-24273-2_6

    2005 doi

  35. [43]

    1973 , publisher=

    Linear Statistical Inference and its Applications , author=. 1973 , publisher=

    1973

Pith tools

Reviewed July 8, 2026 · model on record in the stance chip above.