Pith. sign in

REVIEW 3 major objections 4 minor 5 cited by

Statistical physics can now compute the Bayes-optimal learning limit of a deep neural network with width proportional to its input, in the interpolation regime.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 07:39 UTC pith:WOOKY7V5

load-bearing objection A serious, mostly honest attack on a hard open problem, with an exactness claim that currently outruns the proof — especially for generic activations with a second Hermite component. the 3 major comments →

arxiv 2510.24616 v5 pith:WOOKY7V5 submitted 2025-10-28 stat.ML cond-mat.dis-nncond-mat.stat-mechcs.ITcs.LGmath.IT

Statistical physics of deep learning: Optimal learning of a multi-layer perceptron near interpolation

Jean Barbier , Francesco Camilli , Minh-Toan Nguyen , Mauro Pastore , Rudy Skerk This is my paper
classification stat.ML cond-mat.dis-nncond-mat.stat-mechcs.ITcs.LGmath.IT MSC 82B4468T07
keywords multi-layer perceptroninterpolation regimeBayes-optimal generalizationreplica methodHCIZ integralfeature learningspecialisation transitionstatistical physics of learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that the optimal (Bayes-optimal) learning of a multi-layer perceptron in the interpolation regime — where widths are proportional to the input dimension and the number of samples is proportional to the square of it — is governed by a replica-symmetric free entropy formula built from a small set of functional overlaps between teacher and student weights. If correct, the theory exactly predicts the minimum achievable generalization error, the amount of data needed to reach it, and the sequence of 'specialisation' transitions through which hidden units align with target features. The analysis covers shallow networks with generic activation functions, two-layer networks with odd activations, and deeper networks under a restricted activation class, and it explains why feature learning outperforms kernels and random features: non-linear terms beyond the linear and quadratic components can only be exploited once the student specialises. A rich phenomenology follows, including layer-wise specialisation propagating from inner to outer layers, neuron-wise inhomogeneity driven by readout amplitudes, and metastable states that trap practical training algorithms.

Core claim

Under the Gaussian hypothesis (11) — that the replicated post-activations of teacher and student converge to a jointly Gaussian law with covariance K* — combined with a measure simplification (31) that replaces the true conditional law of the quadratic sufficient statistics by a generalized Wishart prior with exponential tilt fixed by moment matching, the paper derives replica-symmetric formulas (Results 1, 3, 4) for the limiting free entropy of an MLP with L hidden layers in the proportional-width, quadratic-sample regime. The formulas express the free entropy as a variational problem over a few functional order parameters: overlaps labelled by readout amplitudes (and, for L=2, by effective

What carries the argument

The central objects are (i) the Gaussian ansatz on replicated post-activations, which reduces the energetic part to a low-dimensional covariance K*; (ii) the replacement of the conditional law of the quadratic composite S_2 = W^⊤ diag(v0) W by a generalized Wishart prior with an exponential tilt, whose Lagrange multiplier is fixed by matching the moment E[v² Q(v)²] + γ v̄² — this step is the crux that allows the theory to handle matrices that lack rotational invariance; and (iii) the use of HCIZ spherical integrals (and their rectangular counterpart for L=2) to evaluate the entropy of the matrix order parameters. The order parameters themselves — functional overlaps Q(v), Q1(v^(2)), Q2(v, v^

Load-bearing premise

The load-bearing premise is the Gaussian ansatz (11) — that the replicated post-activations of teacher and student converge to a jointly Gaussian vector with covariance K* — supplemented by the moment-matching replacement of the conditional law of S_2 (31); if either fails, the replica formulas do not follow.

What would settle it

Run a large-d Bayesian sampling (e.g., HMC) of a shallow MLP in the interpolation regime with a generic activation (µ2 ≠ 0) and measure the moment generating function of the student's post-activations on test inputs, as in the paper's FIG. 4. If the relative error between the empirical and theoretical Gaussian MGF does not vanish as d grows — e.g., stays above O(1/√d) — the Gaussian hypothesis (11) is violated and Result 1 cannot hold exactly.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • The Bayes-optimal generalization error for proportional-width MLPs in the interpolation regime is computable by maximizing a low-dimensional RS potential (Results 1–4), giving sharp limits that any algorithm trained on the same data cannot beat.
  • Feature learning beats kernels and random features because higher-order components of the teacher can only be exploited once the student's weights align (specialise) with those of the target; kernels never specialise, which explains the performance gap shown in FIG. 2.
  • Specialisation transitions are generically present and can be partial: sub-populations of neurons connected to larger readout amplitudes specialise first, and for L≥2 the transitions are layer-wise, propagating from inner to outer layers.
  • Deeper targets are harder: the overlap of the l-th layer decreases with layer index, and more data per layer is needed as L grows (FIG. 19).
  • Algorithms such as HMC, GAMP-RIE and ADAM get trapped in metastable states predicted by the theory; over-parameterisation (wider students) can recover part of the gap but the specialised equilibrium remains exponentially hard to reach in some cases.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the Gaussian ansatz extends to mismatched teacher-student settings (as the authors suggest), the same variational formulas could predict how much data a network needs to learn from a different function class, a step toward quantitative scaling laws.
  • The shallow-to-deep specialisation ordering implies a testable transfer-learning prediction: representations from early layers of a trained network should transfer to new tasks with smaller data budgets than those from deeper layers, because deeper layers need more data to specialise.
  • The formalism's success suggests that other extensive-rank matrix inference problems lacking rotational invariance, beyond the matrix-sensing problems the paper explicitly names, might be treated by the same replica-plus-HCIZ blend.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper develops a replica-symmetric statistical physics theory for Bayes-optimal learning of a teacher MLP by a matched student MLP in the proportional-width, interpolation scaling d,k_l,n → ∞ with k_l/d → γ_l and n/d^2 → α. The central results are formulas for the limiting free entropy and order parameters for shallow MLPs (Result 1), two-hidden-layer MLPs (Result 3), and arbitrary-depth MLPs under restrictive hypotheses (Result 4). These formulas determine the Bayes-optimal generalization error (Result 2) and predict layer-wise and neuron-wise specialization transitions. The theory is tested extensively against HMC, Metropolis, GAMP-RIE, and ADAM, and the paper identifies the Gaussian ansatz on replicated post-activations (Eq. (11)) and a moment-matched generalized-Wishart replacement (Eq. (31)) as the main unproved ingredients, with partial proofs for special cases.

Significance. If correct, the paper would be a significant step toward a quantitative theory of feature learning in fully trained, finite-width MLPs in the interpolation regime, going beyond kernel, random-feature, and mean-field limits. The identification of functional order parameters indexed by readout amplitudes and effective readouts gives a concrete, falsifiable picture of how specialization propagates across layers and neurons. Strengths include the absence of fitted constants, the breadth of numerical validation with multiple algorithm families, the direct test of the Gaussian hypothesis in Fig. 4, and the partial proof in App. B6 for μ2 = 0. The phenomenological predictions — e.g., shallow-to-deep propagation of specialization and the difficulty of reaching the specialized state — are interesting and well supported by the simulations. However, the exactness claim for generic shallow activations is not established, and the paper itself documents a concrete tension with a rigorous quadratic-activation result. The safest and most defensible core is the μ2 = 0 shallow case and the deep results under (H2)/(H3), where matrix-integral approximations are absent or less central.

major comments (3)
  1. [Result 1 and Remark 4] Result 1 is stated for arbitrary shallow activations with μ0 = 0, but for μ2 ≠ 0 its derivation relies on replacing the true conditional measure (30) by the moment-matched generalized-Wishart measure (31). This replacement is not derived, and a single moment condition does not determine the large-deviation rate function needed to select the equilibrium order parameters. The paper itself, in Remark 4, reports that for σ(x)=x², which satisfies the hypotheses of Result 1, numerical maximization of the RS potential selects Q(v)>0 for γ≲1 whereas the rigorous equations of [95] give Q(v)=0 for all (α,γ). The response that the free-entropy difference is ≤1% and the potential is flat does not resolve the issue: a small free-energy error can shift the location of the maximizing Q and change the predicted specialization transitions. Since σ=x² is inside the stated domain, Result 1 is not exact as
  2. [Eqs. (27)–(28) and (31)] The diagonal-concentration assumptions on Hadamard powers (27)–(28) and the measure simplification (31) are load-bearing for the entropic potential, not merely technical. The paper states these are assumptions and validates them only a posteriori through the same learning curves the theory is meant to predict. This circularity is particularly acute for the μ2 ≠ 0 shallow case, where the HCIZ integral is evaluated under the simplified measure. The manuscript should either provide a direct test of (27)–(28) at the level of the large-deviation functional (not just of the resulting generalization error), or clearly mark the μ2 ≠ 0 formula as conjectural. Without this, Results 1 and 2 for generic activations such as ReLU cannot be regarded as established.
  3. [Results 3 and 4 / App. C1] The deep-layer results also rely on unproved simplifications, although matrix integrals are absent for L≥3. For L=2, the entropic contribution is evaluated using a relaxation of the conditional law of W^(2:1) with an exponential tilt fixed by moment matching (App. C1). The rectangular spherical integral then gives the result. This is a further instance of the same moment-matching issue: a single overlap moment is matched, but the full measure is replaced by a Gaussian-product base measure. The numerical agreement is good, but the claims of exactness in Remarks 4 and the text for L≥2 should be softened unless a proof strategy or a rigorous check of the measure equivalence is supplied. The paper has a partial proof only for the shallow μ2 = 0 case (App. B6).
minor comments (4)
  1. [Notation around Eq. (10) and Eq. (16)] The symbol K* is used both for the asymptotic off-diagonal covariance in the Gaussian hypothesis and for the evaluated function K(R2*,Q*). This is a potential source of confusion; consider using K∞ for the object in (10).
  2. [Section II A, τ(Q) in Eq. (14)] The definition of τ(Q) via mmse^{-1}_S is terse; the reader must consult App. B1 to see that this is the Lagrange multiplier enforcing the moment condition. A one-sentence intuitive explanation would help.
  3. [Remark 4] The claim that 'the free-entropy difference never exceeds ≈1%' is not documented with a figure or table. Given that the paper makes a quantitative claim about the size of the error, this statement should be backed by a plot of the RS potential versus Q in the problematic γ≲1 regime.
  4. [App. B6] The partial proof for μ2 = 0 is a strength, but the precise hypotheses under which it applies (e.g., bounded activation, finite Hermite support) are not stated in the main text. Please state them explicitly.

Circularity Check

0 steps flagged

No significant circularity: Results 1-4 are explicitly conditional on stated hypotheses, and no fitted quantity is relabeled as a prediction.

full rationale

The paper's central derivations are conditional on two transparently labeled assumptions, and neither reduces to the target quantities. The Gaussian hypothesis (11) is introduced as "our main assumption" and later called "an ansatz we cannot prove"; all replica formulas are stated as consequences of this hypothesis, with the covariance K* obtained self-consistently from saddle-point equations rather than from data. The measure replacement in Eq. (31) is presented as "the simplification we consider," with the Lagrange multiplier tau fixed by the moment-matching condition (32) to the true conditional measure; the resulting saddle-point solution for R2 and Q is not forced to equal that matched moment, and no empirical constant is inserted into the RS potential (15). The numerical validation of the Gaussian ansatz in Sec. III (e.g., Fig. 4) is a self-consistency check rather than a parameter fit to the predicted learning curves; this is an epistemic limitation, not a circular reduction. Remark 4 explicitly concedes a discrepancy with the rigorous quadratic-activation result [95] for the case mu2 != 0 and declines to claim exactness there, while App. B2b flags the HCIZ steps as "possibly approximative." These admissions show that the authors do not treat the ansatz's conclusion as defining its inputs. There is no load-bearing self-citation chain, no imported uniqueness theorem, no fitted input relabeled as a prediction, and no renaming of a known result. Unproved assumptions are correctness/rigor risks, not circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 6 axioms · 1 invented entities

The ledger shows the central result is not a free fit: no constants are fitted to data, since the priors, activation Hermite coefficients, and scaling ratios are inputs. But the derivation rests on several ansatze internal to the paper, most importantly the Gaussian ansatz and the measure replacement. These are the price paid for tractability; numerical agreement is the only independent check.

axioms (6)
  • domain assumption Joint Gaussianity of replicated post-activations (Gaussian ansatz)
    Equation (11) assumes (λ_test(θ^a))_a converge to a jointly Gaussian vector with covariance K*. All replica free-entropy formulas rely on this; the paper states it cannot be proved but is validated a posteriori.
  • domain assumption Diagonal concentration of Hadamard powers of overlap matrices
    Equations (27)-(28) assume (Ω^{ab}_{ij})^ℓ ≈ δ_ij Q^{ab}(v)^ℓ for ℓ≥3, discarding off-diagonal terms. This is needed to reduce infinitely many overlap tensors to the few functional order parameters.
  • ad hoc to paper Measure replacement by generalized Wishart with moment matching
    Equation (31) replaces the true conditional law of S_2^a by independent generalized Wishart priors P_S with exponential tilt τ(Q), justified only by matching one moment. This is an ansatz, not derived, and it enables the HCIZ step.
  • domain assumption Replica symmetry (RS ansatz)
    The replica limit s→0 is evaluated under a replica-symmetric ansatz on the order parameters. The paper cites Bayes-optimality/Nishimori identities as justification, but no rigorous proof is given.
  • domain assumption For L=2, recursive Gaussianity of pre-activations and exact recovery of effective readouts
    Section IV B assumes pairs of pre-activations are jointly Gaussian for Mehler expansions, and that the O(k_1) effective readout components v^{(2)} are exactly reconstructible with Θ(d^2) data. Both are structural to the L=2 formulas.
  • domain assumption For L≥3, activations with µ0=µ1=µ2=0 and recursive diagonal concentration
    Hypothesis (H3) and Section IV C assume no linear or quadratic Hermite components, making only full specialization possible, and impose diagonal concentration of g^{(l)}(Ω^{(l)}) at every layer. This excludes standard activations for L≥3.
invented entities (1)
  • Generalized Wishart matrix S_2^a = W^{a⊤} diag(v0) W^a / √k with effective prior P_S no independent evidence
    purpose: Proxy for the true conditional law of quadratic feature tensors; permits HCIZ spherical integration in the replica calculation.
    The replacement (31) is an invented tractable measure; its only support is moment matching and subsequent numerical agreement, with no external falsifiable handle.

pith-pipeline@v1.3.0-alltime-deepseek · 71000 in / 12645 out tokens · 127413 ms · 2026-08-04T07:39:29.541622+00:00 · methodology

0 comments
read the original abstract

For four decades statistical physics has been providing a framework to analyse neural networks. A long-standing question remained on its capacity to tackle deep learning models capturing rich feature learning effects, thus going beyond the narrow networks or kernel methods analysed until now. We positively answer through the study of the supervised learning of a multi-layer perceptron. Importantly, (i) its width scales as the input dimension, making it more prone to feature learning than ultra wide networks, and more expressive than narrow ones or ones with fixed embedding layers; and (ii) we focus on the challenging interpolation regime where the number of trainable parameters and data are comparable, which forces the model to adapt to the task. We consider the matched teacher-student setting. Therefore, we provide the fundamental limits of learning random deep neural network targets and identify the sufficient statistics describing what is learnt by an optimally trained network as the data budget increases. A rich phenomenology emerges with various learning transitions. With enough data, optimal performance is attained through the model's "specialisation" towards the target, but it can be hard to reach for training algorithms which get attracted by sub-optimal solutions predicted by the theory. Specialisation occurs inhomogeneously across layers, propagating from shallow towards deep ones, but also across neurons in each layer. Furthermore, deeper targets are harder to learn. Despite its simplicity, the Bayes-optimal setting provides insights on how the depth, non-linearity and finite (proportional) width influence neural networks in the feature learning regime that are potentially relevant in much more general settings.

Figures

Figures reproduced from arXiv: 2510.24616 by Francesco Camilli, Jean Barbier, Mauro Pastore, Minh-Toan Nguyen, Rudy Skerk.

Figure 1
Figure 1. Figure 1: FIG. 1. Classification of models of fully connected feed-forward neural networks analysed in the theoretical literature (see [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: FIG. 2. Bayes-optimal mean-square generalisation error [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: FIG. 3. The teacher-student scenario for the case of two hid [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: FIG. 4. Experimental evidence for the Gaussian hypothe [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: FIG. 5. Theoretical prediction (solid curves) of the Bayes-optimal mean-square generalisation error (with the irreducible error [PITH_FULL_IMAGE:figures/full_fig_p015_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: FIG. 6. Theoretical prediction (solid curves) of the Bayes [PITH_FULL_IMAGE:figures/full_fig_p016_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: FIG. 7. Theoretical prediction (solid curves) for the equilib [PITH_FULL_IMAGE:figures/full_fig_p017_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: FIG. 8. Half Gibbs error of HMC from random initialisa [PITH_FULL_IMAGE:figures/full_fig_p018_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: FIG. 10. Generalisation error of different estimators, ini [PITH_FULL_IMAGE:figures/full_fig_p019_10.png] view at source ↗
Figure 12
Figure 12. Figure 12: FIG. 12 [PITH_FULL_IMAGE:figures/full_fig_p020_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: FIG. 13. Theoretical prediction (green solid curve) of the Bayes-optimal mean-square generalisation error for [PITH_FULL_IMAGE:figures/full_fig_p021_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: FIG. 14. Solid and dotted curves represent, respectively, the [PITH_FULL_IMAGE:figures/full_fig_p021_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: FIG. 15. Theoretical predictions (solid curves) for the [PITH_FULL_IMAGE:figures/full_fig_p022_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: FIG. 16. Heat-maps of all the theoretical equilibrium over [PITH_FULL_IMAGE:figures/full_fig_p023_16.png] view at source ↗
Figure 18
Figure 18. Figure 18: FIG. 18. Generalisation errors, computed empirically from [PITH_FULL_IMAGE:figures/full_fig_p024_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: FIG. 19. Theoretical predictions for deep NNs with the activation on every layer given by [PITH_FULL_IMAGE:figures/full_fig_p025_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: FIG. 20. Hamiltonian Monte Carlo dynamics of the overlaps [PITH_FULL_IMAGE:figures/full_fig_p027_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: FIG. 21. Trajectories of [PITH_FULL_IMAGE:figures/full_fig_p043_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: FIG. 22. Different theoretical curves and numerical results for ReLU( [PITH_FULL_IMAGE:figures/full_fig_p055_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: FIG. 23. Generalisation error for ReLU activation and Rademacher readout prior [PITH_FULL_IMAGE:figures/full_fig_p056_23.png] view at source ↗
Figure 24
Figure 24. Figure 24: FIG. 24. Theoretical prediction (solid curves) of the Bayes-optimal mean-square generalisation error for [PITH_FULL_IMAGE:figures/full_fig_p059_24.png] view at source ↗
Figure 25
Figure 25. Figure 25: FIG. 25. Semilog ( [PITH_FULL_IMAGE:figures/full_fig_p061_25.png] view at source ↗
Figure 26
Figure 26. Figure 26: FIG. 26. Same as in FIG [PITH_FULL_IMAGE:figures/full_fig_p062_26.png] view at source ↗
Figure 27
Figure 27. Figure 27: FIG. 27. Trajectories of the generalisation error of neural networks trained with ADAM at fixed batch size [PITH_FULL_IMAGE:figures/full_fig_p062_27.png] view at source ↗
Figure 28
Figure 28. Figure 28: FIG. 28. Trajectories of the overlap [PITH_FULL_IMAGE:figures/full_fig_p063_28.png] view at source ↗
Figure 29
Figure 29. Figure 29: FIG. 29. Semilog ( [PITH_FULL_IMAGE:figures/full_fig_p064_29.png] view at source ↗
Figure 30
Figure 30. Figure 30: FIG. 30 [PITH_FULL_IMAGE:figures/full_fig_p068_30.png] view at source ↗
Figure 31
Figure 31. Figure 31: FIG. 31 [PITH_FULL_IMAGE:figures/full_fig_p069_31.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Signal-to-Noise Ratio and Sample Size Govern Representational Alignment in Neural Networks

    stat.ML 2026-05 unverdicted novelty 6.0

    Representational alignment varies monotonically with SNR and non-monotonically with sample size (minimized near interpolation threshold) across linear and nonlinear networks, and is decoupled from generalization error.

  2. Asymmetric Scaling Laws from Sparse Features

    stat.ML 2026-05 unverdicted novelty 5.0

    A sparse-activation model predicts double-descent loss with distinct under- and over-parameterized scaling exponents set by sparsity, plus a compute-optimal frontier favoring dataset growth.

  3. Statistical Properties of Training & Generalization

    stat.ML 2026-06 unverdicted novelty 2.0

    Neural scaling laws in deep learning interact with physics constraints and inductive biases beyond classical statistics.

  4. There Will Be a Scientific Theory of Deep Learning

    stat.ML 2026-04 unverdicted novelty 2.0

    A mechanics of the learning process is emerging in deep learning theory, characterized by dynamics, coarse statistics, and falsifiable predictions across idealized settings, limits, laws, hyperparameters, and universa...

  5. Statistical Properties of Training & Generalization

    stat.ML 2026-06 unverdicted novelty 1.0

    Review of neural scaling laws and their relation to constraints and inductive biases when applying machine learning to physics problems.

Reference graph

Works this paper leans on

221 extracted references · 71 canonical work pages · cited by 4 Pith papers · 1 internal anchor

  1. [1]

    author author P. L. \ Bartlett , author A. Montanari ,\ and\ author A. Rakhlin ,\ title title Deep learning: a statistical viewpoint ,\ https://doi.org/10.1017/S0962492921000027 journal journal Acta Numerica \ volume 30 ,\ pages 87 ( year 2021 ) NoStop

    doi:10.1017/s0962492921000027 2021

  2. [2]

    LeCun , author Y

    author author Y. LeCun , author Y. Bengio ,\ and\ author G. Hinton ,\ title title Deep learning ,\ https://doi.org/10.1038/nature14539 journal journal Nature \ volume 521 ,\ pages 436 ( year 2015 ) NoStop

    doi:10.1038/nature14539 2015

  3. [3]

    author author D. J. \ Amit , author H. Gutfreund ,\ and\ author H. Sompolinsky ,\ title title Spin-glass models of neural networks ,\ https://doi.org/10.1103/PhysRevA.32.1007 journal journal Phys. Rev. A \ volume 32 ,\ pages 1007 ( year 1985 ) NoStop

    doi:10.1103/physreva.32.1007 1985

  4. [4]

    author author E. Gardner ,\ title title The space of interactions in neural network models ,\ https://doi.org/10.1088/0305-4470/21/1/030 journal journal Journal of Physics A: Mathematical and General \ volume 21 ,\ pages 257 ( year 1988 ) NoStop

    doi:10.1088/0305-4470/21/1/030 1988

  5. [5]

    Gardner \ and\ author B

    author author E. Gardner \ and\ author B. Derrida ,\ title title Three unfinished works on the optimal storage capacity of networks ,\ https://doi.org/10.1088/0305-4470/22/12/004 journal journal Journal of Physics A: Mathematical and General \ volume 22 ,\ pages 1983 ( year 1989 ) NoStop

    doi:10.1088/0305-4470/22/12/004 1983

  6. [6]

    author author H. S. \ Seung , author M. Opper ,\ and\ author H. Sompolinsky ,\ title title Query by committee ,\ in\ https://doi.org/10.1145/130385.130417 booktitle Proceedings of the Fifth Annual Workshop on Computational Learning Theory ,\ series and number COLT '92 \ ( publisher Association for Computing Machinery ,\ address New York, NY, USA ,\ year 1...

    arXiv 1992

  7. [7]

    Engel , author H

    author author A. Engel , author H. M. \ K\"ohler , author F. Tschepke , author H. Vollmayr ,\ and\ author A. Zippelius ,\ title title Storage capacity and learning algorithms for two-layer neural networks ,\ https://doi.org/10.1103/PhysRevA.45.7590 journal journal Phys. Rev. A \ volume 45 ,\ pages 7590 ( year 1992 ) NoStop

    work page doi:10.1103/physreva.45.7590 1992

  8. [8]

    Kang , author J.-H

    author author K. Kang , author J.-H. \ Oh , author C. Kwon ,\ and\ author Y. Park ,\ title title Generalization in a two-layer neural network ,\ https://doi.org/10.1103/PhysRevE.48.4805 journal journal Phys. Rev. E \ volume 48 ,\ pages 4805 ( year 1993 ) NoStop

    work page doi:10.1103/physreve.48.4805 1993

  9. [9]

    O'Kane \ and\ author O

    author author D. O'Kane \ and\ author O. Winther ,\ title title Learning to classify in large committee machines ,\ https://doi.org/10.1103/PhysRevE.50.3201 journal journal Phys. Rev. E \ volume 50 ,\ pages 3201 ( year 1994 ) NoStop

    work page doi:10.1103/physreve.50.3201 1994

  10. [10]

    Schwarze \ and\ author J

    author author H. Schwarze \ and\ author J. Hertz ,\ title title Generalization in fully connected committee machines ,\ https://doi.org/10.1209/0295-5075/21/7/012 journal journal Europhysics Letters \ volume 21 ,\ pages 785 ( year 1993 ) NoStop

    work page doi:10.1209/0295-5075/21/7/012 1993

  11. [11]

    author author R. Urbanczik ,\ title title Storage capacity of the fully-connected committee machine ,\ https://doi.org/10.1088/0305-4470/30/11/007 journal journal Journal of Physics A: Mathematical and General \ volume 30 ,\ pages L387 ( year 1997 ) NoStop

    work page doi:10.1088/0305-4470/30/11/007 1997

  12. [12]

    Winther , author B

    author author O. Winther , author B. Lautrup ,\ and\ author J.-B. \ Zhang ,\ title title Optimal learning in multilayer neural networks ,\ https://doi.org/10.1103/PhysRevE.55.836 journal journal Phys. Rev. E \ volume 55 ,\ pages 836 ( year 1997 ) NoStop

    work page doi:10.1103/physreve.55.836 1997

  13. [13]

    Schwarze \ and\ author J

    author author H. Schwarze \ and\ author J. Hertz ,\ title title Generalization in a large committee machine ,\ https://doi.org/10.1209/0295-5075/20/4/015 journal journal Europhysics Letters \ volume 20 ,\ pages 375 ( year 1992 ) NoStop

    work page doi:10.1209/0295-5075/20/4/015 1992

  14. [14]

    Schwarze , author M

    author author H. Schwarze , author M. Opper ,\ and\ author W. Kinzel ,\ title title Generalization in a two-layer neural network ,\ https://doi.org/10.1103/PhysRevA.46.R6185 journal journal Phys. Rev. A \ volume 46 ,\ pages R6185 ( year 1992 ) NoStop

    work page doi:10.1103/physreva.46.r6185 1992

  15. [15]

    Mato \ and\ author N

    author author G. Mato \ and\ author N. Parga ,\ title title Generalization properties of multilayered neural networks ,\ https://doi.org/10.1088/0305-4470/25/19/017 journal journal Journal of Physics A: Mathematical and General \ volume 25 ,\ pages 5047 ( year 1992 ) NoStop

    work page doi:10.1088/0305-4470/25/19/017 1992

  16. [16]

    Monasson \ and\ author R

    author author R. Monasson \ and\ author R. Zecchina ,\ title title Weight space structure and internal representations: A direct approach to learning and generalization in multilayer neural networks ,\ https://doi.org/10.1103/PhysRevLett.75.2432 journal journal Phys. Rev. Lett. \ volume 75 ,\ pages 2432 ( year 1995 ) NoStop

    work page doi:10.1103/physrevlett.75.2432 1995

  17. [17]

    author author B. Schottky ,\ title title Phase transitions in the generalization behaviour of multilayer neural networks ,\ https://doi.org/10.1088/0305-4470/28/16/010 journal journal Journal of Physics A: Mathematical and General \ volume 28 ,\ pages 4515 ( year 1995 ) NoStop

    work page doi:10.1088/0305-4470/28/16/010 1995

  18. [18]

    author author A. Engel ,\ title title Correlation of internal representations in feed-forward neural networks ,\ https://doi.org/10.1088/0305-4470/29/13/003 journal journal Journal of Physics A: Mathematical and General \ volume 29 ,\ pages L323 ( year 1996 ) NoStop

    work page doi:10.1088/0305-4470/29/13/003 1996

  19. [19]

    Malzahn , author A

    author author D. Malzahn , author A. Engel ,\ and\ author I. Kanter ,\ title title Storage capacity of correlated perceptrons ,\ https://doi.org/10.1103/PhysRevE.55.7369 journal journal Phys. Rev. E \ volume 55 ,\ pages 7369 ( year 1997 ) NoStop

    work page doi:10.1103/physreve.55.7369 1997

  20. [20]

    Malzahn \ and\ author A

    author author D. Malzahn \ and\ author A. Engel ,\ title title Correlations between hidden units in multilayer neural networks and replica symmetry breaking ,\ https://doi.org/10.1103/PhysRevE.60.2097 journal journal Phys. Rev. E \ volume 60 ,\ pages 2097 ( year 1999 ) NoStop

    work page doi:10.1103/physreve.60.2097 2097

  21. [21]

    Sompolinsky , author N

    author author H. Sompolinsky , author N. Tishby ,\ and\ author H. S. \ Seung ,\ title title Learning from examples in large neural networks ,\ https://doi.org/10.1103/PhysRevLett.65.1683 journal journal Phys. Rev. Lett. \ volume 65 ,\ pages 1683 ( year 1990 ) NoStop

    work page doi:10.1103/physrevlett.65.1683 1990

  22. [22]

    Gy\"orgyi ,\ title title First-order transition to perfect generalization in a neural network with binary synapses ,\ https://doi.org/10.1103/PhysRevA.41.7097 journal journal Phys

    author author G. Gy\"orgyi ,\ title title First-order transition to perfect generalization in a neural network with binary synapses ,\ https://doi.org/10.1103/PhysRevA.41.7097 journal journal Phys. Rev. A \ volume 41 ,\ pages 7097 ( year 1990 ) NoStop

    doi:10.1103/physreva.41.7097 1990

  23. [23]

    Meir \ and\ author J

    author author R. Meir \ and\ author J. F. \ Fontanari ,\ title title Learning from examples in weight-constrained neural networks ,\ https://doi.org/10.1088/0305-4470/25/5/021 journal journal Journal of Physics A: Mathematical and General \ volume 25 ,\ pages 1149 ( year 1992 ) NoStop

    work page doi:10.1088/0305-4470/25/5/021 1992

  24. [24]

    author author D. M. L. \ Barbato \ and\ author J. F. \ Fontanari ,\ title title The effects of lesions on the generalization ability of a perceptron ,\ https://doi.org/10.1088/0305-4470/26/8/013 journal journal Journal of Physics A: Mathematical and General \ volume 26 ,\ pages 1847 ( year 1993 ) NoStop

    work page doi:10.1088/0305-4470/26/8/013 1993

  25. [25]

    Engel \ and\ author L

    author author A. Engel \ and\ author L. Reimers ,\ title title Reliability of replica symmetry for the generalization problem of a toy multilayer neural network ,\ https://doi.org/10.1209/0295-5075/28/7/013 journal journal Europhysics Letters \ volume 28 ,\ pages 531 ( year 1994 ) NoStop

    work page doi:10.1209/0295-5075/28/7/013 1994

  26. [26]

    author author G. J. \ Bex , author R. Serneels ,\ and\ author C. Van den Broeck ,\ title title Storage capacity and generalization error for the reversed-wedge I sing perceptron ,\ https://doi.org/10.1103/PhysRevE.51.6309 journal journal Phys. Rev. E \ volume 51 ,\ pages 6309 ( year 1995 ) NoStop

    work page doi:10.1103/physreve.51.6309 1995

  27. [27]

    Barkai , author D

    author author E. Barkai , author D. Hansel ,\ and\ author H. Sompolinsky ,\ title title Broken symmetries in multilayered perceptrons ,\ https://doi.org/10.1103/PhysRevA.45.4146 journal journal Phys. Rev. A \ volume 45 ,\ pages 4146 ( year 1992 ) NoStop

    work page doi:10.1103/physreva.45.4146 1992

  28. [28]

    author author H. Schwarze ,\ title title Learning a rule in a multilayer neural network ,\ https://doi.org/10.1088/0305-4470/26/21/017 journal journal Journal of Physics A: Mathematical and General \ volume 26 ,\ pages 5781 ( year 1993 ) NoStop

    work page doi:10.1088/0305-4470/26/21/017 1993

  29. [29]

    Engel \ and\ author C

    author author A. Engel \ and\ author C. Van den Broeck ,\ https://doi.org/10.1017/CBO9781139164542 title Statistical mechanics of learning \ ( publisher Cambridge University Press ,\ year 2001 ) NoStop

    doi:10.1017/cbo9781139164542 2001

  30. [30]

    author author H. Cui ,\ title title High-dimensional learning of narrow neural networks ,\ https://doi.org/10.1088/1742-5468/adb1d6 journal journal Journal of Statistical Mechanics: Theory and Experiment \ volume 2025 ,\ pages 023402 ( year 2025 ) NoStop

    work page doi:10.1088/1742-5468/adb1d6 2025

  31. [31]

    Bruna \ and\ author D

    author author J. Bruna \ and\ author D. Hsu ,\ title title Survey on Algorithms for Multi-Index Models ,\ https://doi.org/10.1214/25-STS993 journal journal Statistical Science \ volume 40 ,\ pages 378 ( year 2025 ) NoStop

    work page doi:10.1214/25-sts993 2025

  32. [32]

    author author G. B. \ Arous , author R. Gheissari ,\ and\ author A. Jagannath ,\ title title Online stochastic gradient descent on non-convex losses from high-dimensional inference ,\ journal journal J. Mach. Learn. Res. \ volume 22 ,\ https://doi.org/10.5555/3546258.3546364 10.5555/3546258.3546364 ( year 2021 ) NoStop

    arXiv 2021

  33. [33]

    Damian , author L

    author author A. Damian , author L. Pillaud-Vivien , author J. Lee ,\ and\ author J. Bruna ,\ title title Computational-statistical gaps in G aussian single-index models (extended abstract) ,\ in\ https://proceedings.mlr.press/v247/damian24a.html booktitle Proceedings of Thirty Seventh Conference on Learning Theory ,\ series Proceedings of Machine Learnin...

    2024

  34. [34]

    Abbe , author E

    author author E. Abbe , author E. Boix-Adsera , author M. Brennan , author G. Bresler ,\ and\ author D. Nagaraj ,\ title title The staircase property: how hierarchical structure can guide deep learning ,\ in\ https://doi.org/10.5555/3540261.3542328 booktitle Proceedings of the 35th International Conference on Neural Information Processing Systems ,\ serie...

    arXiv 2021

  35. [35]

    Abbe , author E

    author author E. Abbe , author E. B. \ Adser \`a ,\ and\ author T. Misiakiewicz ,\ title title SGD learning on neural networks: leap complexity and saddle-to-saddle dynamics ,\ in\ https://proceedings.mlr.press/v195/abbe23a.html booktitle Proceedings of Thirty Sixth Conference on Learning Theory ,\ series Proceedings of Machine Learning Research , Vol.\ v...

    2023

  36. [36]

    Troiani , author Y

    author author E. Troiani , author Y. Dandi , author L. Defilippis , author L. Zdeborova , author B. Loureiro ,\ and\ author F. Krzakala ,\ title title Fundamental computational limits of weak learnability in high-dimensional multi-index models ,\ in\ https://proceedings.mlr.press/v258/troiani25a.html booktitle Proceedings of The 28th International Confere...

    2025

  37. [37]

    author author R. M. \ Neal ,\ title Priors for infinite networks ,\ in\ https://doi.org/10.1007/978-1-4612-0745-0_2 booktitle Bayesian Learning for Neural Networks \ ( publisher Springer New York ,\ address New York, NY ,\ year 1996 )\ pp.\ pages 29--53 NoStop

    doi:10.1007/978-1-4612-0745-0_2 1996

  38. [38]

    author author C. Williams ,\ title title Computing with infinite networks ,\ in\ https://proceedings.neurips.cc/paper/1996/file/ae5e3ce40e0404a45ecacaaf05e5f735-Paper.pdf booktitle Advances in Neural Information Processing Systems ,\ Vol. volume 9 ,\ editor edited by\ editor M. Mozer , editor M. Jordan ,\ and\ editor T. Petsche \ ( publisher MIT Press ,\ ...

    1996

  39. [39]

    Lee , author J

    author author J. Lee , author J. Sohl-dickstein , author J. Pennington , author R. Novak , author S. Schoenholz ,\ and\ author Y. Bahri ,\ title title Deep neural networks as G aussian processes ,\ in\ https://openreview.net/forum?id=B1EA-M-0Z booktitle International Conference on Learning Representations \ ( year 2018 ) NoStop

    2018

  40. [40]

    author author A. G. D. G. \ Matthews , author J. Hron , author M. Rowland , author R. E. \ Turner ,\ and\ author Z. Ghahramani ,\ title title Gaussian process behaviour in wide deep neural networks ,\ in\ https://openreview.net/forum?id=H1-nGgWC- booktitle International Conference on Learning Representations \ ( year 2018 ) NoStop

    2018

  41. [41]

    author author B. Hanin ,\ title title Random neural networks in the infinite width limit as Gaussian processes ,\ https://doi.org/10.1214/23-AAP1933 journal journal The Annals of Applied Probability \ volume 33 ,\ pages 4798 ( year 2023 ) NoStop

    doi:10.1214/23-aap1933 2023

  42. [42]

    Yoon \ and\ author J.-H

    author author H. Yoon \ and\ author J.-H. \ Oh ,\ title title Learning of higher-order perceptrons with tunable complexities ,\ https://doi.org/10.1088/0305-4470/31/38/012 journal journal Journal of Physics A: Mathematical and General \ volume 31 ,\ pages 7771 ( year 1998 ) NoStop

    work page doi:10.1088/0305-4470/31/38/012 1998

  43. [43]

    Dietrich , author M

    author author R. Dietrich , author M. Opper ,\ and\ author H. Sompolinsky ,\ title title Statistical mechanics of support vector networks ,\ https://doi.org/10.1103/PhysRevLett.82.2975 journal journal Phys. Rev. Lett. \ volume 82 ,\ pages 2975 ( year 1999 ) NoStop

    work page doi:10.1103/physrevlett.82.2975 1999

  44. [44]

    Gerace , author B

    author author F. Gerace , author B. Loureiro , author F. Krzakala , author M. Mézard ,\ and\ author L. Zdeborová ,\ title title Generalisation error in learning with random features and the hidden manifold model ,\ https://doi.org/10.1088/1742-5468/ac3ae6 journal journal Journal of Statistical Mechanics: Theory and Experiment \ volume 2021 ,\ pages 124013...

    doi:10.1088/1742-5468/ac3ae6 2021

  45. [45]

    Bordelon , author A

    author author B. Bordelon , author A. Canatar ,\ and\ author C. Pehlevan ,\ title title Spectrum dependent learning curves in kernel regression and wide neural networks ,\ in\ https://proceedings.mlr.press/v119/bordelon20a.html booktitle Proceedings of the 37th International Conference on Machine Learning ,\ series Proceedings of Machine Learning Research...

    2020

  46. [46]

    Canatar , author B

    author author A. Canatar , author B. Bordelon ,\ and\ author C. Pehlevan ,\ title title Spectral bias and task-model alignment explain generalization in kernel regression and infinitely wide neural networks ,\ https://doi.org/10.1038/s41467-021-23103-1 journal journal Nature Communications \ volume 12 ,\ pages 2914 ( year 2021 ) NoStop

    doi:10.1038/s41467-021-23103-1 2021

  47. [47]

    Xiao , author H

    author author L. Xiao , author H. Hu , author T. Misiakiewicz , author Y. M. \ Lu ,\ and\ author J. Pennington ,\ title title Precise learning curves and higher-order scaling limits for dot-product kernel regression ,\ https://doi.org/10.1088/1742-5468/ad01b7 journal journal Journal of Statistical Mechanics: Theory and Experiment \ volume 2023 ,\ pages 11...

    work page doi:10.1088/1742-5468/ad01b7 2023

  48. [48]

    Ghorbani , author S

    author author B. Ghorbani , author S. Mei , author T. Misiakiewicz ,\ and\ author A. Montanari ,\ title title Linearized two-layers neural networks in high dimension ,\ https://doi.org/10.1214/20-AOS1990 journal journal The Annals of Statistics \ volume 49 ,\ pages 1029 ( year 2021 ) NoStop

    doi:10.1214/20-aos1990 2021

  49. [49]

    Rahimi \ and\ author B

    author author A. Rahimi \ and\ author B. Recht ,\ title title Random features for large-scale kernel machines ,\ in\ https://proceedings.neurips.cc/paper_files/paper/2007/file/013a006f03dbc5392effeb8f18fda755-Paper.pdf booktitle Advances in Neural Information Processing Systems ,\ Vol. volume 20 ,\ editor edited by\ editor J. Platt , editor D. Koller , ed...

    2007

  50. [50]

    Jacot , author F

    author author A. Jacot , author F. Gabriel ,\ and\ author C. Hongler ,\ title title Neural tangent kernel: Convergence and generalization in neural networks ,\ in\ https://proceedings.neurips.cc/paper/2018/file/5a4be1fa34e62bb8a6ec6b91d2462f5a-Paper.pdf booktitle Advances in Neural Information Processing Systems ,\ Vol. volume 31 ,\ editor edited by\ edit...

    2018

  51. [51]

    Chizat , author E

    author author L. Chizat , author E. Oyallon ,\ and\ author F. Bach ,\ title title On lazy training in differentiable programming ,\ in\ https://proceedings.neurips.cc/paper/2019/file/ae614c557843b1df326cb29c57225459-Paper.pdf booktitle Advances in Neural Information Processing Systems ,\ Vol. volume 32 ,\ editor edited by\ editor H. Wallach , editor H. La...

    2019

  52. [52]

    Ghorbani , author S

    author author B. Ghorbani , author S. Mei , author T. Misiakiewicz ,\ and\ author A. Montanari ,\ title title When do neural networks outperform kernel methods? ,\ in\ https://proceedings.neurips.cc/paper_files/paper/2020/file/a9df2255ad642b923d95503b9a7958d8-Paper.pdf booktitle Advances in Neural Information Processing Systems ,\ Vol. volume 33 ,\ editor...

    2020

  53. [53]

    Refinetti , author S

    author author M. Refinetti , author S. Goldt , author F. Krzakala ,\ and\ author L. Zdeborova ,\ title title Classifying high-dimensional G aussian mixtures: Where kernel methods fail and neural networks succeed ,\ in\ https://proceedings.mlr.press/v139/refinetti21b.html booktitle Proceedings of the 38th International Conference on Machine Learning ,\ ser...

    2021

  54. [54]

    Dyer \ and\ author G

    author author E. Dyer \ and\ author G. Gur-Ari ,\ title title Asymptotics of wide networks from F eynman diagrams ,\ in\ https://openreview.net/forum?id=S1gFvANKDS booktitle International Conference on Learning Representations \ ( year 2020 ) NoStop

    2020

  55. [55]

    author author S. Yaida ,\ title title Non- G aussian processes and neural networks at finite widths ,\ in\ https://proceedings.mlr.press/v107/yaida20a.html booktitle Proceedings of The First Mathematical and Scientific Machine Learning Conference ,\ series Proceedings of Machine Learning Research , Vol.\ volume 107 ,\ editor edited by\ editor J. Lu \ and\...

    2020

  56. [56]

    Zavatone-Veth , author A

    author author J. Zavatone-Veth , author A. Canatar , author B. Ruben ,\ and\ author C. Pehlevan ,\ title title Asymptotics of representation learning in finite B ayesian neural networks ,\ in\ https://proceedings.neurips.cc/paper_files/paper/2021/file/cf9dc5e4e194fc21f397b4cac9cc3ae9-Paper.pdf booktitle Advances in Neural Information Processing Systems ,\...

    2021

  57. [57]

    author author K. T. \ Grosvenor \ and\ author R. Jefferson ,\ title title The edge of chaos: quantum field theory and deep neural networks ,\ https://doi.org/10.21468/SciPostPhys.12.3.081 journal journal SciPost Phys. \ volume 12 ,\ pages 081 ( year 2022 ) NoStop

    doi:10.21468/scipostphys.12.3.081 2022

  58. [58]

    Fischer , author J

    author author K. Fischer , author J. Lindner , author D. Dahmen , author Z. Ringel , author M. Kr\" a mer ,\ and\ author M. Helias ,\ title title Critical feature learning in deep neural networks ,\ in\ https://proceedings.mlr.press/v235/fischer24a.html booktitle Proceedings of the 41st International Conference on Machine Learning ,\ series Proceedings of...

    2024

  59. [59]

    Banta , author T

    author author I. Banta , author T. Cai , author N. Craig ,\ and\ author Z. Zhang ,\ title title Structures of neural network effective theories ,\ https://doi.org/10.1103/PhysRevD.109.105007 journal journal Phys. Rev. D \ volume 109 ,\ pages 105007 ( year 2024 ) NoStop

    doi:10.1103/physrevd.109.105007 2024

  60. [60]

    Guillen , author P

    author author M. Guillen , author P. Misof ,\ and\ author J. E. \ Gerken ,\ https://arxiv.org/abs/2508.11522 title Finite-width neural tangent kernels from F eynman diagrams ( year 2025 ),\ https://arxiv.org/abs/2508.11522 arXiv:2508.11522 [cs.LG] NoStop

    arXiv 2025

  61. [61]

    Bahri , author B

    author author Y. Bahri , author B. Hanin , author A. Brossollet , author V. Erba , author C. Keup , author R. Pacelli ,\ and\ author J. B. \ Simon ,\ title title Les H ouches lectures on deep learning at large and infinite width* ,\ https://doi.org/10.1088/1742-5468/ad2dd3 journal journal Journal of Statistical Mechanics: Theory and Experiment \ volume 20...

    work page doi:10.1088/1742-5468/ad2dd3 2024

  62. [62]

    Ringel , author N

    author author Z. Ringel , author N. Rubin , author E. Mor , author M. Helias ,\ and\ author I. Seroussi ,\ https://arxiv.org/abs/2502.18553 title Applications of statistical field theory in deep learning ( year 2025 ),\ https://arxiv.org/abs/2502.18553 arXiv:2502.18553 [stat.ML] NoStop

    Pith/arXiv arXiv 2025

  63. [63]

    Mei , author A

    author author S. Mei , author A. Montanari ,\ and\ author P.-M. \ Nguyen ,\ title title A mean field view of the landscape of two-layer neural networks ,\ https://doi.org/10.1073/pnas.1806579115 journal journal Proceedings of the National Academy of Sciences \ volume 115 ,\ pages E7665 ( year 2018 ) NoStop

    doi:10.1073/pnas.1806579115 2018

  64. [64]

    Mei , author T

    author author S. Mei , author T. Misiakiewicz ,\ and\ author A. Montanari ,\ title title Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limit ,\ in\ https://proceedings.mlr.press/v99/mei19a.html booktitle Proceedings of the Thirty-Second Conference on Learning Theory ,\ series Proceedings of Machine Learning Research , V...

    2019

  65. [65]

    Yang \ and\ author E

    author author G. Yang \ and\ author E. J. \ Hu ,\ title title Tensor programs IV : Feature learning in infinite-width neural networks ,\ in\ https://proceedings.mlr.press/v139/yang21c.html booktitle Proceedings of the 38th International Conference on Machine Learning ,\ series Proceedings of Machine Learning Research , Vol.\ volume 139 ,\ editor edited by...

    2021

  66. [66]

    Rotskoff \ and\ author E

    author author G. Rotskoff \ and\ author E. Vanden-Eijnden ,\ title title Trainability and accuracy of artificial neural networks: An interacting particle system approach ,\ https://doi.org/https://doi.org/10.1002/cpa.22074 journal journal Communications on Pure and Applied Mathematics \ volume 75 ,\ pages 1889 ( year 2022 ) NoStop

    doi:10.1002/cpa.22074 2022

  67. [67]

    Sirignano \ and\ author K

    author author J. Sirignano \ and\ author K. Spiliopoulos ,\ title title Mean field analysis of neural networks: A central limit theorem ,\ https://doi.org/https://doi.org/10.1016/j.spa.2019.06.003 journal journal Stochastic Processes and their Applications \ volume 130 ,\ pages 1820 ( year 2020 ) NoStop

    work page doi:10.1016/j.spa.2019.06.003 2019

  68. [68]

    Bordelon \ and\ author C

    author author B. Bordelon \ and\ author C. Pehlevan ,\ title title Self-consistent dynamical field theory of kernel evolution in wide neural networks ,\ in\ https://proceedings.neurips.cc/paper_files/paper/2022/file/d027a5c93d484a4312cc486d399c62c1-Paper-Conference.pdf booktitle Advances in Neural Information Processing Systems ,\ Vol. volume 35 ,\ editor...

    2022

  69. [69]

    \ Nguyen \ and\ author H

    author author P.-M. \ Nguyen \ and\ author H. T. \ Pham ,\ title title A rigorous framework for the mean field limit of multilayer neural networks ,\ https://doi.org/10.4171/msl/42 journal journal Mathematical Statistics and Learning \ volume 6 ,\ pages 201 ( year 2023 ) NoStop

    doi:10.4171/msl/42 2023

  70. [70]

    Bassetti , author M

    author author F. Bassetti , author M. Gherardi , author A. Ingrosso , author M. Pastore ,\ and\ author P. Rotondo ,\ title title Feature learning in finite-width B ayesian deep linear networks with multiple outputs and convolutional layers ,\ http://jmlr.org/papers/v26/24-1158.html journal journal Journal of Machine Learning Research \ volume 26 ,\ pages ...

    2025

  71. [71]

    Rubin , author Z

    author author N. Rubin , author Z. Ringel , author I. Seroussi ,\ and\ author M. Helias ,\ title title A unified approach to feature learning in B ayesian neural networks ,\ in\ https://openreview.net/forum?id=ZmOSJ2MV2R booktitle High-dimensional Learning Dynamics 2024: The Emergence of Structure and Reasoning \ ( year 2024 ) NoStop

    2024

  72. [72]

    van Meegen \ and\ author H

    author author A. van Meegen \ and\ author H. Sompolinsky ,\ title title Coding schemes in neural networks learning classification tasks ,\ https://doi.org/10.1038/s41467-025-58276-6 journal journal Nature Communications \ volume 16 ,\ pages 3354 ( year 2025 ) NoStop

    doi:10.1038/s41467-025-58276-6 2025

  73. [73]

    Lauditi , author B

    author author C. Lauditi , author B. Bordelon ,\ and\ author C. Pehlevan ,\ https://arxiv.org/abs/2502.07998 title Adaptive kernel predictors from feature-learning infinite limits of neural networks ( year 2025 ),\ https://arxiv.org/abs/2502.07998 arXiv:2502.07998 [cs.LG] NoStop

    Pith/arXiv arXiv 2025

  74. [74]

    author author A. X. \ Yang , author M. Robeyns , author E. Milsom , author B. Anson , author N. Schoots ,\ and\ author L. Aitchison ,\ title title A theory of representation learning gives a deep generalisation of kernel methods ,\ in\ https://proceedings.mlr.press/v202/yang23k.html booktitle Proceedings of the 40th International Conference on Machine Lea...

    2023

  75. [75]

    Rubin , author I

    author author N. Rubin , author I. Seroussi ,\ and\ author Z. Ringel ,\ title title Grokking as a first order phase transition in two layer networks ,\ in\ https://openreview.net/forum?id=3ROGsTX3IR booktitle The Twelfth International Conference on Learning Representations \ ( year 2024 ) NoStop

    2024

  76. [76]

    author author A. M. \ Saxe , author J. McClelland ,\ and\ author S. Ganguli ,\ title title Exact solutions to the nonlinear dynamics of learning in deep linear neural networks ,\ in\ https://doi.org/10.48550/arXiv.1312.6120 booktitle Proceedings of the International Conference on Learning Representations 2014 \ ( year 2014 ) NoStop

    Pith/arXiv arXiv doi:10.48550/arxiv.1312.6120 2014

  77. [77]

    Li \ and\ author H

    author author Q. Li \ and\ author H. Sompolinsky ,\ title title Statistical mechanics of deep linear neural networks: The backpropagating kernel renormalization ,\ https://doi.org/10.1103/PhysRevX.11.031059 journal journal Phys. Rev. X \ volume 11 ,\ pages 031059 ( year 2021 ) NoStop

    doi:10.1103/physrevx.11.031059 2021

  78. [78]

    author author L. Aitchison ,\ title title Why bigger is not always better: on finite and infinite neural networks ,\ in\ https://arxiv.org/abs/1910.08013 booktitle International Conference on Machine Learning \ ( organization PMLR ,\ year 2020 )\ pp.\ pages 156--164 NoStop

    Pith/arXiv arXiv 1910

  79. [79]

    Hanin \ and\ author A

    author author B. Hanin \ and\ author A. Zlokapa ,\ title title Bayesian interpolation with deep linear networks ,\ https://doi.org/10.1073/pnas.2301345120 journal journal Proceedings of the National Academy of Sciences \ volume 120 ,\ pages e2301345120 ( year 2023 ) NoStop

    work page doi:10.1073/pnas.2301345120 2023

  80. [80]

    author author J. A. \ Zavatone-Veth , author W. L. \ Tong ,\ and\ author C. Pehlevan ,\ title title Contrasting random and learned features in deep B ayesian linear regression ,\ https://doi.org/10.1103/PhysRevE.105.064118 journal journal Phys. Rev. E \ volume 105 ,\ pages 064118 ( year 2022 ) NoStop

    work page doi:10.1103/physreve.105.064118 2022

Showing first 80 references.