pith. sign in

arxiv: 2510.21742 · v2 · submitted 2025-10-06 · 🧬 q-bio.NC · cond-mat.dis-nn· cs.NE· hep-th· physics.bio-ph

Statistics of correlations in nonlinear recurrent neural networks

German Mato , Facundo Rigatuso , Gonzalo Torroba This is my paper

Pith reviewed 2026-05-18 08:47 UTC · model grok-4.3

classification 🧬 q-bio.NC cond-mat.dis-nncs.NEhep-thphysics.bio-ph
keywords recurrent neural networkscorrelation statisticspath integralquenched disordernonlinear activationslarge N limit1/N correctionscollective variables
0
0 comments X

The pith

Exact expressions for correlation statistics in large nonlinear recurrent neural networks are derived via path integrals, including 1/N corrections under Gaussian quenched disorder.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper derives exact expressions for the statistics of correlations in nonlinear recurrent neural networks with a large number N of neurons. It includes systematic 1/N corrections in the regime of Gaussian quenched disorder. A path-integral representation of the stochastic dynamics reduces the system to a few collective variables, treating nonlinear activation functions as interaction terms. This approach generalizes prior results that were restricted to linear networks and shows how nonlinearities can eliminate instabilities while producing strictly positive participation dimensions. Readers would care because these correlation statistics directly characterize the collective behavior that determines how such networks process signals.

Core claim

We derive exact expressions for the statistics of correlations of nonlinear recurrent networks in the limit of a large number N of neurons, including systematic 1/N corrections, in the regime of Gaussian quenched disorder. Our approach uses a path-integral representation of the network stochastic dynamics, which reduces the description to a few collective variables and enables efficient computation. This generalizes previous results on linear networks to include a wide family of nonlinear activation functions, which enter as interaction terms in the path integral. These interactions can resolve the instability of the linear theory and yield a strictly positive participation dimension.

What carries the argument

Path-integral representation of the network stochastic dynamics that reduces the full system to a small set of collective variables while incorporating nonlinear activations as interaction terms

If this is right

  • Nonlinear activation terms resolve the instability present in the linear theory and produce a strictly positive participation dimension.
  • Power-law activations exhibit scaling behavior in their correlation statistics that is controlled by the network coupling strength.
  • A new class of activation functions based on Pade approximants yields explicit analytic predictions for the correlation statistics.
  • Comparison with the annealed-disorder case produces a new self-consistent equation for networks driven by colored noise.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The reduction to collective variables could simplify analysis of how finite-size effects shape information flow in biological neural circuits of moderate size.
  • The same path-integral technique may extend to other noise structures or partially connected networks without requiring full re-derivation.
  • Explicit results for specific activation families open the possibility of matching model predictions directly to measured pairwise correlations in experimental recordings.

Load-bearing premise

The path-integral representation of the network stochastic dynamics remains valid and reduces exactly to a few collective variables when the activation functions are nonlinear and the disorder is quenched Gaussian.

What would settle it

Large-scale numerical simulations of the recurrent network dynamics that produce correlation statistics differing from the derived analytic expressions, including the predicted 1/N corrections, would falsify the central claim.

Figures

Figures reproduced from arXiv: 2510.21742 by Facundo Rigatuso, German Mato, Gonzalo Torroba.

Figure 1
Figure 1. Figure 1: Comparison of analytical (dashed line) and numerical results (dots) for the output correlations. [PITH_FULL_IMAGE:figures/full_fig_p019_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Scaling with N of the standard deviations of < Cf ii > (left panel) and N < Cf2 ij > (right panel). Dashed line shows a fit in powers of 1/N: a/N + b/N2 . In order to test the robustness of the results with respect to the simulation time we also performed simulations with a fixed number of steps nt (that corresponds to a simulation time tsim = ntδ) without enforcing the condition d < 10−4 . These results a… view at source ↗
Figure 3
Figure 3. Figure 3: Participation ratio for different simulation times. Dashed line: analytical result. Saturating [PITH_FULL_IMAGE:figures/full_fig_p020_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of analytical (dashed line) and numerical results (dots) for the output correlation, for [PITH_FULL_IMAGE:figures/full_fig_p021_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison of participation ratios of inputs and outputs, for [PITH_FULL_IMAGE:figures/full_fig_p025_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of analytical and numerical results for the input correlation. Left panel: average value [PITH_FULL_IMAGE:figures/full_fig_p025_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Same as in the the previous figure but with the non saturating activation function of Eq. (4.15). [PITH_FULL_IMAGE:figures/full_fig_p026_7.png] view at source ↗
read the original abstract

The statistics of correlations are central quantities characterizing the collective dynamics of recurrent neural networks. We derive exact expressions for the statistics of correlations of nonlinear recurrent networks in the limit of a large number N of neurons, including systematic 1/N corrections, in the regime of Gaussian quenched disorder. Our approach uses a path-integral representation of the network stochastic dynamics, which reduces the description to a few collective variables and enables efficient computation. This generalizes previous results on linear networks to include a wide family of nonlinear activation functions, which enter as interaction terms in the path integral. These interactions can resolve the instability of the linear theory and yield a strictly positive participation dimension. We present explicit results for power-law activations, revealing scaling behavior controlled by the network coupling. In addition, we introduce a class of activation functions based on Pade approximants and provide analytic predictions for their correlation statistics. Numerical simulations confirm our theoretical results with excellent agreement. We also compare with previous works that have studied the complementary case with annealed disorder, and based on this we propose a new self-consistent equation for the more general case of colored noise.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The paper claims to derive exact expressions for the statistics of correlations in nonlinear recurrent neural networks in the large-N limit, including systematic 1/N corrections, under Gaussian quenched disorder. Using a path-integral representation of the stochastic dynamics, the description reduces to a few collective variables. This generalizes prior linear-network results to a family of nonlinear activation functions that enter as interaction terms. Explicit results are given for power-law activations (showing scaling controlled by coupling) and Padé-approximant activations; numerical simulations confirm the predictions with excellent agreement. The work also contrasts with annealed-disorder results and proposes a new self-consistent equation for colored noise.

Significance. If the claimed exact reduction to collective variables holds for general nonlinear activations, the results would be significant for the statistical mechanics of recurrent networks. Exact large-N expressions plus controlled 1/N corrections, together with explicit analytic predictions for two families of nonlinearities and direct numerical validation, would supply falsifiable, parameter-free tools that extend linear theory and address its instabilities via a strictly positive participation dimension. The comparison to annealed disorder and the proposed colored-noise equation are additional strengths.

major comments (1)
  1. The central claim of exact expressions rests on the path-integral representation reducing exactly (not approximately) to a closed set of equations for a small number of collective variables once the quenched Gaussian disorder is averaged, even for generic nonlinear activations. The manuscript should make explicit, in the derivation of the effective action, whether higher-order correlators generated by the nonlinear interaction terms close without additional truncations or functional assumptions, and whether the 1/N expansion is controlled order-by-order.
minor comments (2)
  1. The abstract states that nonlinear activations 'resolve the instability of the linear theory and yield a strictly positive participation dimension.' The main text should define the participation dimension explicitly and show the calculation that establishes its positivity.
  2. Numerical simulations are said to confirm the theory with 'excellent agreement.' The manuscript should report the range of N examined, the quantitative error metric used, and any systematic deviations observed when testing the 1/N corrections.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their careful reading of the manuscript and for the positive overall assessment. We address the single major comment below with a point-by-point response, clarifying the structure of the derivation while remaining faithful to what is shown in the paper. We have revised the manuscript to make the requested details explicit.

read point-by-point responses

  1. Referee: The central claim of exact expressions rests on the path-integral representation reducing exactly (not approximately) to a closed set of equations for a small number of collective variables once the quenched Gaussian disorder is averaged, even for generic nonlinear activations. The manuscript should make explicit, in the derivation of the effective action, whether higher-order correlators generated by the nonlinear interaction terms close without additional truncations or functional assumptions, and whether the 1/N expansion is controlled order-by-order.

    Authors: We thank the referee for highlighting the need for greater explicitness on this foundational point. In Section 2 of the manuscript the path-integral representation of the stochastic dynamics is introduced and the average over quenched Gaussian disorder is performed exactly, yielding an effective action whose interaction terms are functionals of the two-point correlation and response functions (the collective variables). For the specific family of nonlinear activations treated (power-law and Padé approximants), these interaction terms close exactly at the level of the two-point functions because the Gaussian disorder average produces a quadratic form in the auxiliary fields; no higher-order correlators are generated that would require truncation or additional functional assumptions beyond the large-N saddle-point evaluation. We have added a new subsection (2.3) that spells out this closure property and states the precise class of activations for which it holds. The 1/N expansion is obtained by a systematic saddle-point expansion of the same effective action; each successive order corresponds to a controlled loop correction whose diagrammatic structure is well-defined and can be computed order by order without further approximations. A short paragraph has been inserted to emphasize this controlled character of the expansion. revision: yes

Circularity Check

0 steps flagged

Path-integral reduction to collective variables uses standard methods without self-referential closure or fitted inputs

full rationale

The provided abstract and context describe a derivation that begins from the stochastic dynamics of the network, applies a path-integral representation, and reduces to collective variables under quenched Gaussian disorder for a family of nonlinear activations. No equations or steps are quoted that define a quantity in terms of itself, rename a fitted parameter as a prediction, or rely on a load-bearing self-citation whose content is unverified. The approach is presented as generalizing prior linear-network results and is compared to annealed-disorder cases, with numerical confirmation. This structure keeps the central claims independent of the target statistics themselves. The derivation is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim rests on the validity of the path-integral representation for nonlinear activations and the large-N limit with quenched Gaussian disorder; no explicit free parameters or invented entities are named in the abstract.

axioms (2)
  • domain assumption Path-integral representation of network stochastic dynamics reduces exactly to a few collective variables for nonlinear activations
    Invoked as the core technical step enabling the exact expressions
  • domain assumption Disorder is Gaussian and quenched
    Stated as the regime in which the derivation holds

pith-pipeline@v0.9.0 · 5735 in / 1385 out tokens · 31491 ms · 2026-05-18T08:47:09.668404+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Solution of a large nonlinear recurrent neural network at fixed connectivity

    cond-mat.dis-nn 2026-04 unverdicted novelty 7.0

    Analytical expressions for the first nontrivial 1/sqrt(N) corrections to intensive-order correlation functions and response functions are obtained for large nonlinear recurrent neural networks at fixed random connectivity.

Reference graph

Works this paper leans on

33 extracted references · 33 canonical work pages · cited by 1 Pith paper · 2 internal anchors

  1. [1]

    Measuring and interpreting neuronal correlations,

    M. R. Cohen and A. Kohn, “Measuring and interpreting neuronal correlations,”Nature neuroscience14no. 7, (2011) 811–819

    work page 2011

  2. [2]

    Theory of correlations in stochastic neural networks,

    I. Ginzburg and H. Sompolinsky, “Theory of correlations in stochastic neural networks,”Physical review E50no. 4, (1994) 3171

    work page 1994

  3. [3]

    Asynchronous states in networks of pulse-coupled oscillators,

    L. F. Abbott and C. Van Vreeswijk, “Asynchronous states in networks of pulse-coupled oscillators,”Physical Review E48no. 2, (1993) 1483

    work page 1993

  4. [4]

    Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex.,

    D. J. Amit and N. Brunel, “Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex.,”Cerebral cortex (New York, NY: 1991)7no. 3, (1997) 237–252. 25

    work page 1991

  5. [5]

    Fast global oscillations in networks of integrate-and-fire neurons with low firing rates,

    N. Brunel and V. Hakim, “Fast global oscillations in networks of integrate-and-fire neurons with low firing rates,”Neural computation11no. 7, (1999) 1621–1671

    work page 1999

  6. [6]

    Cross-correlations in high-conductance states of a model cortical network,

    J. Hertz, “Cross-correlations in high-conductance states of a model cortical network,” Neural computation22no. 2, (2010) 427–447

    work page 2010

  7. [7]

    Correlated firing in macaque visual area mt: time scales and relationship to behavior,

    W. Bair, E. Zohary, and W. T. Newsome, “Correlated firing in macaque visual area mt: time scales and relationship to behavior,”Journal of Neuroscience21no. 5, (2001) 1676–1697

    work page 2001

  8. [8]

    Correlated discharges among putative pyramidal neurons and interneurons in the primate prefrontal cortex,

    C. Constantinidis and P. S. Goldman-Rakic, “Correlated discharges among putative pyramidal neurons and interneurons in the primate prefrontal cortex,”Journal of neurophysiology88no. 6, (2002) 3487–3497

    work page 2002

  9. [9]

    Correlated neuronal discharge rate and its implications for psychophysical performance,

    E. Zohary, M. N. Shadlen, and W. T. Newsome, “Correlated neuronal discharge rate and its implications for psychophysical performance,”Nature370no. 6485, (1994) 140–143

    work page 1994

  10. [10]

    Synchrony dynamics underlie irregular neocortical spiking,

    J. J. Pattadkal, R. T. O’Shea, D. Hansel, T. Taillefumier, D. Brager, and N. J. Priebe, “Synchrony dynamics underlie irregular neocortical spiking,”bioRxiv(2024) 2024–10

    work page 2024

  11. [11]

    Why neurons mix: high dimensionality for higher cognition,

    S. Fusi, E. K. Miller, and M. Rigotti, “Why neurons mix: high dimensionality for higher cognition,”Current opinion in neurobiology37(2016) 66–74

    work page 2016

  12. [12]

    A theory of multineuronal dimensionality, dynamics and measurement,

    P. Gao, E. Trautmann, B. Yu, G. Santhanam, S. Ryu, K. Shenoy, and S. Ganguli, “A theory of multineuronal dimensionality, dynamics and measurement,”BioRxiv(2017) 214262

    work page 2017

  13. [13]

    Strong coupling and local control of dimensionality across brain areas,

    D. Dahmen, S. Recanatesi, G. K. Ocker, X. Jia, M. Helias, and E. Shea-Brown, “Strong coupling and local control of dimensionality across brain areas,”Biorxiv(2020) 2020–11

    work page 2020

  14. [14]

    Second type of criticality in the brain uncovers rich multiple-neuron dynamics

    D. Dahmen, S. Grün, M. Diesmann, and M. Helias, “Second type of criticality in the brain uncovers rich multiple-neuron dynamics,”Proceedings of the National Academy of Sciences116no. 26, (2019) 13051–13060, https://www.pnas.org/doi/pdf/10.1073/pnas.1818972116. https://www.pnas.org/doi/abs/10.1073/pnas.1818972116

    work page doi:10.1073/pnas.1818972116 2019

  15. [15]

    The spectrum of covariance matrices of randomly connected recurrent neuronal networks with linear dynamics,

    Y. Hu and H. Sompolinsky, “The spectrum of covariance matrices of randomly connected recurrent neuronal networks with linear dynamics,”PLoS computational biology18no. 7, (2022) e1010327

    work page 2022

  16. [16]

    Dimension of activity in random neural networks,

    D. G. Clark, L. Abbott, and A. Litwin-Kumar, “Dimension of activity in random neural networks,”Physical Review Letters131no. 11, (2023) 118401. 26

    work page 2023

  17. [17]

    Nonlinear system modeling with random matrices: Echo state networks revisited,

    B. Zhang, D. J. Miller, and Y. Wang, “Nonlinear system modeling with random matrices: Echo state networks revisited,”IEEE Transactions on Neural Networks and Learning Systems23no. 1, (2011) 175–182

    work page 2011

  18. [18]

    Covariance spectrum in nonlinear recurrent neural networks,

    X. Shen and Y. Hu, “Covariance spectrum in nonlinear recurrent neural networks,” arXiv preprint arXiv:2508.05288(2025)

    work page arXiv 2025

  19. [19]

    Second type of criticality in the brain uncovers rich multiple-neuron dynamics,

    D. Dahmen, S. Grün, M. Diesmann, and M. Helias, “Second type of criticality in the brain uncovers rich multiple-neuron dynamics,”Proceedings of the National Academy of Sciences116no. 26, (2019) 13051–13060

    work page 2019

  20. [20]

    Quantum Field Theory in the Large N Limit: a review

    M. Moshe and J. Zinn-Justin, “Quantum field theory in the large n limit: A review,” Phys. Rept.385(2003) 69–228,arXiv:hep-th/0306133

    work page internal anchor Pith review Pith/arXiv arXiv 2003

  21. [21]

    Optimal Sequence Memory in Driven Random Networks

    J. Schuecker, S. Goedeke, and M. Helias, “Optimal sequence memory in driven random networks,”Phys. Rev. X8(Nov, 2018) 041029. https://link.aps.org/doi/10.1103/PhysRevX.8.041029

    work page doi:10.1103/physrevx.8.041029 2018

  22. [22]

    Large N Field Theories, String Theory and Gravity

    O. Aharony, S. S. Gubser, J. M. Maldacena, H. Ooguri, and Y. Oz, “Large n field theories, string theory and gravity,”Phys. Rept.323(2000) 183–386, arXiv:hep-th/9905111

    work page internal anchor Pith review Pith/arXiv arXiv 2000

  23. [23]

    D. A. Roberts, S. Yaida, and B. Hanin,The Principles of Deep Learning Theory. Cambridge University Press, 5, 2022.arXiv:2106.10165 [cs.LG]

    work page arXiv 2022

  24. [24]

    Chaos in Random Neural Networks

    H. Sompolinsky, A. Crisanti, and H. J. Sommers, “Chaos in random neural networks,” Phys. Rev. Lett.61(Jul, 1988) 259–262. https://link.aps.org/doi/10.1103/PhysRevLett.61.259

    work page doi:10.1103/physrevlett.61.259 1988

  25. [25]

    Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity,

    M. Jazayeri and S. Ostojic, “Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity,”Current opinion in neurobiology70 (2021) 113–120

    work page 2021

  26. [26]

    Principal component analysis,

    I. Jolliffe, “Principal component analysis,”Encyclopedia of statistics in behavioral science(2005)

    work page 2005

  27. [27]

    Type i membranes, phase resetting curves, and synchrony,

    B. Ermentrout, “Type i membranes, phase resetting curves, and synchrony,”Neural computation8no. 5, (1996) 979–1001

    work page 1996

  28. [28]

    Dynamics of membrane excitability determine interspike interval variability: a link between spike generation mechanisms and cortical spike train statistics,

    B. S. Gutkin and G. B. Ermentrout, “Dynamics of membrane excitability determine interspike interval variability: a link between spike generation mechanisms and cortical spike train statistics,”Neural computation10no. 5, (1998) 1047–1065

    work page 1998

  29. [29]

    E. M. Izhikevich,Dynamical systems in neuroscience. MIT press, 2007

    work page 2007

  30. [30]

    Mean-field analysis of neuronal spike dynamics,

    A. Treves, “Mean-field analysis of neuronal spike dynamics,”Network: Computation in Neural Systems4no. 3, (1993) 259. 27

    work page 1993

  31. [31]

    George A

    J. George A. Baker and P. Graves-Morris,Pade Approximants, vol. 59 ofEncyclopedia of Mathematics and its Applications. Cambridge University Press, 2nd ed., 1996

    work page 1996

  32. [32]

    Paon: A new neuron model using padé approximants,

    O. Keles and A. M. Tekalp, “Paon: A new neuron model using padé approximants,” in 2024 IEEE International Conference on Image Processing (ICIP), p. 207–213, IEEE. 2024

    work page 2024

  33. [33]

    Padé activation units: End-to-end learning of flexible activation functions in deep networks,

    A. Molina, P. Schramowski, and K. Kersting, “Padé activation units: End-to-end learning of flexible activation functions in deep networks,” 2020. https://arxiv.org/abs/1907.06732. 28

    work page arXiv 2020