Pith. sign in

REVIEW 3 major objections 2 minor 42 references

Memory Uncertainty Relation and Harmonic Memory in Random Recurrent Networks

T0 review · 3 major / 2 minor · reviewed 2026-06-30 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An inequality sets a lower bound on short-term memory capacity of dynamical systems using input-induced state fluctuations.

desk verdict The paper introduces a lower bound on short-term memory via an uncertainty relation with state fluctuations in recurrent networks, realized by harmonic memory, plus a noise-induced memory effect under regularization, but the abstract leaves the core definitions unspecified. read the letter →

arxiv 2605.24628 v1 pith:PKIO7BGG submitted 2026-05-23 nlin.AO cond-mat.dis-nncs.NE

Taichi Haruna , Kohei Nakajima This is my paper · ORCID
classification nlin.AOcond-mat.dis-nncs.NE
keywords memoryuncertaintyrelationharmonicshort-termcapacityreservoircomputingrandomrecurrentnetworksnoise-induceddynamicalsystemsinputnoise
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper derives an inequality that places a minimum value on the short-term memory capability of any dynamical system, expressed in terms of the size of fluctuations in its internal states caused by input signals. This bound is framed as an uncertainty relation and is attained exactly by a particular choice of readout weights called harmonic memory. The authors analyze and simulate the inequality in several reservoir systems driven by input noise, identifying cases of exact equality, asymptotic equality, and strict inequality. They further show that state-space regularization can generate additional memory from noise, a phenomenon they term noise-induced memory, which causes the uncertainty relation to fail.

What carries the argument

The memory uncertainty relation inequality, which lower-bounds short-term memory capacity by a measure of input-induced state fluctuation size, with the bound achieved by harmonic memory readout weights.

What would settle it

A dynamical system in which measured short-term memory capacity falls below the numerical value of the lower bound computed from its observed state fluctuations under the same inputs.

Watch

Extended reading notes

Core claim

We present an inequality that bounds the short-term memory capability of dynamical systems from below. It can be interpreted as an uncertainty relation between a measure of short-term memory and that of the size of state fluctuations induced by input signals. The lower bound can be achieved by a readout weight and thus represents a suboptimal memory called harmonic memory. We examine analytically and numerically the inequality in a number of reservoir systems subject to input noise. We illustrate cases in which equality is achieved exactly, equality holds asymptotically, and the inequality is strict. We also study the effect of a state-space regularization to elucidate the inequality in term

Load-bearing premise

The inequality and its interpretation depend on the particular quantitative definitions chosen for short-term memory capacity and the size of state fluctuations.

Editorial extensions

If this is right

  • The bound applies to the memory performance of any dynamical system used for temporal tasks.
  • Harmonic memory supplies an explicit readout construction that saturates the bound in some systems.
  • Input noise combined with regularization can produce memory capacity above the bound.
  • The uncertainty relation ceases to apply once regularization activates noise-induced memory.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Network designers could deliberately adjust fluctuation levels to enforce or relax the memory lower bound.
  • The same fluctuation-memory trade-off might appear in biological or physical systems outside reservoir computing.
  • Identifying the precise conditions that trigger noise-induced memory could allow controlled memory enhancement.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 2 minor

Summary. The manuscript presents an inequality that lower-bounds a quantitative measure of short-term memory capacity in dynamical systems by a measure of the size of state fluctuations induced by input signals. This is interpreted as an uncertainty relation, with the bound attained exactly by a specific linear readout weight vector termed 'harmonic memory.' The authors derive and examine the relation analytically and numerically in random recurrent reservoir networks driven by input noise, documenting cases of exact equality, asymptotic equality, and strict inequality. They further analyze the effects of state-space regularization, identifying a 'noise-induced memory' phenomenon under certain regularization strengths, and note that the uncertainty relation does not hold in general for the regularized memory measure.

Significance. If the inequality is non-tautological and holds under the stated definitions for a reasonably broad class of systems, the result would supply a concrete lower bound on memory performance in recurrent dynamical systems and a mechanistic explanation for suboptimal but analytically tractable memory (harmonic memory). The explicit treatment of noise-induced memory under regularization and the classification of equality cases add concrete value for reservoir-computing applications. The work is strongest where it supplies reproducible numerical protocols and explicit constructions; its broader impact hinges on whether the definitions of memory capacity and fluctuation size are standard or ad-hoc and on the scope of the derivation beyond the reservoir networks studied.

major comments (3)
  1. [Introduction and §3] The abstract and introduction claim the inequality applies to general dynamical systems, yet all analytic derivations and numerical tests are performed exclusively on random recurrent networks with linear readouts. The manuscript should clarify whether the derivation uses only properties common to all dynamical systems or relies on the specific structure of reservoir state updates (e.g., the echo-state property or the form of the recurrent weight matrix).
  2. [§2 (Definitions) and §3 (Derivation)] The central inequality is described as bounding memory capacity from below by a fluctuation measure, with equality achieved by harmonic memory. Without the explicit definitions of both quantities (e.g., whether memory capacity is the standard sum-of-squared correlations or a different functional, and whether fluctuation size is an L2 norm of the response operator or input-induced variance), it is impossible to determine whether the bound is a non-trivial consequence of Cauchy-Schwarz or an identity by construction. The manuscript must supply these definitions and the derivation steps in a dedicated section.
  3. [§5 (Regularization and noise-induced memory)] The claim that 'the memory uncertainty relation does not hold in general for the regularized memory and harmonic memory' is presented as a consequence of the noise-induced memory mechanism. This statement requires a precise counter-example or analytic condition under which the regularized quantities violate the inequality; the current discussion appears to rest on numerical observation alone.
minor comments (2)
  1. [Throughout] Notation for the memory capacity measure and the fluctuation measure should be introduced once with explicit symbols and then used consistently; several passages refer to 'the memory' without distinguishing the unregularized, regularized, and harmonic variants.
  2. [Figure captions] Figure captions should state the precise parameter values (network size, spectral radius, noise variance, regularization strength) used in each panel so that the equality/asymptotic/strict cases can be reproduced from the caption alone.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive comments, which help clarify the scope and presentation of our results. We address each major comment below.

read point-by-point responses

  1. Referee: [Introduction and §3] The abstract and introduction claim the inequality applies to general dynamical systems, yet all analytic derivations and numerical tests are performed exclusively on random recurrent networks with linear readouts. The manuscript should clarify whether the derivation uses only properties common to all dynamical systems or relies on the specific structure of reservoir state updates (e.g., the echo-state property or the form of the recurrent weight matrix).

    Authors: The inequality is derived from the Cauchy-Schwarz inequality applied to the inner product between the readout vector and the state response operator, which is a general property once a well-defined state response to inputs is assumed. However, the explicit analytic expressions and all numerical tests rely on the linear state update and echo-state property of random recurrent networks. We will revise the introduction and §3 to state the assumptions explicitly and limit the general claim to systems admitting a linear readout and bounded state response. revision: yes

  2. Referee: [§2 (Definitions) and §3 (Derivation)] The central inequality is described as bounding memory capacity from below by a fluctuation measure, with equality achieved by harmonic memory. Without the explicit definitions of both quantities (e.g., whether memory capacity is the standard sum-of-squared correlations or a different functional, and whether fluctuation size is an L2 norm of the response operator or input-induced variance), it is impossible to determine whether the bound is a non-trivial consequence of Cauchy-Schwarz or an identity by construction. The manuscript must supply these definitions and the derivation steps in a dedicated section.

    Authors: We agree that the definitions and derivation must be stated explicitly. Memory capacity is the sum of squared correlations (standard definition), and the fluctuation measure is the squared L2 norm of the input-induced state deviation. The bound follows directly from Cauchy-Schwarz on these quantities, with equality when the readout equals the normalized fluctuation vector. We will insert a new dedicated subsection in §2 with the full definitions followed by the step-by-step derivation in §3. revision: yes

  3. Referee: [§5 (Regularization and noise-induced memory)] The claim that 'the memory uncertainty relation does not hold in general for the regularized memory and harmonic memory' is presented as a consequence of the noise-induced memory mechanism. This statement requires a precise counter-example or analytic condition under which the regularized quantities violate the inequality; the current discussion appears to rest on numerical observation alone.

    Authors: The current manuscript presents the violation through numerical observation under varying regularization strengths. We will add an analytic derivation showing that the inequality fails when the regularized covariance matrix rotates the effective memory vector away from the harmonic direction by an angle whose cosine falls below the normalized fluctuation term; a concrete counter-example with explicit regularization parameter and noise variance will be included in the revised §5. revision: partial

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity detected; derivation presented as independent inequality

full rationale

The abstract and provided context introduce an inequality bounding short-term memory capability as a derived result for general dynamical systems, interpreted as an uncertainty relation, with equality achieved via specific readout weights (harmonic memory). No equations, parameter fittings, self-citations, or ansatzes are visible that reduce the claimed bound to its own inputs by construction. The reader's assessment confirms absence of such reductions in the abstract, and the central claim is framed as holding beyond the examined reservoir networks without load-bearing self-referential steps. This is the normal non-finding for papers whose core result is not shown to collapse into tautology or fitted renaming.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

No free parameters, axioms, or invented entities can be identified from the abstract alone; the inequality itself is the central object but its mathematical form and any background assumptions are not stated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Memory Uncertainty Relation and Harmonic Memory in Random Recurrent Networks." pith.science (2026). https://pith.science/paper/PKIO7BGG

@misc{pith2026260524628,
  author       = {Pith},
  title        = {Pith review of: Memory Uncertainty Relation and Harmonic Memory in Random Recurrent Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PKIO7BGG}},
  note         = {Machine review of arXiv:2605.24628}
}
read the original abstract

We present an inequality that bounds the short-term memory capability of dynamical systems from below. It can be interpreted as an uncertainty relation between a measure of short-term memory and that of the size of state fluctuations induced by input signals. The lower bound can be achieved by a readout weight and thus represents a suboptimal memory called harmonic memory. We examine analytically and numerically the inequality in a number of reservoir systems subject to input noise. We illustrate cases in which equality is achieved exactly, equality holds asymptotically, and the inequality is strict. We also study the effect of a state-space regularization to elucidate the inequality in terms of the fluctuation structure of the state-space. We find that a certain strength of input noise induces extra memory under the regularization, and we refer to this phenomenon as noise-induced memory. We observe that the memory uncertainty relation does not hold in general for the regularized memory and harmonic memory. This fact is explained in terms of the mechanism of noise-induced memory.

Figures

Figures reproduced from arXiv: 2605.24628 by the authors.

Figure 1
Figure 1. FIG. 1. (a) [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. FIG. 2. (a) [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. FIG. 3. (a) [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: FIG. 4. (a) [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: (a) shows MCϵ (= HMCϵ) for the DSRN with N = 800, α = 0.9 and ϵ = 0.1. The result for α = 0.8 is shown in [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: FIG. 6 [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: FIG. 7 [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: FIG. 8. Dependence of the eigenvalues of [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: FIG. 9 [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: FIG. 10 [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 42 canonical work pages

  1. [1]

    holds, then we have the strict inequalitym(k)> h(k). This argument is formally similar to the justification of consistency between the second law of thermodynamics and Maxwell’s demon in general information exchange processes [37, 38], in that the inequality that holds in the whole system does not necessarily restrict to subsystems. This could indicate a ...

    work page

  2. [2]

    Maass, T

    W. Maass, T. Natschl¨ ager, and H. Markram, Neural Comput.14, 2531 (2002)

    work page 2002

  3. [3]

    Jaeger and H

    H. Jaeger and H. Haas, Science304, 78 (2004)

    work page 2004

  4. [4]

    Lukoˇ seviˇ cius and H

    M. Lukoˇ seviˇ cius and H. Jaeger, Comput. Sci. Rev.3, 127 (2009)

    work page 2009

  5. [5]

    Tanaka, T

    G. Tanaka, T. Yamane, J. B. H´ eroux, R. Nakane, N. Kanazawa, S. Takeda, H. Numata, D. Nakano, and A. Hirose, Neural Netw.115, 100 (2019)

    work page 2019

  6. [6]

    Nakajima, Jpn

    K. Nakajima, Jpn. J. Appl. Phys.59, 060501 (2020)

    work page 2020

  7. [7]

    Nakajima and I

    K. Nakajima and I. Fischer, eds.,Reservoir Comput- ing: Theory, Physical Implementations, and Applications (Springer Singapore, 2021)

    work page 2021

  8. [8]

    Jaeger, Short term memory in echo state networks (2002), GMD-Report 152, GMD-German National Re- search Institute for Computer Science

    H. Jaeger, Short term memory in echo state networks (2002), GMD-Report 152, GMD-German National Re- search Institute for Computer Science

    work page 2002

Show all 42 references
  1. [9]

    Dambre, D

    J. Dambre, D. Verstraeten, B. Schrauwen, and S. Massar, Sci. Rep.2, 514 (2012). 14

    2012

  2. [10]

    Ganguli, D

    S. Ganguli, D. Huh, and H. Sompolinsky, Proc. Natl. Acad. Sci. USA105, 18970 (2008)

    2008

  3. [11]

    O. L. White, D. D. Lee, and H. Sompolinsky, Phys. Rev. Lett.92, 148102 (2004)

    2004

  4. [12]

    Hermans and B

    M. Hermans and B. Schrauwen, Neural Netw.23, 341 (2010)

    2010

  5. [13]

    Tiˇ no, Neurocomputing298, 4 (2018)

    P. Tiˇ no, Neurocomputing298, 4 (2018)

    2018

  6. [14]

    Toyoizumi and L

    T. Toyoizumi and L. F. Abbott, Phys. Rev. E84, 051908 (2011)

    2011

  7. [15]

    Schuecker, S

    J. Schuecker, S. Goedeke, and M. Helias, Phys. Rev. X 8, 041029 (2018)

    2018

  8. [16]

    Haruna and K

    T. Haruna and K. Nakajima, Phys. Rev. E100, 062312 (2019)

    2019

  9. [17]

    Bertschinger and T

    N. Bertschinger and T. Natschl¨ ager, Neural Comput.16, 1413 (2004)

    2004

  10. [18]

    Boedecker, O

    J. Boedecker, O. Obst, J. T. Lizier, N. M. Mayer, and M. Asada, Theory Biosci.131, 205 (2012)

    2012

  11. [19]

    Farkaˇ s, R

    I. Farkaˇ s, R. Bos´ ak, and P. Gergeˇl, Neural Netw.83, 109 (2016)

    2016

  12. [20]

    Marzen, Phys

    S. Marzen, Phys. Rev. E96, 032308 (2017)

    2017

  13. [21]

    Gonon, L

    L. Gonon, L. Grigoryeva, and J.-P. Ortega, Physica D 414, 132721 (2020)

    2020

  14. [22]

    J. C. Guan, T. Kubota, Y. Kuniyoshi, and K. Nakajima, Phys. Rev. Research7, 023049 (2025)

    2025

  15. [23]

    Inubushi and K

    M. Inubushi and K. Yoshimura, Sci. Rep.7, 10199 (2017)

    2017

  16. [24]

    Kubota, H

    T. Kubota, H. Takahashi, and K. Nakajima, Phys. Rev. Res.3, 043135 (2021)

    2021

  17. [25]

    Schulte to Brinke, M

    T. Schulte to Brinke, M. Dick, R. Duarte, and A. Morri- son, Sci. Rep.13, 10517 (2023)

    2023

  18. [26]

    P. Tino, J. Mach. Learn. Res.21, 1 (2020)

    2020

  19. [27]

    Verzelli, C

    P. Verzelli, C. Alippi, L. Livi, and P. Tiˇ no, IEEE Trans. Neural Netw. Learn. Syst.33, 4598 (2021)

    2021

  20. [28]

    echo state

    H. Jaeger, The “echo state” approach to analysing and training recurrent neural networks (2001), GMD-Report 148, GMD-German National Research Institute for Com- puter Science

    2001

  21. [29]

    Massar and S

    M. Massar and S. Massar, Phys. Rev. E87, 042809 (2013)

    2013

  22. [30]

    A. C. Barato and U. Seifert, Phys. Rev. Lett.114, 158101 (2015)

    2015

  23. [31]

    T. R. Gingrich, J. M. Horowitz, N. Perunov, and J. L. England, Phys. Rev. Lett.116, 120601 (2016)

    2016

  24. [32]

    T. M. Cover and J. A. Thomas,Elements of Informa- tion Theory, 2nd ed.(John Wiley & Sons, Hoboken, NJ, 2006)

    2006

  25. [33]

    G. H. Hardy, J. E. Littlewood, and G. P´ olya,Inequalities, Second Edition(Cambridge University Press, 1988)

    1988

  26. [34]

    Løkse, F

    S. Løkse, F. Bianchi, and R. Jenssen, Cogn. Comput.9, 364 (2017)

    2017

  27. [35]

    Vershynin,High-Dimensional Probability(Cambridge University Press, 2018)

    R. Vershynin,High-Dimensional Probability(Cambridge University Press, 2018)

    2018

  28. [36]

    Krauss, K

    P. Krauss, K. Prebeck, A. Schilling, and C. Metzner, Front. Comput. Neurosci.13, 64 (2019)

    2019

  29. [37]

    Metzner, A

    C. Metzner, A. Schilling, A. Maier, and P. Krauss, Front. Complex Syst.2, 1479417 (2024)

    2024

  30. [38]

    Sagawa and M

    T. Sagawa and M. Ueda, Phys. Rev. Lett.109, 180602 (2012)

    2012

  31. [39]

    Sagawa and M

    T. Sagawa and M. Ueda, New. J. Phys.15, 125012 (2013)

    2013

  32. [40]

    Ballarin, L

    G. Ballarin, L. Grigoryeva, and J.-P. Ortega, J. Mach. Learn. Res.25, 1 (2024)

    2024

  33. [41]

    Ballarin, L

    G. Ballarin, L. Grigoryeva, and J.-P. Ortega, inGeo- metric Science of Information, edited by F. Nielsen and F. Barbaresco (Springer Nature Switzerland, 2026) pp. 53–64

    2026

  34. [42]

    Haruna and K

    T. Haruna and K. Nakajima, (2026), https://github.com/taichiharuna/mur

    2026

Pith tools

Reviewed June 30, 2026 · model on record in the stance chip above.