Are cortical microcircuits optimized for information flux? -- A simulation-based reverse engineering study
Pith reviewed 2026-06-30 19:45 UTC · model grok-4.3
The pith
A larger embedding network around a dense core population markedly increases the core's information flux through biases and stochastic drive.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
In the model, the embedding network exerts a pronounced flux-enhancing effect on the core dynamics by generating effective biases that shift core neurons into a higher-entropy operating regime and by supplying stochastic fluctuations that prevent trapping in simple fixed-point or oscillatory attractors through the mechanism of Recurrence Resonance.
What carries the argument
The embedding network, which generates effective biases and stochastic fluctuations to enable recurrence resonance in the core.
If this is right
- Information flux rises further when core neurons receive individually optimized biases.
- Such biases can emerge from a simple self-organization principle.
- The mechanism is relevant for interpreting biological neural circuits and for designing artificial recurrent systems such as reservoir computers.
Where Pith is reading between the lines
- This embedding structure may represent a general design principle for recurrent networks to sustain high-entropy dynamics.
- Similar embedding could improve performance in engineered reservoir computers by mimicking the biological case.
- Testing whether real cortical tissue shows higher flux than de-embedded cores would directly check the model's prediction.
Load-bearing premise
The simplified model of cortical layer 5 architecture, with its densely interconnected core embedded in a larger supporting network, sufficiently captures the structural and dynamical features relevant to information flux in real biological microcircuits.
What would settle it
Direct measurement of mutual information between successive states in biological cortical layer 5 circuits, compared against versions where the surrounding network is removed or silenced.
Figures
read the original abstract
A sufficiently large information flux in recurrent neural networks, quantified by the mutual information between successive network states, is considered a prerequisite for rich information processing capabilities. This raises the question of whether biological neural networks, such as cortical microcolumns, may be structurally organized to enhance information flux. To investigate this possibility, we study a simplified model of the cortical layer 5 architecture, in which a densely and strongly interconnected core population is embedded within a larger supporting network. Surprisingly, we find that the embedding network exerts a pronounced flux-enhancing effect on the core dynamics. Systematic reverse-engineering analyses reveal that the embedding network provides two key contributions: first, it generates effective biases that shift core neurons into a higher-entropy operating regime; second, it supplies stochastic fluctuations that prevent the network from becoming trapped in simple fixed-point or oscillatory attractors through the mechanism of Recurrence Resonance. We further show that the information flux can be increased even beyond the biologically embedded case by applying individually optimized biases to the core neurons, and that these biases can emerge from a simple self-organization principle. Our findings are relevant both for the functional interpretation of biological neural circuits and for the design of artificial recurrent systems such as reservoir computers.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims that in a simplified model of cortical layer 5, a densely interconnected core embedded in a larger supporting network exhibits enhanced information flux (mutual information between successive states) due to effective biases shifting neurons to higher-entropy regimes and stochastic fluctuations enabling 'Recurrence Resonance' to avoid simple attractors; it further claims this flux can be increased via individually optimized biases arising from a self-organization principle, with implications for biological circuit interpretation and reservoir computing design.
Significance. If the model architecture and flux measure prove representative of biological layer 5 and the enhancement is robust, the work would provide a mechanistic account of how embedding connectivity could support rich dynamics, offering testable predictions for circuit optimization and design rules for artificial recurrent systems. The reverse-engineering approach is a methodological strength when accompanied by parameter sweeps and external benchmarks.
major comments (3)
- [Model architecture] Model architecture (Methods/Results sections): the claim that the embedding network exerts a 'pronounced flux-enhancing effect' relies on a core-plus-surround topology, but the manuscript provides no comparison of the chosen connection probabilities, weight distributions, or neuron-type ratios to empirical layer-5 statistics from paired recordings or EM reconstructions; without this, the enhancement may be an artifact of the specific dense-core parameters rather than a general feature of cortical architecture.
- [Results] Information flux definition and Recurrence Resonance (abstract and §Results): the mutual-information measure is defined internally to the simulations and the mechanism is identified post-hoc; the manuscript does not report external benchmarks, alternative flux measures, or controls that would demonstrate the enhancement is not reducible to quantities fitted within the same model runs.
- [Results] Validation and robustness (Results): the abstract and described analyses report no quantitative parameter values, statistical controls, sensitivity analyses across connection strengths, or direct validation against biological firing statistics, leaving the central flux-enhancement claim difficult to assess for robustness or artifact.
minor comments (2)
- [Abstract] The abstract would benefit from one or two quantitative effect sizes (e.g., fold-change in mutual information) to allow readers to gauge the magnitude of the reported enhancement.
- [Methods] Notation for the information-flux quantity and the precise definition of 'Recurrence Resonance' should be introduced with an equation in the main text rather than left implicit.
Simulated Author's Rebuttal
We are grateful to the referee for their thorough review and valuable suggestions, which have helped us identify areas for improvement. Below, we address each major comment in detail and outline the changes we plan to implement in the revised version of the manuscript.
read point-by-point responses
-
Referee: [Model architecture] Model architecture (Methods/Results sections): the claim that the embedding network exerts a 'pronounced flux-enhancing effect' relies on a core-plus-surround topology, but the manuscript provides no comparison of the chosen connection probabilities, weight distributions, or neuron-type ratios to empirical layer-5 statistics from paired recordings or EM reconstructions; without this, the enhancement may be an artifact of the specific dense-core parameters rather than a general feature of cortical architecture.
Authors: We acknowledge that the manuscript does not include direct comparisons to empirical layer-5 statistics. Our model is a simplified representation designed to investigate the general principle of embedding effects on information flux. In the revision, we will add to the Methods section references to empirical studies on cortical connectivity and discuss how our parameters are consistent with reported biological ranges. This will help mitigate concerns about the results being specific to arbitrary choices. revision: partial
-
Referee: [Results] Information flux definition and Recurrence Resonance (abstract and §Results): the mutual-information measure is defined internally to the simulations and the mechanism is identified post-hoc; the manuscript does not report external benchmarks, alternative flux measures, or controls that would demonstrate the enhancement is not reducible to quantities fitted within the same model runs.
Authors: While the mutual information measure is defined based on the simulation states, it is a standard and objective metric for information flux. The identification of Recurrence Resonance was based on systematic variations and ablations. We will revise the Results section to include alternative flux measures (e.g., state entropy and transfer entropy) and additional controls with randomized networks to demonstrate that the enhancement is due to the architecture. revision: yes
-
Referee: [Results] Validation and robustness (Results): the abstract and described analyses report no quantitative parameter values, statistical controls, sensitivity analyses across connection strengths, or direct validation against biological firing statistics, leaving the central flux-enhancement claim difficult to assess for robustness or artifact.
Authors: The manuscript includes quantitative results, but we agree that more explicit statistical controls and sensitivity analyses are warranted. We will add sensitivity analyses varying connection strengths and report means and standard deviations from repeated simulations. For direct validation against biological firing statistics, this lies outside the scope of the current simulation study; we will include a new subsection on model limitations and qualitative comparisons to known cortical dynamics. revision: partial
Circularity Check
No circularity in simulation-based derivation chain
full rationale
The paper defines information flux as mutual information between successive network states, then runs simulations on a simplified L5 model to observe that embedding enhances flux via biases and stochastic fluctuations (named Recurrence Resonance). These effects are emergent outputs of the model dynamics, not presupposed or fitted to force the result. No load-bearing step reduces a claimed prediction to a self-defined quantity, fitted input, or self-citation chain; the central claims rest on explicit simulation comparisons rather than tautological re-expression of inputs. The derivation is therefore self-contained.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
Deep learning.nature, 521(7553):436–444, 2015
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning.nature, 521(7553):436–444, 2015
2015
-
[2]
Re- view of deep learning: Concepts, cnn architectures, challenges, applications, future directions
Laith Alzubaidi, Jinglan Zhang, Amjad J Humaidi, Ayad Al-Dujaili, Ye Duan, Omran Al- Shamma, Jos´ e Santamar´ ıa, Mohammed A Fadhel, Muthana Al-Amidie, and Laith Farhan. Re- view of deep learning: Concepts, cnn architectures, challenges, applications, future directions. Journal of big Data, 8(1):1–74, 2021
2021
-
[3]
Recent advances in natural language pro- cessing via large pre-trained language models: A survey.ACM Computing Surveys, 56(2):1–40, 2023
Bonan Min, Hayley Ross, Elior Sulem, Amir Pouran Ben Veyseh, Thien Huu Nguyen, Oscar Sainz, Eneko Agirre, Ilana Heintz, and Dan Roth. Recent advances in natural language pro- cessing via large pre-trained language models: A survey.ACM Computing Surveys, 56(2):1–40, 2023
2023
-
[4]
Universality and individuality in neural dynamics across large populations of recurrent networks
Niru Maheswaranathan, Alex H Williams, Matthew D Golub, Surya Ganguli, and David Sussillo. Universality and individuality in neural dynamics across large populations of recurrent networks. Advances in neural information processing systems, 2019:15629, 2019
2019
-
[5]
Recurrent neural networks are univer- sal approximators
Anton Maximilian Sch¨ afer and Hans Georg Zimmermann. Recurrent neural networks are univer- sal approximators. InInternational Conference on Artificial Neural Networks, pages 632–640. Springer, 2006
2006
-
[6]
Universal approximation of flows of control systems by recurrent neural networks
Miguel Aguiar, Amritam Das, and Karl H Johansson. Universal approximation of flows of control systems by recurrent neural networks. In2023 62nd IEEE Conference on Decision and Control (CDC), pages 2320–2327. IEEE, 2023
2023
-
[7]
echo state
Herbert Jaeger. The “echo state” approach to analysing and training recurrent neural networks- with an erratum note.Bonn, Germany: German National Research Center for Information Technology GMD Technical Report, 148(34):13, 2001
2001
-
[8]
Optimal sequence memory in driven random networks.Physical Review X, 8(4):041029, 2018
Jannis Schuecker, Sven Goedeke, and Moritz Helias. Optimal sequence memory in driven random networks.Physical Review X, 8(4):041029, 2018
2018
-
[9]
Connectivity, dynamics, and memory in reservoir computing with binary and analog neurons.Neural computation, 22(5):1272–1311, 2010
Lars B¨ using, Benjamin Schrauwen, and Robert Legenstein. Connectivity, dynamics, and memory in reservoir computing with binary and analog neurons.Neural computation, 22(5):1272–1311, 2010
2010
-
[10]
Information process- ing capacity of dynamical systems.Scientific reports, 2(1):1–7, 2012
Joni Dambre, David Verstraeten, Benjamin Schrauwen, and Serge Massar. Information process- ing capacity of dynamical systems.Scientific reports, 2(1):1–7, 2012
2012
-
[11]
Randomly connected networks have short temporal memory.Neural computation, 25(6):1408–1439, 2013
Edward Wallace, Hamid Reza Maei, and Peter E Latham. Randomly connected networks have short temporal memory.Neural computation, 25(6):1408–1439, 2013
2013
-
[12]
Fading memory echo state networks are universal.Neural Networks, 138:10–13, 2021
Lukas Gonon and Juan-Pablo Ortega. Fading memory echo state networks are universal.Neural Networks, 138:10–13, 2021
2021
-
[13]
Gradient-based learning drives robust representations in recurrent neural networks by balancing compression and expansion.Nature Machine Intelligence, 4(6):564–573, 2022
Matthew Farrell, Stefano Recanatesi, Timothy Moore, Guillaume Lajoie, and Eric Shea-Brown. Gradient-based learning drives robust representations in recurrent neural networks by balancing compression and expansion.Nature Machine Intelligence, 4(6):564–573, 2022
2022
-
[14]
Stimulus-dependent suppression of chaos in recurrent neural networks.Physical Review E, 82(1):011903, 2010
Kanaka Rajan, LF Abbott, and Haim Sompolinsky. Stimulus-dependent suppression of chaos in recurrent neural networks.Physical Review E, 82(1):011903, 2010
2010
-
[15]
arXiv preprint arXiv:1403.3369 , year=
Herbert Jaeger. Controlling recurrent neural networks by conceptors.arXiv preprint arXiv:1403.3369, 2014. 17
-
[16]
Understanding and controlling memory in recurrent neural networks
Doron Haviv, Alexander Rivkind, and Omri Barak. Understanding and controlling memory in recurrent neural networks. InInternational Conference on Machine Learning, pages 2663–2671. PMLR, 2019
2019
-
[17]
Suppressing chaos in neural networks by noise.Physical review letters, 69(26):3717, 1992
Lutz Molgedey, Johannes Schuchhardt, and Heinz Georg Schuster. Suppressing chaos in neural networks by noise.Physical review letters, 69(26):3717, 1992
1992
-
[18]
Noise-modulated neural networks as an application of stochastic resonance.Neurocomputing, 277:29–37, 2018
Shuhei Ikemoto, Fabio DallaLibera, and Koh Hosoda. Noise-modulated neural networks as an application of stochastic resonance.Neurocomputing, 277:29–37, 2018
2018
-
[19]
Recurrence resonance” in three-neuron motifs.Frontiers in computational neuroscience, 13, 2019
Patrick Krauss, Karin Prebeck, Achim Schilling, and Claus Metzner. Recurrence resonance” in three-neuron motifs.Frontiers in computational neuroscience, 13, 2019
2019
-
[20]
Florian B¨ onsel, Patrick Krauss, Claus Metzner, and Marius E Yamakou. Control of noise- induced coherent oscillations in time-delayed neural motifs.arXiv preprint arXiv:2106.11361, 2021
-
[21]
Dynamics and information import in recurrent neural net- works.Frontiers in Computational Neuroscience, 16, 2022
Claus Metzner and Patrick Krauss. Dynamics and information import in recurrent neural net- works.Frontiers in Computational Neuroscience, 16, 2022
2022
-
[22]
Recurrent neural networks as versatile tools of neuroscience research.Current opinion in neurobiology, 46:1–6, 2017
Omri Barak. Recurrent neural networks as versatile tools of neuroscience research.Current opinion in neurobiology, 46:1–6, 2017
2017
-
[23]
Highly nonrandom features of synaptic connectivity in local cortical circuits.PLoS biology, 3(3):e68, 2005
Sen Song, Per Jesper Sj¨ ostr¨ om, Markus Reigl, Sacha Nelson, and Dmitri B Chklovskii. Highly nonrandom features of synaptic connectivity in local cortical circuits.PLoS biology, 3(3):e68, 2005
2005
-
[24]
Is cortical connectivity optimized for storing information?Nature neuroscience, 19(5):749–755, 2016
Nicolas Brunel. Is cortical connectivity optimized for storing information?Nature neuroscience, 19(5):749–755, 2016
2016
-
[25]
Exploring Sparsity in Recurrent Neural Networks
Sharan Narang, Erich Elsen, Gregory Diamos, and Shubho Sengupta. Exploring sparsity in recurrent neural networks.arXiv preprint arXiv:1704.05119, 2017
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[26]
Sparsity through evolutionary pruning prevents neuronal networks from overfitting.Neural Networks, 128:305– 312, 2020
Richard C Gerum, Andr´ e Erpenbeck, Patrick Krauss, and Achim Schilling. Sparsity through evolutionary pruning prevents neuronal networks from overfitting.Neural Networks, 128:305– 312, 2020
2020
-
[27]
Effect of dilution in asym- metric recurrent neural networks.Neural Networks, 104:50–59, 2018
Viola Folli, Giorgio Gosti, Marco Leonetti, and Giancarlo Ruocco. Effect of dilution in asym- metric recurrent neural networks.Neural Networks, 104:50–59, 2018
2018
-
[28]
Analysis of structure and dynamics in three-neuron motifs.Frontiers in Computational Neuroscience, 13:5, 2019
Patrick Krauss, Alexandra Zankl, Achim Schilling, Holger Schulze, and Claus Metzner. Analysis of structure and dynamics in three-neuron motifs.Frontiers in Computational Neuroscience, 13:5, 2019
2019
-
[29]
Weight statistics controls dynamics in recurrent neural networks.PloS one, 14(4):e0214541, 2019
Patrick Krauss, Marc Schuster, Verena Dietrich, Achim Schilling, Holger Schulze, and Claus Met- zner. Weight statistics controls dynamics in recurrent neural networks.PloS one, 14(4):e0214541, 2019
2019
-
[30]
Nonlinear neural dynamics and classification accuracy in reservoir computing.Neural Computation, 37(8):1469–1504, 2025
Claus Metzner, Achim Schilling, Andreas Maier, and Patrick Krauss. Nonlinear neural dynamics and classification accuracy in reservoir computing.Neural Computation, 37(8):1469–1504, 2025
2025
-
[31]
Organizational regularities in recurrent neural networks.Frontiers in Complex Systems, 3:1636222, 2025
Claus Metzner, Achim Schilling, Andreas Maier, and Patrick Krauss. Organizational regularities in recurrent neural networks.Frontiers in Complex Systems, 3:1636222, 2025
2025
-
[32]
Modular brain networks.Annual Review of Psychology, 67:613–640, 2016
Olaf Sporns and Richard F Betzel. Modular brain networks.Annual Review of Psychology, 67:613–640, 2016
2016
-
[33]
Modular and hierarchically mod- ular organization of brain networks.Frontiers in Neuroscience, 4:200, 2010
David Meunier, Renaud Lambiotte, and Edward T Bullmore. Modular and hierarchically mod- ular organization of brain networks.Frontiers in Neuroscience, 4:200, 2010. 18
2010
-
[34]
Neural networks and physical systems with emergent collective computational abilities.Proceedings of the national academy of sciences, 79(8):2554–2558, 1982
John J Hopfield. Neural networks and physical systems with emergent collective computational abilities.Proceedings of the national academy of sciences, 79(8):2554–2558, 1982
1982
-
[35]
Intrinsic noise improves speech recognition in a computational model of the auditory pathway.Frontiers in Neuroscience, 16:908330, 2022
Achim Schilling, Richard Gerum, Claus Metzner, Andreas Maier, and Patrick Krauss. Intrinsic noise improves speech recognition in a computational model of the auditory pathway.Frontiers in Neuroscience, 16:908330, 2022
2022
-
[36]
Patrick Krauss, Konstantin Tziridis, Claus Metzner, Achim Schilling, Ulrich Hoppe, and Holger Schulze. Stochastic resonance controlled upregulation of internal noise after hearing loss as a putative cause of tinnitus-related neuronal hyperactivity.Frontiers in neuroscience, 10:597, 2016
2016
-
[37]
Achim Schilling, Konstantin Tziridis, Holger Schulze, and Patrick Krauss. The stochastic res- onance model of auditory perception: A unified explanation of tinnitus development, zwicker tone illusion, and residual inhibition.Progress in Brain Research, 262:139–157, 2021
2021
-
[38]
Predictive coding and stochastic resonance as fundamental principles of auditory phantom perception.Brain, 146(12):4809–4825, 2023
Achim Schilling, William Sedley, Richard Gerum, Claus Metzner, Konstantin Tziridis, Andreas Maier, Holger Schulze, Fan-Gang Zeng, Karl J Friston, and Patrick Krauss. Predictive coding and stochastic resonance as fundamental principles of auditory phantom perception.Brain, 146(12):4809–4825, 2023
2023
-
[39]
Recurrence resonance- noise-enhanced dynamics in recurrent neural networks.Frontiers in Complex Systems, 2:1479417, 2024
Claus Metzner, Achim Schilling, Andreas Maier, and Patrick Krauss. Recurrence resonance- noise-enhanced dynamics in recurrent neural networks.Frontiers in Complex Systems, 2:1479417, 2024
2024
-
[40]
Computation at the edge of chaos: Phase transitions and emergent compu- tation.Physica D: Nonlinear Phenomena, 42(1-3):12–37, 1990
Chris G Langton. Computation at the edge of chaos: Phase transitions and emergent compu- tation.Physica D: Nonlinear Phenomena, 42(1-3):12–37, 1990
1990
-
[41]
Real-time computation at the edge of chaos in recurrent neural networks.Neural computation, 16(7):1413–1436, 2004
Nils Bertschinger and Thomas Natschl¨ ager. Real-time computation at the edge of chaos in recurrent neural networks.Neural computation, 16(7):1413–1436, 2004
2004
-
[42]
The mechanism of stochastic resonance
Roberto Benzi, Alfonso Sutera, and Angelo Vulpiani. The mechanism of stochastic resonance. Journal of Physics A: mathematical and general, 14(11):L453–L457, 1981
1981
-
[43]
Stochastic resonance and the benefits of noise: from ice ages to crayfish and squids.Nature, 373(6509):33–36, 1995
Kurt Wiesenfeld and Frank Moss. Stochastic resonance and the benefits of noise: from ice ages to crayfish and squids.Nature, 373(6509):33–36, 1995
1995
-
[44]
drivers” from “modulators
S Murray Sherman and RW Guillery. On the actions that one nerve cell can have on another: distinguishing “drivers” from “modulators”.Proceedings of the National Academy of Sciences, 95(12):7121–7126, 1998
1998
-
[45]
Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects.Nature neuroscience, 2(1):79–87, 1999
Rajesh PN Rao and Dana H Ballard. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects.Nature neuroscience, 2(1):79–87, 1999
1999
-
[46]
A theory of cortical responses.Philosophical transactions of the Royal Society B: Biological sciences, 360(1456):815–836, 2005
Karl Friston. A theory of cortical responses.Philosophical transactions of the Royal Society B: Biological sciences, 360(1456):815–836, 2005
2005
-
[47]
Homeostatic plasticity in the developing nervous system
Gina G Turrigiano and Sacha B Nelson. Homeostatic plasticity in the developing nervous system. Nature reviews neuroscience, 5(2):97–107, 2004
2004
-
[48]
Theory for the development of neu- ron selectivity: orientation specificity and binocular interaction in visual cortex.Journal of Neuroscience, 2(1):32–48, 1982
Elie L Bienenstock, Leon N Cooper, and Paul W Munro. Theory for the development of neu- ron selectivity: orientation specificity and binocular interaction in visual cortex.Journal of Neuroscience, 2(1):32–48, 1982
1982
-
[49]
The self-tuning neuron: synaptic scaling of excitatory synapses.Cell, 135(3):422–435, 2008
Gina G Turrigiano. The self-tuning neuron: synaptic scaling of excitatory synapses.Cell, 135(3):422–435, 2008. 19
2008
-
[50]
Detect- ing rich-club ordering in complex networks.Nature physics, 2(2):110–115, 2006
Vittoria Colizza, Alessandro Flammini, M Angeles Serrano, and Alessandro Vespignani. Detect- ing rich-club ordering in complex networks.Nature physics, 2(2):110–115, 2006
2006
-
[51]
inter” and column “peri
Martijn P Van Den Heuvel and Olaf Sporns. Rich-club organization of the human connectome. Journal of Neuroscience, 31(44):15775–15786, 2011. 20 core periphery interneurons t0 = 1 t0 = 5 (a) (b) (c) (d) (e) intra-triplet inter-triplet intra-triplet inter-triplet (f) Figure 1:The model system.•(a)Three neural sub-populations and their connections.•(b) Conne...
2011
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.