Harnessing behavioral diversity to understand circuits for cognition
Pith reviewed 2026-05-25 17:44 UTC · model grok-4.3
The pith
Artificial neural networks connect neural dynamics to rich behavioral data to explain diversity in cognitive strategies.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
With the increasing acquisition of large-scale neural recordings comes the challenge of inferring the computations they perform and understanding how these give rise to behavior. Recording neural data during rich behavioral tasks, quantifying movements to capture brain states, and employing artificial neural networks to connect dynamics with behavior can reveal optimal solutions for particular behaviors and explain diversity in behavioral strategies.
What carries the argument
Artificial neural networks trained on behavioral tasks, used to connect observed neural dynamics with rich behavioral data and generate hypotheses about how activity drives behavior.
If this is right
- Neural data recorded during rich behavioral tasks will better model cognitive processes and estimate latent behavioral variables.
- Quantifying movements will provide a more complete picture of how they shape neural dynamics and reflect changes in brain state.
- Artificial neural networks will reveal how particular behaviors can be optimally solved.
- Diversity in behavioral strategies across subjects or conditions can be explained through differences in underlying neural activity.
Where Pith is reading between the lines
- This framework could guide the design of new experiments that directly test ANN-derived predictions in biological circuits.
- Incorporating movement quantification might reveal hidden motor influences on circuits previously studied only in cognitive contexts.
- Success would encourage shifting neuroscience away from simplified tasks toward more naturalistic behavioral paradigms.
Load-bearing premise
Hypotheses generated by artificial neural networks about optimal behavioral solutions will map meaningfully onto the mechanisms in biological neural circuits rather than remaining loose analogies.
What would settle it
Recordings from animals performing a task show neural activity patterns that contradict the specific predictions derived from an artificial neural network trained on the same task.
Figures
read the original abstract
With the increasing acquisition of large-scale neural recordings comes the challenge of inferring the computations they perform and understanding how these give rise to behavior. Here, we review emerging conceptual and technological advances that begin to address this challenge, garnering insights from both biological and artificial neural networks. We argue that neural data should be recorded during rich behavioral tasks, to model cognitive processes and estimate latent behavioral variables. Careful quantification of animal movements can also provide a more complete picture of how movements shape neural dynamics and reflect changes in brain state, such as arousal or stress. Artificial neural networks (ANNs) could serve as an important tool to connect neural dynamics and rich behavioral data. ANNs have already begun to reveal how particular behaviors can be optimally solved, generating hypotheses about how observed neural activity might drive behavior and explaining diversity in behavioral strategies.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript reviews conceptual and technological advances for inferring computations from large-scale neural recordings and linking them to behavior. It advocates recording neural data during rich behavioral tasks to model cognitive processes and latent variables, careful quantification of movements to capture brain states such as arousal, and the use of artificial neural networks (ANNs) to connect neural dynamics with behavioral data by revealing optimal solutions, generating hypotheses about how neural activity drives behavior, and explaining diversity in behavioral strategies.
Significance. If the proposed synthesis holds, the perspective could help bridge neural recordings and behavior by positioning ANNs as hypothesis generators for circuit mechanisms underlying cognitive diversity. The emphasis on rich behavioral tasks and movement quantification is timely given advances in recording technologies. The argument's impact hinges on whether ANN-derived solutions can be shown to correspond to biological mechanisms beyond analogy, a point the abstract asserts has already begun but does not substantiate with concrete mappings in the provided text.
major comments (1)
- [Abstract] Abstract (final paragraph): The claim that 'ANNs have already begun to reveal how particular behaviors can be optimally solved, generating hypotheses about how observed neural activity might drive behavior' is load-bearing for the central argument that ANNs serve as a tool to connect neural dynamics and behavior. No specific literature examples, circuit elements, or validated predictions (e.g., an ANN-derived unit matching a recorded biological population statistic under matched task conditions) are referenced to establish this correspondence rather than loose functional analogy.
Simulated Author's Rebuttal
We thank the referee for the constructive comments on our perspective. We address the single major comment below and will revise the manuscript accordingly.
read point-by-point responses
-
Referee: [Abstract] Abstract (final paragraph): The claim that 'ANNs have already begun to reveal how particular behaviors can be optimally solved, generating hypotheses about how observed neural activity might drive behavior' is load-bearing for the central argument that ANNs serve as a tool to connect neural dynamics and behavior. No specific literature examples, circuit elements, or validated predictions (e.g., an ANN-derived unit matching a recorded biological population statistic under matched task conditions) are referenced to establish this correspondence rather than loose functional analogy.
Authors: We agree that the abstract would be strengthened by explicit citations supporting the claim. The body of the manuscript reviews relevant literature on ANN-derived solutions for behavioral tasks (e.g., optimal control in navigation and perceptual decisions) and their comparison to neural data, but these are not referenced in the abstract itself. In revision we will add one or two targeted citations to the abstract (and ensure the corresponding examples are highlighted in the main text) to illustrate cases where ANN units or dynamics have been directly compared to biological recordings under matched conditions. revision: yes
Circularity Check
No derivations or predictions; conceptual review only
full rationale
This is a conceptual review paper with no equations, fitted parameters, model derivations, or quantitative predictions. The abstract and text discuss emerging ideas about recording neural data during rich behaviors and using ANNs as a tool, but present no derivation chain, self-citation load-bearing premise, ansatz, or renaming of results. The central argument is argumentative rather than deductive, so none of the enumerated circularity patterns apply. The paper is self-contained as a perspective piece and receives the default non-circularity finding.
Axiom & Free-Parameter Ledger
axioms (2)
- domain assumption Large-scale neural recordings during behavior can be used to infer the computations performed by circuits for cognition.
- domain assumption Quantification of animal movements provides information about brain state changes such as arousal or stress that shape neural dynamics.
Reference graph
Works this paper leans on
-
[1]
Nat Neurosci 2003, 6:1224–1229
Uchida N, Mainen ZF: Speed and accuracy of olfactory discrimination in the rat. Nat Neurosci 2003, 6:1224–1229
work page 2003
-
[2]
Freedman DJ, Assad JA: Experience-dependent representation of visual categories in parietal cortex. Nature 2006, 443:85–88
work page 2006
-
[3]
J Neurosci Off J Soc Neurosci 2012, 32:3726–3735
Raposo D, Sheppard JP, Schrater PR, Churchland AK: Multisensory decision-making in rats and humans. J Neurosci Off J Soc Neurosci 2012, 32:3726–3735
work page 2012
-
[4]
Mante V, Sussillo D, Shenoy KV, Newsome WT: Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature 2013, 503:78–84
work page 2013
-
[5]
McGinley MJ, David SV, McCormick DA: Cortical Membrane Potential Signature of Optimal States for Sensory Signal Detection. Neuron 2015, 87:179–192
work page 2015
-
[6]
Vinck M, Batista-Brito R, Knoblich U, Cardin JA: Arousal and Locomotion Make Distinct Contributions to Cortical Activity Patterns and Visual Encoding. Neuron 2015, 86:740–754
work page 2015
-
[7]
Niell CM, Stryker MP: Modulation of Visual Responses by Behavioral State in Mouse Visual Cortex. Neuron 2010, 65:472–479
work page 2010
-
[8]
Crochet S, Petersen CCH: Correlating whisker behavior with membrane potential in barrel cortex of awake mice. Nat Neurosci 2006, 9:608–610
work page 2006
-
[9]
bioRxiv 2018, doi:10.1101/306019
Stringer C, Pachitariu M, Steinmetz N, Reddy CB, Carandini M, Harris KD: Spontaneous behaviors drive multidimensional, brain-wide population activity. bioRxiv 2018, doi:10.1101/306019
-
[10]
bioRxiv 2018, doi:10.1101/308288
Musall S, Kaufman MT, Gluf S, Churchland AK: Movement-related activity dominates cortex during sensory-guided decision making. bioRxiv 2018, doi:10.1101/308288
-
[11]
Nat Neurosci 2014, 17:1455–1462
Gomez-Marin A, Paton JJ, Kampff AR, Costa RM, Mainen ZF: Big behavioral data: psychology, ethology and the foundations of neuroscience. Nat Neurosci 2014, 17:1455–1462
work page 2014
-
[12]
Curr Opin Neurobiol 2015, 32:148–155
Gao P, Ganguli S: On simplicity and complexity in the brave new world of large-scale neuroscience. Curr Opin Neurobiol 2015, 32:148–155
work page 2015
-
[13]
Krakauer JW, Ghazanfar AA, Gomez-Marin A, MacIver MA, Poeppel D: Neuroscience Needs Behavior: Correcting a Reductionist Bias. Neuron 2017, 93:480–490
work page 2017
-
[14]
Carnevale F, de Lafuente V, Romo R, Barak O, Parga N: Dynamic Control of Response Criterion in Premotor Cortex during Perceptual Detection under Temporal Uncertainty. Neuron 2015, 86:1067–1077
work page 2015
-
[15]
Nat Neurosci 2015, 18:1025–1033
Sussillo D, Churchland MM, Kaufman MT, Shenoy KV: A neural network that finds a naturalistic solution for the production of muscle activity. Nat Neurosci 2015, 18:1025–1033
work page 2015
-
[16]
Rajan K, Harvey CD, Tank DW: Recurrent Network Models of Sequence Generation and Memory. Neuron 2016, 90:128–142
work page 2016
-
[17]
Remington ED, Narain D, Hosseini EA, Jazayeri M: Flexible Sensorimotor Computations through Rapid Reconfiguration of Cortical Dynamics. Neuron 2018, 98:1005-1019.e5
work page 2018
-
[18]
bioRxiv 2018, doi:10.1101/465419
Sohn H, Narain D, Meirhaeghe N, Jazayeri M: Bayesian computation through cortical latent dynamics. bioRxiv 2018, doi:10.1101/465419
-
[19]
Wang J, Narain D, Hosseini EA, Jazayeri M: Flexible timing by temporal scaling of cortical responses. Nat Neurosci 2018, 21:102–110
work page 2018
-
[20]
Kurikawa T, Haga T, Handa T, Harukuni R, Fukai T: Neuronal stability in medial frontal cortex sets individual variability in decision-making. Nat Neurosci 2018, 21:1764
work page 2018
-
[21]
Brunton BW, Botvinick MM, Brody CD: Rats and Humans Can Optimally Accumulate Evidence for Decision-Making. Science 2013, 340:95–98. 12
work page 2013
-
[22]
Nature 2015, advance online publication
Hanks TD, Kopec CD, Brunton BW, Duan CA, Erlich JC, Brody CD: Distinct relationships of parietal and prefrontal cortices to evidence accumulation. Nature 2015, advance online publication
work page 2015
-
[23]
Nat Neurosci 2017, 20:1285–1292
Yates JL, Park IM, Katz LN, Pillow JW, Huk AC: Functional dissection of signal and noise in MT and LIP during decision-making. Nat Neurosci 2017, 20:1285–1292
work page 2017
-
[24]
Sheppard JP, Raposo D, Churchland AK: Dynamic weighting of multisensory stimuli shapes decision-making in rats and humans. J Vis 2013, 13:4–4
work page 2013
-
[25]
Fetsch CR, Pouget A, DeAngelis GC, Angelaki DE: Neural correlates of reliability-based cue weighting during multisensory integration. Nat Neurosci 2012, 15:146–154
work page 2012
-
[26]
Roitman JD, Shadlen MN: Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J Neurosci 2002, 22:9475–9489
work page 2002
-
[27]
Annu Rev Neurosci 2007, 30:535–574
Gold JI, Shadlen MN: The Neural Basis of Decision Making. Annu Rev Neurosci 2007, 30:535–574
work page 2007
-
[28]
Lak A, Costa GM, Romberg E, Koulakov AA, Mainen ZF, Kepecs A: Orbitofrontal Cortex Is Required for Optimal Waiting Based on Decision Confidence. Neuron 2014, 84:190–201
work page 2014
-
[29]
Lak A, Nomoto K, Keramati M, Sakagami M, Kepecs A: Midbrain Dopamine Neurons Signal Belief in Choice Accuracy during a Perceptual Decision. Curr Biol CB 2017, 27:821–832
work page 2017
-
[30]
Fleming SM, Putten EJ van der, Daw ND: Neural mediators of changes of mind about perceptual decisions. Nat Neurosci 2018, 21:617
work page 2018
-
[31]
Kaufman MT, Churchland MM, Ryu SI, Shenoy KV: Vacillation, indecision and hesitation in moment-by-moment decoding of monkey motor cortex. eLife 2015, 4:e04677
work page 2015
-
[32]
bioRxiv 2016, doi:10.1101/051912
Burgess CP, Steinmetz N, Lak A, Zatka-Haas P, Ranson A, Wells M, Schroeder S, Jacobs EAK, Reddy CB, Soares S, et al.: High-yield methods for accurate two-alternative visual psychophysics in head-fixed mice. bioRxiv 2016, doi:10.1101/051912
-
[33]
Nat Neurosci 2016, advance online publication
Morcos AS, Harvey CD: History-dependent variability in population dynamics during evidence accumulation in cortex. Nat Neurosci 2016, advance online publication
work page 2016
-
[34]
Huk A, Bonnen K, He BJ: Beyond Trial-Based Paradigms: Continuous Behavior, Ongoing Neural Activity, and Natural Stimuli. J Neurosci 2018, 38:7551–7558
work page 2018
-
[35]
Erlich JC, Bialek M, Brody CD: A Cortical Substrate for Memory-Guided Orienting in the Rat. Neuron 2011, 72:330–343
work page 2011
-
[36]
Mathis A, Mamidanna P, Cury KM, Abe T, Murthy VN, Mathis MW, Bethge M: DeepLabCut: markerless pose estimation of user-defined body parts with deep learning. Nat Neurosci 2018, 21:1281
work page 2018
-
[37]
Pereira TD, Aldarondo DE, Willmore L, Kislin M, Wang SS-H, Murthy M, Shaevitz JW: Fast animal pose estimation using deep neural networks. Nat Methods 2019, 16:117
work page 2019
-
[38]
Drugowitsch J, Wyart V, Devauchelle A-D, Koechlin E: Computational Precision of Mental Inference as Critical Source of Human Choice Suboptimality. Neuron 2016, 92:1–14
work page 2016
-
[39]
bioRxiv 2018, doi:10.1101/461129
Miller KJ, Botvinick MM, Brody CD: From predictive models to cognitive models: An analysis of rat behavior in the two-armed bandit task. bioRxiv 2018, doi:10.1101/461129
-
[40]
Purcell BA, Kiani R: Neural Mechanisms of Post-error Adjustments of Decision Policy in Parietal Cortex. Neuron 2016, 89:658–671
work page 2016
-
[41]
Ann N Y Acad Sci 2007, 1104:35–53
O’doherty JP, Hampton A, Kim H: Model-Based fMRI and Its Application to Reward Learning and Decision Making. Ann N Y Acad Sci 2007, 1104:35–53
work page 2007
-
[42]
Cereb Cortex N Y N 1991 2014, 24:3232–3245
Fernandes HL, Stevenson IH, Phillips AN, Segraves MA, Kording KP: Saliency and saccade encoding in the frontal eye field during natural scene search. Cereb Cortex N Y N 1991 2014, 24:3232–3245. 13
work page 1991
-
[43]
In Advances in Neural Information Processing Systems 31
Roy NG, Bak JH, Akrami A, Brody C, Pillow JW: Efficient inference for time-varying behavior during learning. In Advances in Neural Information Processing Systems 31. Edited by Bengio S, Wallach H, Larochelle H, Grauman K, Cesa-Bianchi N, Garnett R. Curran Associates, Inc.; 2018:5700–5710
work page 2018
-
[44]
bioRxiv 2017, doi:10.1101/214262
Gao P, Trautmann E, Yu BM, Santhanam G, Ryu S, Shenoy K, Ganguli S: A theory of multineuronal dimensionality, dynamics and measurement. bioRxiv 2017, doi:10.1101/214262
-
[45]
Tinbergen N: The Study of Instinct: with a new Preface. Oxford University Press; 1989
work page 1989
-
[46]
Proc Natl Acad Sci U S A 1991, 88:11339–11343
Doupe AJ, Konishi M: Song-selective auditory circuits in the vocal control system of the zebra finch. Proc Natl Acad Sci U S A 1991, 88:11339–11343
work page 1991
-
[47]
Stensmyr MC, Dweck HKM, Farhan A, Ibba I, Strutz A, Mukunda L, Linz J, Grabe V, Steck K, Lavista-Llanos S, et al.: A conserved dedicated olfactory circuit for detecting harmful microbes in Drosophila. Cell 2012, 151:1345–1357
work page 2012
-
[48]
Anderson DJ, Perona P: Toward a Science of Computational Ethology. Neuron 2014, 84:18–31
work page 2014
-
[49]
Neuropsychopharmacology 2019, doi:10.1038/s41386-018-0303-6
Coffey KR, Marx RG, Neumaier JF: DeepSqueak: a deep learning-based system for detection and analysis of ultrasonic vocalizations. Neuropsychopharmacology 2019, doi:10.1038/s41386-018-0303-6
-
[50]
Torquet N, Marti F, Campart C, Tolu S, Nguyen C, Oberto V, Benallaoua M, Naudé J, Didienne S, Debray N, et al.: Social interactions impact on the dopaminergic system and drive individuality. Nat Commun 2018, 9:3081
work page 2018
-
[51]
Stephens GJ, Johnson-Kerner B, Bialek W, Ryu WS: Dimensionality and Dynamics in the Behavior of C. elegans. PLOS Comput Biol 2008, 4:e1000028
work page 2008
-
[52]
Girdhar K, Gruebele M, Chemla YR: The Behavioral Space of Zebrafish Locomotion and Its Neural Network Analog. PloS One 2015, 10:e0128668
work page 2015
-
[53]
Berman GJ, Choi DM, Bialek W, Shaevitz JW: Mapping the stereotyped behaviour of freely moving fruit flies. J R Soc Interface 2014, 11
work page 2014
-
[54]
Wiltschko AB, Johnson MJ, Iurilli G, Peterson RE, Katon JM, Pashkovski SL, Abraira VE, Adams RP, Datta SR: Mapping Sub-Second Structure in Mouse Behavior. Neuron 2015, 88:1121–1135
work page 2015
-
[55]
Markowitz JE, Gillis WF, Beron CC, Neufeld SQ, Robertson K, Bhagat ND, Peterson RE, Peterson E, Hyun M, Linderman SW, et al.: The Striatum Organizes 3D Behavior via Moment-to-Moment Action Selection. Cell 2018, 174:44-58.e17
work page 2018
-
[56]
bioRxiv 2018, doi:10.1101/366765
Clemens J, Deutsch D, Thiberge SY, Murthy M: Shared Song Object Detector Neurons in Drosophila Male and Female Brains Drive Divergent, Sex-Specific Behaviors. bioRxiv 2018, doi:10.1101/366765
-
[57]
Buzsáki G, Draguhn A: Neuronal oscillations in cortical networks. Science 2004, 304:1926–1929
work page 2004
-
[58]
Fox MD, Snyder AZ, Vincent JL, Raichle ME: Intrinsic Fluctuations within Cortical Systems Account for Intertrial Variability in Human Behavior. Neuron 2007, 56:171– 184
work page 2007
-
[59]
Nat Neurosci 2009, 12:1594–1600
Cohen MR, Maunsell JHR: Attention improves performance primarily by reducing interneuronal correlations. Nat Neurosci 2009, 12:1594–1600
work page 2009
-
[60]
Fu Y, Tucciarone JM, Espinosa JS, Sheng N, Darcy DP, Nicoll RA, Huang ZJ, Stryker MP: A Cortical Circuit for Gain Control by Behavioral State. Cell 2014, 156:1139–1152
work page 2014
-
[61]
Reimer J, Froudarakis E, Cadwell CR, Yatsenko D, Denfield GH, Tolias AS: Pupil fluctuations track fast switching of cortical states during quiet wakefulness. Neuron 2014, 84:355–362
work page 2014
-
[62]
McGinley MJ, Vinck M, Reimer J, Batista-Brito R, Zagha E, Cadwell CR, Tolias AS, Cardin JA, McCormick DA: Waking State: Rapid Variations Modulate Neural and Behavioral Responses. Neuron 2015, 87:1143–1161. 14
work page 2015
-
[63]
J Neurosci Methods 2009, 178:323–326
Pham J, Cabrera SM, Sanchis-Segura C, Wood MA: Automated scoring of fear related behavior using EthoVision software. J Neurosci Methods 2009, 178:323–326
work page 2009
-
[64]
Füzesi T, Daviu N, Wamsteeker Cusulin JI, Bonin RP, Bains JS: Hypothalamic CRH neurons orchestrate complex behaviours after stress. Nat Commun 2016, 7:11937
work page 2016
-
[65]
Behav Brain Res 2009, 205:38–44
Egan RJ, Bergner CL, Hart PC, Cachat JM, Canavello PR, Elegante MF, Elkhayat SI, Bartels BK, Tien AK, Tien DH, et al.: Understanding behavioral and physiological phenotypes of stress and anxiety in zebrafish. Behav Brain Res 2009, 205:38–44
work page 2009
-
[66]
Hess EH, Polt JM: Pupil size as related to interest value of visual stimuli. Science 1960, 132:349–350
work page 1960
-
[67]
Hess EH, Polt JM: Pupil Size in Relation to Mental Activity during Simple Problem- Solving. Science 1964, 143:1190–1192
work page 1964
-
[68]
Reimer J, McGinley MJ, Liu Y, Rodenkirch C, Wang Q, McCormick DA, Tolias AS: Pupil fluctuations track rapid changes in adrenergic and cholinergic activity in cortex. Nat Commun 2016, 7
work page 2016
-
[69]
Urai AE, Braun A, Donner TH: Pupil-linked arousal is driven by decision uncertainty and alters serial choice bias. Nat Commun 2017, 8:14637
work page 2017
-
[70]
Nat Neurosci 2013, 16:1331–1339
Polack P-O, Friedman J, Golshani P: Cellular mechanisms of brain state–dependent gain modulation in visual cortex. Nat Neurosci 2013, 16:1331–1339
work page 2013
-
[71]
Shimaoka D, Harris KD, Carandini M: Effects of Arousal on Mouse Sensory Cortex Depend on Modality. Cell Rep 2018, 22:3160–3167
work page 2018
-
[72]
Schneider DM, Sundararajan J, Mooney R: A cortical filter that learns to suppress the acoustic consequences of movement. Nature 2018, 561:391
work page 2018
-
[73]
Zipser D, Andersen RA: A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons. Nature 1988, 331:679– 684
work page 1988
-
[74]
Fetz EE: Are movement parameters recognizably coded in the activity of single neurons? Behav Brain Sci 1992, 15:679–690
work page 1992
-
[75]
Banino A, Barry C, Uria B, Blundell C, Lillicrap T, Mirowski P, Pritzel A, Chadwick MJ, Degris T, Modayil J, et al.: Vector-based navigation using grid-like representations in artificial agents. Nature 2018, 557:429–433
work page 2018
-
[76]
ArXiv180307770 Cs Q-Bio Stat 2018,
Cueva CJ, Wei X-X: Emergence of grid-like representations by training recurrent neural networks to perform spatial localization. ArXiv180307770 Cs Q-Bio Stat 2018,
work page 2018
-
[77]
Proc Natl Acad Sci 2018, 115:E10313–E10322
Flesch T, Balaguer J, Dekker R, Nili H, Summerfield C: Comparing continual task learning in minds and machines. Proc Natl Acad Sci 2018, 115:E10313–E10322
work page 2018
-
[78]
Nat Neurosci 2019, doi:10.1038/s41593-018-0310-2
Yang GR, Joglekar MR, Song HF, Newsome WT, Wang X-J: Task representations in neural networks trained to perform many cognitive tasks. Nat Neurosci 2019, doi:10.1038/s41593-018-0310-2
-
[79]
Yang GR, Ganichev I, Wang X-J, Shlens J, Sussillo D: A Dataset and Architecture for Visual Reasoning with a Working Memory. ArXiv180306092 Cs 2018,
work page 2018
-
[80]
Driscoll LN, Golub MD, Sussillo D: Computation through Cortical Dynamics. Neuron 2018, 98:873–875
work page 2018
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.