Neurons and Cognition · arXiv:2611.17011 v1 · submitted 12 Nov 2026 · 54 references
Random Exploration is Random
The paper says
“Decision making often requires balancing exploitation of familiar options with exploration of uncertain alternatives. One strategy for exploration is random exploration, in which choices become more variable and less tied to the option that currently appears best.”
more of the abstract
However, it remains unclear whether this variability reflects genuine stochasticity or instead a deterministic structure that traditional cognitive models fail to capture. We examined this question using the Horizon Task in an aggregated dataset of 1,392 participants completing over 1,270,695 trials. In this task, exploration becomes more valuable when participants have more future choices, and choice variability increases under longer horizons. We tested whether more expressive neural network models could recover deterministic structure from this additional variability. Although neural networks provided better overall fits to behavior than the traditional model, long-horizon choices remained consistently less predictable than short-horizon choices. These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 13 Nov 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. A careful negative-result study with a stronger title than its evidence: flexible models improve fit but don't close the short- versus long-horizon predictability gap, which is a meaningful result even if 'truly substantially random' overstates it. This line is wrong
3 major objections · 9 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 54 references say.
The paper says, at §2.5, Fig. 3
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.94):
“If similar unmeasured factors influence exploratory choices, their effects would appear as unexplained variability in our models even if they were deterministic.”
Pith says: Objection 1 of 3 · major
The central claim rests on the inability of neural networks to close the H1–H6 predictability gap. However, the authors acknowledge in §3 (b184–b189) that unmeasured factors (e.g., eye movements, fidgeting, internal states) could account for the residual variability. This is a structural limitation of the negative result: the absence of evidence for structure is not evidence of absence.
more of objection 1
The paper's strongest claim (b8) that 'random exploration truly is substantially random' goes beyond what the data can support given this acknowledged limitation. The authors should either soften the conclusion to match the evidence (e.g., 'consistent with a stochastic component') or provide a stronger argument that the measured variables capture the relevant deterministic structure. As written, the conclusion overstates the strength of the negative result.
The paper says, at §2.5, Fig. 3; §2.6, Fig. 4
The review did not quote this passage.
Pith says: Objection 2 of 3 · major
The H1–H6 accuracy gap in the subject-level analysis is 6.0 percentage points (b122), and in the first-game analysis it is 7.5 percentage points (b130). The logistic baseline gaps are 6.1 and 11.3 percentage points respectively. While the gap narrows in some comparisons, it is not statistically tested whether the reduction is significant.
more of objection 2
The paper reports confidence intervals for individual accuracies but does not provide a formal test (e.g., interaction between horizon and model type) to assess whether the neural networks significantly reduce the gap relative to the baseline. Without such a test, the claim that 'limited model expressiveness cannot fully account' is weakened because the gap reduction might be within chance. The authors should report a statistical comparison (e.g., bootstrap test of the gap difference) to support their interpretation.
The paper says, at §2.8, Fig. 6
The review did not quote this passage.
Pith says: Objection 3 of 3 · major
The within-H6 trial analysis shows that predictability increases as the remaining horizon shortens, which is consistent with the value of future exploration driving variability. However, the final H6 trial accuracy remains below the H1 benchmark (Fig. 6). The authors do not discuss whether this residual gap could be due to the different number of free-choice trials (1 vs. 6) affecting the model's ability to predict, or to other factors like fatigue or attention decrement over longer games.
more of objection 3
A more detailed analysis of the final trial, controlling for the number of preceding choices, would strengthen the interpretation that the gap is specifically due to horizon-related exploration rather than other confounds.
9 minor objections
The paper says, at Abstract
Pith says: minor objection 1 of 9
The phrase 'substantially random' is vague. Consider rephrasing to 'consistent with a stochastic component' or 'cannot be fully explained by deterministic structure' to match the paper's own cautious language in the discussion.
The paper says, at §2.3, Table 1
Pith says: minor objection 2 of 9
The first-game fit shows a larger H1–H6 gap (11.3 pp) than the global fit (7.5 pp). The authors note this but do not discuss why the gap is larger in the first game. A brief comment would be helpful.
show 7 more
The paper says, at §2.4, Table 2
Pith says: minor objection 3 of 9
The pseudo-R² values are reported for subject-level models. It would be useful to also report the global model pseudo-R² for comparison, as the global analysis is the simplest test of the hypothesis.
The paper says, at §4.4.1
Pith says: minor objection 4 of 9
The data splitting procedure for subject-level models (80/10/10 within each participant) is described. However, it is not clear whether the test set contains games from participants seen during training (which is the case for subject-level models) or entirely unseen participants (which would be a stricter generalization test). Clarify this distinction.
The paper says, at §4.4.2
Pith says: minor objection 5 of 9
The early stopping patience is five epochs. Given the large dataset, this seems reasonable, but the authors should report the average number of epochs trained to ensure convergence was achieved.
The paper says, at Fig. 3, Fig. 4, Fig. 5
Pith says: minor objection 6 of 9
The error bars are 95% Wilson confidence intervals. It would be helpful to state in the caption whether these are computed across participants or across test trials. For the global model, it is presumably across trials; for subject-level models, it could be across participants. Clarify.
The paper says, at §2.7, Fig. 5
Pith says: minor objection 7 of 9
The cross-game history analysis uses a persistence parameter β that is learned. The authors report low β values, but do not provide confidence intervals or tests against zero. A brief note on whether β is significantly different from zero would strengthen the claim that cross-game history has little effect.
The paper says, at §3, b194–b195
Pith says: minor objection 8 of 9
The second limitation (interpretability) is acknowledged. However, the paper could briefly mention whether any post-hoc analysis of network representations was attempted (e.g., probing for known exploration signals) to partially address this.
The paper says, at References
Pith says: minor objection 9 of 9
Reference [33] (Wang & Wilson, 2026) is listed as published in PLOS Computational Biology. Verify that this is indeed published and not 'in press' or 'to appear', as the paper relies on it for the repeated-game approach.
What would settle it
Re-analyze the data including a proxy for unmeasured internal states, such as trial-by-trial reaction time or response time variability, which is available in most Horizon Task datasets. If including RT as an additional feature in the neural network models reduces the H1–H6 predictability gap by more than 2 percentage points, then the residual variability is partially attributable to a measurable but previously omitted factor, weakening the claim of genuine stochasticity.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 13 Nov 2026
- Reader holds up, with conditions
- Skeptic agrees
- Editor wrote the verdict above
- Referee major revision: 3 major objections, 9 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Short version: this is a careful negative-result paper with a stronger title than its evidence. The core finding—that bigger, more flexible models improve overall fit but fail to close the H1–H6 predictability gap—is real and worth knowing. The conclusion that random exploration is "substantially random" is a reasonable inference, not a proof, and the authors mostly say so themselves.
What's new: earlier work used repeated identical games to separate random from deterministic sources; this paper asks whether deep neural networks can recover deterministic structure that simpler models miss. On 1.3 million trials, across MLP, RNN, LSTM, GRU, and transformer-style architectures, the answer is consistently no. The analysis has real strengths: logistic baselines are evaluated out-of-sample on the same splits, subject-level and population-level fits are both reported, first-game analyses rule out learning across games, cross-game history is tested with a learned persistence parameter, and the H6 trajectory analysis shows predictability rising as the remaining horizon shortens. That convergence of evidence is the best part of the paper.
Where it's soft: the inference from absence is structurally limited, and the authors know it. b172 says the residual difference "does not by itself establish irreducible stochasticity," and b184–b189 explicitly list unmeasured deterministic factors—gaze, fidgeting, attention, arousal, boredom—that would look like noise in the current features. So the abstract's "suggesting random exploration truly is substantially random" (b8) overreaches. What the data actually support is: within the measured features, limited model expressiveness cannot close the gap. That is a meaningful constraint on deterministic explanations, but it is not a demonstration that the residual variability is genuinely stochastic. A fully satisfying version of this paper would pair the neural networks with richer measurements or a stronger falsifiable prediction about what additional features would do.
Minor issues: no code or data sharing, a single fixed train/validation/test split, and no explicit significance test on the neural-network accuracy gap across architectures. These are fixable and shouldn't block the core result.
Who this is for: cognitive scientists and neuroscientists working on explore-exploit behavior, and anyone building computational models of choice variability. It deserves a serious referee. I'd send it out, with the request that the authors make code and data available and calibrate the title and abstract to what the design can actually establish.
The strongest objection
The paper's central claim—that random exploration is 'substantially random'—depends on the inability of neural networks to close the predictability gap between H1 and H6. However, this is a negative result: the absence of evidence for structure is not evidence of absence. The authors explicitly acknowledge this limitation (sentence b184: 'A key limitation is that even highly expressive models can only capture deterministic structure that is present in the measured data.') and list unmeasured factors (eye movements, fidgeting, internal states) that could account for the residual variability (b186–b189). The claim therefore rests on an assumption that all relevant deterministic structure is captured by the measured variables (choices, rewards, subject ID, game history). This assumption is not tested and is unlikely to hold given the known influence of physiological and attentional states on choice. The paper's own discussion concedes that future work with richer measurements is needed (b190, b202). Thus, the strongest claim is not supported by the evidence presented; it is a plausible interpretation of a negative result that the authors themselves qualify.
The reader's weakest_assumption correctly identifies that the claim depends on the assumption that the neural networks capture all deterministic structure present in the data. The authors acknowledge this limitation, but it remains structurally inherent to the argument. My concern is essentially the same: the negative result cannot rule out unmeasured confounds. The concrete test I propose (including reaction time) is a feasible check that could strengthen or weaken the claim. The verdict remains CONDITIONAL because the paper's evidence is consistent with the claim but does not definitively establish it. I agree with the reader's assessment and do not recommend changing the verdict.
The referee's report
The paper addresses a fundamental question in decision neuroscience: whether behavioral variability attributed to random exploration is truly stochastic or reflects unmodeled deterministic structure. The study is large-scale (1,392 participants, >1.27M trials) and uses multiple neural network architectures to maximize the chance of capturing any deterministic patterns. The negative result—that even expressive models fail to close the predictability gap—is informative and challenges the view that apparent randomness in exploration can be fully explained by limited model capacity. The authors are transparent about limitations, including the possibility of unmeasured deterministic factors. If the result holds, it supports the notion that random exploration has a genuinely stochastic component, which has implications for theories of decision-making and neural noise.
Pith's reading of the paper (a machine's summary, not the authors')
The paper investigates whether the increased choice variability observed when people have more future opportunities to use new information (long horizon) is genuinely random or merely reflects deterministic structure that simple cognitive models fail to capture. Using neural networks as more expressive models, the authors find that while these networks capture more behavioral structure than traditional logistic models, long-horizon choices remain consistently less predictable than short-horizon choices across multiple analyses. They test several potential sources of deterministic structure—individual differences, learning across games, cross-game history—and find that none eliminate the predictability gap. The paper concludes that limited model expressiveness cannot fully account for the additional variability associated with random exploration, suggesting that random exploration truly has a substantially stochastic component. This matters because it speaks to whether apparent randomness in decision-making reflects irreducible noise or hidden structure, with implications for cognitive models, neuroscience, and AI exploration strategies.
What carries the argument. The key machinery is the comparison of predictive accuracy between short-horizon (H1) and long-horizon (H6) conditions using neural network models of varying expressiveness (MLP, RNN, LSTM, GRU, Transformer, self-attention) as function approximators. The central object is the H1–H6 predictability gap—the difference in how well choices can be predicted in each condition. The argument works by showing that if the gap were due to limited model expressiveness, more expressive models should preferentially improve prediction in H6 and reduce the gap. The persistence of the gap across architectures and analyses carries the argument that the additional variability in H6 is not recoverable deterministic structure.
Core claim. The paper's central claim is that the increased choice variability observed in long-horizon (H6) compared to short-horizon (H1) conditions in the Horizon Task cannot be fully explained by limited model expressiveness, suggesting a genuinely stochastic component of random exploration. This is established by showing that even highly expressive neural network models, which capture more behavioral structure than traditional logistic models, still leave long-horizon choices consistently less predictable than short-horizon choices. The predictability gap persists across multiple analyses: pooled population, subject-level, first-game only, and cross-game history. The paper thus argues that the additional variability associated with random exploration is not merely an artifact of model simplicity but reflects a genuinely stochastic process.
The premise it rests on. The claim that the residual unpredictability is genuinely stochastic depends on the assumption that the neural network models are sufficiently expressive to capture any deterministic structure present in the measured data, but unmeasured deterministic factors (e.g., eye movements, fidgeting, internal states) could account for the gap.
If the paper is right (Pith's inferences, not the authors')
If the central claim is correct, cognitive models of exploration must incorporate a genuinely stochastic component rather than treating apparent randomness as a modeling artifact or hidden deterministic process.
Neuroscientific theories of random exploration should focus on mechanisms that generate irreducible neural noise, such as stochastic neurotransmitter release or noisy decision thresholds, rather than only on deterministic computations.
The findings suggest that behavioral variability in explore-exploit tasks is partly irreducible, which may constrain the precision with which individual choices can be predicted, even with ideal models.
The methodology of using expressive neural networks as a probe for residual stochasticity provides a template for testing whether unexplained variance in other cognitive tasks is genuinely random or reflects unmodeled structure.
The paper's negative result—that the predictability gap persists—does not prove stochasticity but shifts the burden of proof: to demonstrate deterministic structure, future work must measure the specific unobserved variables (e.g., gaze, fidgeting, arousal) that could account for the gap.
The H1–H6 gap decreasing across trials within H6 games suggests that the stochastic component may be strategic: it is engaged when exploration is valuable and reduced when remaining opportunities shrink, consistent with a regulated noise process rather than irreducible neural noise.
If the residual variability is genuinely stochastic, then optimal exploration algorithms in AI might benefit from calibrated noise injection rather than relying solely on deterministic uncertainty-driven exploration, mirroring the human strategy.
The paper's reliance on predictive accuracy as the metric may miss deterministic structure that is not linearly separable or that manifests in choice patterns not captured by next-choice prediction (e.g., sequences of choices across trials).
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Decision making often requires balancing exploitation of familiar options with exploration of uncertain alternatives. One strategy for exploration is random exploration, in which choices become more variable and less tied to the option that currently appears best. However, it remains unclear whether this variability reflects genuine stochasticity or instead a deterministic structure that traditional cognitive models fail to capture. We examined this question using the Horizon Task in an aggregated dataset of 1,392 participants completing over 1,270,695 trials. In this task, exploration becomes more valuable when participants have more future choices, and choice variability increases under longer horizons. We tested whether more expressive neural network models could recover deterministic structure from this additional variability. Although neural networks provided better overall fits to behavior than the traditional model, long-horizon choices remained consistently less predictable than short-horizon choices. These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-11-13
The authors' main claim, as the classifier picked it out 1.00
“These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random.”
How the main claim is stated: asserted 0.64
What the abstract itself states, as probabilities: claims first 0.04 · states limitation 0.42 · human participants 0.98 · controlled comparison 0.22 · randomized assignment 0.03 · states quantitative result 0.85 · compares with prior results 0.89 · mentions code or data availability 0.02
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Decision making often requires balancing exploitation of familiar options with exploration of uncertain alternatives. | background 1.00 | 0.04 | 0.02 | asserted 0.97 |
| One strategy for exploration is random exploration, in which choices become more variable and less tied to the option that currently appears best. | background 1.00 | 0.04 | 0.02 | asserted 0.99 |
| However, it remains unclear whether this variability reflects genuine stochasticity or instead a deterministic structure that traditional cognitive models fail to capture. | gap 1.00 | 0.05 | 0.03 | unclear 0.50 |
| We examined this question using the Horizon Task in an aggregated dataset of 1,392 participants completing over 1,270,695 trials. | method 0.94 | 0.96 | 0.02 | asserted 0.96 |
| In this task, exploration becomes more valuable when participants have more future choices, and choice variability increases under longer horizons. | result 0.50 | 0.50 | 0.03 | asserted 1.00 |
| We tested whether more expressive neural network models could recover deterministic structure from this additional variability. | method 0.87 | 0.92 | 0.02 | hedged possibility 0.49 |
| Although neural networks provided better overall fits to behavior than the traditional model, long-horizon choices remained consistently less predictable than short-horizon choices. | result 1.00 | 0.95 | 0.12 | asserted 1.00 |
| These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. | interpretation 0.99 | 0.93 | 0.96 | asserted 0.64 |
What kind of paper the classifier reports this is: empirical study 1.00 · evidence from analysis of existing data 0.88
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-13: 1 request, 6,121 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (10)
Figures are the authors', reproduced from the paper.
References (54)
- [1] Wilson, R.C., Geana, A., White, J.M., Ludvig, E.A., Cohen, J.D.: Humans use directed and random exploration to solve the explore–exploit dilemma. Journal of Experimental Psychology: General143, 2074–2081 (2014) https://doi.org/10. 1037/a0038199 on Pith
- [2] Feng, S.F., Wang, S., Zarnescu, S., Wilson, R.C.: The dynamics of explore–exploit decisions reveal a signal-to-noise mechanism for random exploration. Scientific Reports11(2021) https://doi.org/10.1038/s41598-021-82530-8 on Pithdoi:10.1038/s41598-021-82530-8
- [3] Sutton, R.S., Barto, A.G.: Reinforcement learning: An introduction. IEEE Trans- actions on Neural Networks9(5), 1054–1054 (1998) https://doi.org/10.1109/ TNN.1998.712192 on Pith
- [4] Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: A survey (1996) arXiv:cs/9605103 [cs.AI] on PithPith's reviewarXiv
- [5] Bellman, R.: A problem in the sequential design of experiments. Sankhy¯ a: The Indian Journal of Statistics (1933-1960)16(3/4), 221–229 (1956) on Pith
- [6] Gittins, J.C.: Bandit processes and dynamic allocation indices. Journal of the Royal Statistical Society Series B: Statistical Methodology41(2), 148–164 (1979) on Pith
- [7] Zhang, S., Yu, A.J.: Forgetful bayes and myopic planning: Human learning and decision-making in a bandit setting. Advances in neural information processing systems26(2013) on Pith
- [8] Averbeck, B.B.: Theory of choice in bandit, information sampling and foraging tasks. PLoS computational biology11(3), 1004164 (2015) 19 on Pith
- [9] Wilson, R.C., Bonawitz, E., Costa, V.D., Ebitz, R.B.: Balancing exploration and exploitation with information and randomization. Current Opinion in Behavioral Sciences38, 49–56 (2021) https://doi.org/10.1016/J.COBEHA.2020.10.001 on Pithdoi:10.1016/j.cobeha.2020.10.001
- [10] Krebs, J.R., Kacelnik, A., Taylor, P.: Test of optimal sampling by foraging great tits. Nature275(5675), 27–31 (1978) on Pith
- [11] Watkins, C., et al.: Learning from delayed rewards. (1989) on Pith
- [12] Gershman, S.J., Tzovaras, B.G.: Dopaminergic genes are associated with both directed and random exploration. Neuropsychologia120, 97–104 (2018) on Pith
- [13] Schulz, E., Gershman, S.J.: The algorithmic architecture of exploration in the human brain. Current opinion in neurobiology55, 7–14 (2019) on Pith
- [14] Gershman, S.J.: Deconstructing the human algorithms for exploration. Cognition 173, 34–42 (2018) on Pith
- [15] Frank, M.J., Doll, B.B., Oas-Terpstra, J., Moreno, F.: Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nature neuroscience12(8), 1062–1068 (2009) on Pith
- [16] Badre, D., Doll, B.B., Long, N.M., Frank, M.J.: Rostrolateral prefrontal cortex and individual differences in uncertainty-driven exploration. Neuron73(3), 595– 607 (2012) on Pith
- [17] Cavanagh, J.F., Figueroa, C.M., Cohen, M.X., Frank, M.J.: Frontal theta reflects uncertainty and unexpectedness during exploration and exploitation. Cerebral cortex22(11), 2575–2586 (2012) on Pith
- [18] Speekenbrink, M., Konstantinidis, E.: Uncertainty and exploration in a restless bandit problem. Topics in cognitive science7(2), 351–367 (2015) on Pith
- [19] Daw, N.D., O’doherty, J.P., Dayan, P., Seymour, B., Dolan, R.J.: Cortical substrates for exploratory decisions in humans. Nature441(7095), 876–879 (2006) on Pith
- [20] Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science220(4598), 671–680 (1983) https://doi.org/10.1126/science.220.4598.671 on Pithdoi:10.1126/science.220.4598.671
show 34 more
- [21] ˇCern´ y, V.: Thermodynamical approach to the traveling salesman problem: An efficient simulation algorithm. Journal of Optimization Theory and Applications 45(1), 41–51 (1985) https://doi.org/10.1007/BF00940812 on Pithdoi:10.1007/bf00940812
- [22] Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine IntelligenceP AMI-6(6), 721–741 (1984) https://doi.org/10.1109/ TPAMI.1984.4767596 20 on Pith
- [23] Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. The Journal of Chemical Physics21(6), 1087–1092 (1953) https://doi.org/10.1063/1.1699114 on Pithdoi:10.1063/1.1699114
- [24] Hastings, W.K.: Monte carlo sampling methods using markov chains and their applications. Biometrika57(1), 97–109 (1970) https://doi.org/10.1093/biomet/ 57.1.97 on Pithdoi:10.1093/biomet/
- [25] Gelfand, A.E., Smith, A.F.M.: Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association85(410), 398–409 (1990) https://doi.org/10.1080/01621459.1990.10476213 on Pithdoi:10.1080/01621459.1990.10476213
- [26] Faisal, A.A., Selen, L.P., Wolpert, D.M.: Noise in the nervous system. Nature reviews neuroscience9(4), 292–303 (2008) on Pith
- [27] Ebitz, R.B., Albarran, E., Moore, T.: Exploration disrupts choice-predictive sig- nals and alters dynamics in prefrontal cortex. Neuron97(2), 450–4619 (2018) https://doi.org/10.1016/j.neuron.2017.12.007 on Pithdoi:10.1016/j.neuron.2017.12.007
- [28] Tervo, D.G.R., Proskurin, M., Manakov, M., Kabra, M., Vollmer, A., Branson, K., Karpova, A.Y.: Behavioral variability through stochastic choice and its gating by anterior cingulate cortex. Cell159, 21–32 (2014) https://doi.org/10.1016/j. cell.2014.08.037 on Pithdoi:10.1016/j
- [29] Budzillo, A., Duffy, A., Miller, K.E., Fairhall, A.L., Perkel, D.J.: Dopaminer- gic modulation of basal ganglia output through coupled excitation–inhibition. Proceedings of the National Academy of Sciences114(22), 5713–5718 (2017) https://doi.org/10.1073/pnas.1611146114 on Pithdoi:10.1073/pnas.1611146114
- [30] Findling, C., Skvortsova, V., Dromnelle, R., Palminteri, S., Wyart, V.: Compu- tational noise in reward-guided learning drives behavioral variability in volatile environments. Nature Neuroscience22(12), 2066–2077 (2019) https://doi.org/10. 1038/s41593-019-0518-9 on Pith
- [31] Osborne, L.C., Lisberger, S.G., Bialek, W.: A sensory source for motor variation. Nature437(7057), 412–416 (2005) on Pith
- [32] Beck, J.M., Ma, W.J., Pitkow, X., Latham, P.E., Pouget, A.: Not noisy, just wrong: The role of suboptimal inference in behavioral variability. Neuron74, 30–39 (2012) https://doi.org/10.1016/j.neuron.2012.03.016 on Pithdoi:10.1016/j.neuron.2012.03.016
- [33] Wang, S., Wilson, R.C.: Separating random and deterministic sources of com- putational noise in explore-exploit decisions. PLOS Computational Biology22, 1014026 (2026) https://doi.org/10.1371/JOURNAL.PCBI.1014026 on Pithdoi:10.1371/journal.pcbi.1014026
- [34] Xiong, H.-D., Ji-An, L., Wilson, R.C., Mattar, M.G.: Human strategy adaptation in reinforcement learning resembles policy gradient ascent. bioRxiv (2025) https: 21 //doi.org/10.1101/2025.07.28.667308 on Pithdoi:10.1101/2025.07.28.667308
- [35] Gershman, S.J.: Uncertainty and exploration. Decision6(3), 277–286 (2019) https://doi.org/10.1037/dec0000101 on Pithdoi:10.1037/dec0000101
- [36] Harms, M.B., Xu, Y., Green, C.S., Woodard, K., Wilson, R., Pollak, S.D.: The structure and development of explore-exploit decision making. Cognitive Psychology150(2024) https://doi.org/10.1016/j.cogpsych.2024.101650 on Pithdoi:10.1016/j.cogpsych.2024.101650
- [37] Phelps, C.E., Frisvold, A.E., Sreeram, M., Antoniou, V., Oliveira, L.E., Tooke, S.R., Lu, Y.Z., Lemon, A.C., Fraire, J.U., Delval, A.G., Smith, B.K., Spangenberg, D.H., Mithelavage, M.N., Ngo, M.N., Keller, E.M., Issac, L.J., Harader, S.A., Mizell, J.-M., Wang, S., Keung, W., ... on Pith
- [38] Mizell, J.M., Wang, S., Frisvold, A., Alvarado, L., Farrell-Skupny, A., Keung, W., Phelps, C.E., Sundman, M.H., Franchetti, M.K., Chou, Y., Alexander, G.E., Wilson, R.C.: Differential impacts of healthy cognitive aging on directed and random exploration. Psychology and aging39... on Pith
- [39] Sadeghiyeh, H., Wang, S., Alberhasky, M.R., Kyllo, H.M., Shenhav, A., Wil- son, R.C.: Temporal discounting correlates with directed exploration but not with random exploration. Scientific Reports10(2020) https://doi.org/10.1038/ s41598-020-60576-4 on Pith
- [40] Smith, R., Taylor, S., Wilson, R.C., Chuning, A.E., Persich, M.R., Wang, S., Killgore, W.D.S.: Lower levels of directed exploration and reflective thinking are associated with greater anxiety and depression. Frontiers in Psychiatry12(2022) https://doi.org/10.3389/fpsyt.2021.782136 on Pithdoi:10.3389/fpsyt.2021.782136
- [41] Somerville, L.H., Sasse, S.F., Garrad, M.C., Drysdale, A.T., Akar, N.A., Insel, C., Wilson, R.C.: Charting the expansion of strategic exploratory behavior during adolescence. Journal of experimental psychology. General146, 155–164 (2017) https://doi.org/10.1037/XGE0000250 on Pithdoi:10.1037/xge0000250
- [42] Waltz, J.A., Wilson, R.C., Albrecht, M.A., Frank, M.J., Gold, J.M.: Differential effects of psychotic illness on directed and random exploration. Computational psychiatry (Cambridge, Mass.)4, 18 (2020) https://doi.org/10.1162/CPSY A 00027 on Pithdoi:10.1162/cpsy
- [43] LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. nature521(7553), 436–444 (2015) on Pith
- [44] Hornik, K.: Approximation capabilities of multilayer feedforward networks. 22 Neural networks4(2), 251–257 (1991) on Pith
- [45] Cybenko, G.: Approximation by superpositions of a sigmoidal function. Math- ematics of Control, Signals and Systems2, 303–314 (1989) https://doi.org/10. 1007/BF02551274 on Pith
- [46] Peterson, J.C., Bourgin, D.D., Agrawal, M., Reichman, D., Griffiths, T.L.: Using large-scale experiments and machine learning to discover theories of human decision-making. Science372(6547), 1209–1214 (2021) https://doi.org/10.1126/ science.abe2629 on Pith
- [47] Musall, S., Kaufman, M.T., Juavinett, A.L., Gluf, S., Churchland, A.K.: Single- trial neural dynamics are dominated by richly varied movements. Nature Neuro- science22(10), 1677–1686 (2019) https://doi.org/10.1038/s41593-019-0502-4 on Pithdoi:10.1038/s41593-019-0502-4
- [48] Li, J.-A., Benna, M.K., Mattar, M.G.: Discovering cognitive strategies with tiny recurrent neural networks. Nature (2025) https://doi.org/10.1038/ s41586-025-09142-4 on Pith
- [49] Wang, S., Wilson, R.C., Ebner, N.C., Lighthall, N.R.: Modeling value integra- tion during decision making under uncertainty with the florida and georgia gambling task. Scientific Reports15, 36826 (2025) https://doi.org/10.1038/ s41598-025-08333-3 on Pith
- [50] Kingma, D.P., Ba, J.L.: Adam: A method for stochastic optimization. 3rd Inter- national Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings (2014) on Pith
- [51] Cho, K., Merri¨ enboer, B.V., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using rnn encoder-decoder for statistical machine translation. EMNLP 2014 - 2014 Conference on Empirical Methods in Natural Language Processing, Pro... on Pithdoi:10.3115/v1/d14-1179
- [52] Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural computation 9(8), 1735–1780 (1997) on Pith
- [53] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems30(2017) on Pith
- [54] Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neu- ral networks. Advances in Neural Information Processing Systems4, 3104–3112 (2014) 23 Fig. 7MLP Architecture. The model received a single concatenated vector of sequential and static inputs. Sequenti... on Pith
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment