{"id":"f7b8f7f1-d4e9-4f80-b294-867994082576","arxiv_id":"2611.17011","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Neural network models cannot fully predict the increased choice variability in long-horizon exploration, supporting a genuinely stochastic component of random exploration.","lead":"This paper tests whether the increased choice variability people show when they have many future opportunities (long horizon) is truly random or just looks random because simple models miss hidden patterns. Using neural networks on over a million trials, the authors find that even powerful models cannot predict long-horizon choices as well as short-horizon ones, suggesting a genuinely random component to exploration.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"The central claim rests on a negative result that cannot rule out unmeasured deterministic factors, as the authors themselves acknowledge.","rationale":"The reader's weakest_assumption correctly identifies that the claim depends on the assumption that the neural networks capture all deterministic structure present in the data. The authors acknowledge this limitation, but it remains structurally inherent to the argument. My concern is essentially the same: the negative result cannot rule out unmeasured confounds. The concrete test I propose (including reaction time) is a feasible check that could strengthen or weaken the claim. The verdict remains CONDITIONAL because the paper's evidence is consistent with the claim but does not definitively establish it. I agree with the reader's assessment and do not recommend changing the verdict.","tokens_in":16827,"tokens_out":514,"duration_ms":7383,"concrete_test":"Re-analyze the data including a proxy for unmeasured internal states, such as trial-by-trial reaction time or response time variability, which is available in most Horizon Task datasets. If including RT as an additional feature in the neural network models reduces the H1–H6 predictability gap by more than 2 percentage points, then the residual variability is partially attributable to a measurable but previously omitted factor, weakening the claim of genuine stochasticity.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim—that random exploration is 'substantially random'—depends on the inability of neural networks to close the predictability gap between H1 and H6. However, this is a negative result: the absence of evidence for structure is not evidence of absence. The authors explicitly acknowledge this limitation (sentence b184: 'A key limitation is that even highly expressive models can only capture deterministic structure that is present in the measured data.') and list unmeasured factors (eye movements, fidgeting, internal states) that could account for the residual variability (b186–b189). The claim therefore rests on an assumption that all relevant deterministic structure is captured by the measured variables (choices, rewards, subject ID, game history). This assumption is not tested and is unlikely to hold given the known influence of physiological and attentional states on choice. The paper's own discussion concedes that future work with richer measurements is needed (b190, b202). Thus, the strongest claim is not supported by the evidence presented; it is a plausible interpretation of a negative result that the authors themselves qualify.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"The paper investigates whether the increased choice variability observed in long-horizon (H6) compared to short-horizon (H1) conditions in the Horizon Task reflects genuine stochasticity or deterministic structure missed by traditional cognitive models. Using neural network models with varying expressiveness, the authors find that while these models capture more behavioral structure than a logistic baseline, long-horizon choices remain consistently less predictable than short-horizon choices across multiple analyses (global, subject-level, first-game, cross-game history, within-H6 trial analysis). They conclude that limited model expressiveness cannot fully account for the increased variability, suggesting a genuinely stochastic component of random exploration.","tokens_in":17140,"tokens_out":1630,"duration_ms":12506,"significance":"The paper addresses a fundamental question in decision neuroscience: whether behavioral variability attributed to random exploration is truly stochastic or reflects unmodeled deterministic structure. The study is large-scale (1,392 participants, >1.27M trials) and uses multiple neural network architectures to maximize the chance of capturing any deterministic patterns. The negative result—that even expressive models fail to close the predictability gap—is informative and challenges the view that apparent randomness in exploration can be fully explained by limited model capacity. The authors are transparent about limitations, including the possibility of unmeasured deterministic factors. If the result holds, it supports the notion that random exploration has a genuinely stochastic component, which has implications for theories of decision-making and neural noise.","major_comments":[{"comment":"The central claim rests on the inability of neural networks to close the H1–H6 predictability gap. However, the authors acknowledge in §3 (b184–b189) that unmeasured factors (e.g., eye movements, fidgeting, internal states) could account for the residual variability. This is a structural limitation of the negative result: the absence of evidence for structure is not evidence of absence. The paper's strongest claim (b8) that 'random exploration truly is substantially random' goes beyond what the data can support given this acknowledged limitation. The authors should either soften the conclusion to match the evidence (e.g., 'consistent with a stochastic component') or provide a stronger argument that the measured variables capture the relevant deterministic structure. As written, the conclusion overstates the strength of the negative result.","section":"§2.5, Fig. 3","authors_acknowledge":"If similar unmeasured factors influence exploratory choices, their effects would appear as unexplained variability in our models even if they were deterministic.","authors_acknowledge_p":0.94},{"comment":"The H1–H6 accuracy gap in the subject-level analysis is 6.0 percentage points (b122), and in the first-game analysis it is 7.5 percentage points (b130). The logistic baseline gaps are 6.1 and 11.3 percentage points respectively. While the gap narrows in some comparisons, it is not statistically tested whether the reduction is significant. The paper reports confidence intervals for individual accuracies but does not provide a formal test (e.g., interaction between horizon and model type) to assess whether the neural networks significantly reduce the gap relative to the baseline. Without such a test, the claim that 'limited model expressiveness cannot fully account' is weakened because the gap reduction might be within chance. The authors should report a statistical comparison (e.g., bootstrap test of the gap difference) to support their interpretation.","section":"§2.5, Fig. 3; §2.6, Fig. 4","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The within-H6 trial analysis shows that predictability increases as the remaining horizon shortens, which is consistent with the value of future exploration driving variability. However, the final H6 trial accuracy remains below the H1 benchmark (Fig. 6). The authors do not discuss whether this residual gap could be due to the different number of free-choice trials (1 vs. 6) affecting the model's ability to predict, or to other factors like fatigue or attention decrement over longer games. A more detailed analysis of the final trial, controlling for the number of preceding choices, would strengthen the interpretation that the gap is specifically due to horizon-related exploration rather than other confounds.","section":"§2.8, Fig. 6","authors_acknowledge":null,"authors_acknowledge_p":null}],"minor_comments":[{"comment":"The phrase 'substantially random' is vague. Consider rephrasing to 'consistent with a stochastic component' or 'cannot be fully explained by deterministic structure' to match the paper's own cautious language in the discussion.","section":"Abstract","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The first-game fit shows a larger H1–H6 gap (11.3 pp) than the global fit (7.5 pp). The authors note this but do not discuss why the gap is larger in the first game. A brief comment would be helpful.","section":"§2.3, Table 1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The pseudo-R² values are reported for subject-level models. It would be useful to also report the global model pseudo-R² for comparison, as the global analysis is the simplest test of the hypothesis.","section":"§2.4, Table 2","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The data splitting procedure for subject-level models (80/10/10 within each participant) is described. However, it is not clear whether the test set contains games from participants seen during training (which is the case for subject-level models) or entirely unseen participants (which would be a stricter generalization test). Clarify this distinction.","section":"§4.4.1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The early stopping patience is five epochs. Given the large dataset, this seems reasonable, but the authors should report the average number of epochs trained to ensure convergence was achieved.","section":"§4.4.2","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The error bars are 95% Wilson confidence intervals. It would be helpful to state in the caption whether these are computed across participants or across test trials. For the global model, it is presumably across trials; for subject-level models, it could be across participants. Clarify.","section":"Fig. 3, Fig. 4, Fig. 5","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The cross-game history analysis uses a persistence parameter β that is learned. The authors report low β values, but do not provide confidence intervals or tests against zero. A brief note on whether β is significantly different from zero would strengthen the claim that cross-game history has little effect.","section":"§2.7, Fig. 5","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The second limitation (interpretability) is acknowledged. However, the paper could briefly mention whether any post-hoc analysis of network representations was attempted (e.g., probing for known exploration signals) to partially address this.","section":"§3, b194–b195","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Reference [33] (Wang & Wilson, 2026) is listed as published in PLOS Computational Biology. Verify that this is indeed published and not 'in press' or 'to appear', as the paper relies on it for the repeated-game approach.","section":"References","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"Short version: this is a careful negative-result paper with a stronger title than its evidence. The core finding—that bigger, more flexible models improve overall fit but fail to close the H1–H6 predictability gap—is real and worth knowing. The conclusion that random exploration is \"substantially random\" is a reasonable inference, not a proof, and the authors mostly say so themselves.\n\nWhat's new: earlier work used repeated identical games to separate random from deterministic sources; this paper asks whether deep neural networks can recover deterministic structure that simpler models miss. On 1.3 million trials, across MLP, RNN, LSTM, GRU, and transformer-style architectures, the answer is consistently no. The analysis has real strengths: logistic baselines are evaluated out-of-sample on the same splits, subject-level and population-level fits are both reported, first-game analyses rule out learning across games, cross-game history is tested with a learned persistence parameter, and the H6 trajectory analysis shows predictability rising as the remaining horizon shortens. That convergence of evidence is the best part of the paper.\n\nWhere it's soft: the inference from absence is structurally limited, and the authors know it. b172 says the residual difference \"does not by itself establish irreducible stochasticity,\" and b184–b189 explicitly list unmeasured deterministic factors—gaze, fidgeting, attention, arousal, boredom—that would look like noise in the current features. So the abstract's \"suggesting random exploration truly is substantially random\" (b8) overreaches. What the data actually support is: within the measured features, limited model expressiveness cannot close the gap. That is a meaningful constraint on deterministic explanations, but it is not a demonstration that the residual variability is genuinely stochastic. A fully satisfying version of this paper would pair the neural networks with richer measurements or a stronger falsifiable prediction about what additional features would do.\n\nMinor issues: no code or data sharing, a single fixed train/validation/test split, and no explicit significance test on the neural-network accuracy gap across architectures. These are fixable and shouldn't block the core result.\n\nWho this is for: cognitive scientists and neuroscientists working on explore-exploit behavior, and anyone building computational models of choice variability. It deserves a serious referee. I'd send it out, with the request that the authors make code and data available and calibrate the title and abstract to what the design can actually establish.","headline":"A careful negative-result study with a stronger title than its evidence: flexible models improve fit but don't close the short- versus long-horizon predictability gap, which is a meaningful result even if 'truly substantially random' overstates it.","tokens_in":17669,"tokens_out":1937,"duration_ms":20692},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"The paper claims that long-horizon choice variability in the Horizon Task cannot be fully explained by limited model expressiveness, suggesting a genuinely stochastic component of random exploration.","keywords":["random exploration","explore-exploit dilemma","decision noise","neural networks","Horizon Task","behavioral variability","stochasticity"],"falsifier":"If future studies measuring richer behavioral and physiological signals (e.g., eye tracking, motion tracking, pupil dilation, heart rate) alongside Horizon Task choices were able to eliminate the H1–H6 predictability gap using models that incorporate these signals, the claim that random exploration is substantially random would be falsified.","tokens_in":18316,"feed_emoji":"🎲","tokens_out":1435,"duration_ms":11390,"temperature":0.7,"pith_summary":"The paper investigates whether the increased choice variability observed when people have more future opportunities to use new information (long horizon) is genuinely random or merely reflects deterministic structure that simple cognitive models fail to capture. Using neural networks as more expressive models, the authors find that while these networks capture more behavioral structure than traditional logistic models, long-horizon choices remain consistently less predictable than short-horizon choices across multiple analyses. They test several potential sources of deterministic structure—individual differences, learning across games, cross-game history—and find that none eliminate the predictability gap. The paper concludes that limited model expressiveness cannot fully account for the additional variability associated with random exploration, suggesting that random exploration truly has a substantially stochastic component. This matters because it speaks to whether apparent randomness in decision-making reflects irreducible noise or hidden structure, with implications for cognitive models, neuroscience, and AI exploration strategies.","feed_headline":"Random exploration may truly be random, not just unmodeled","feed_subtitle":"Neural networks capture more structure but cannot explain why long-horizon choices are less predictable.","key_machinery":"The key machinery is the comparison of predictive accuracy between short-horizon (H1) and long-horizon (H6) conditions using neural network models of varying expressiveness (MLP, RNN, LSTM, GRU, Transformer, self-attention) as function approximators. The central object is the H1–H6 predictability gap—the difference in how well choices can be predicted in each condition. The argument works by showing that if the gap were due to limited model expressiveness, more expressive models should preferentially improve prediction in H6 and reduce the gap. The persistence of the gap across architectures and analyses carries the argument that the additional variability in H6 is not recoverable deterministic structure.","core_discovery":"The paper's central claim is that the increased choice variability observed in long-horizon (H6) compared to short-horizon (H1) conditions in the Horizon Task cannot be fully explained by limited model expressiveness, suggesting a genuinely stochastic component of random exploration. This is established by showing that even highly expressive neural network models, which capture more behavioral structure than traditional logistic models, still leave long-horizon choices consistently less predictable than short-horizon choices. The predictability gap persists across multiple analyses: pooled population, subject-level, first-game only, and cross-game history. The paper thus argues that the additional variability associated with random exploration is not merely an artifact of model simplicity but reflects a genuinely stochastic process.","pith_inferences":["The paper's negative result—that the predictability gap persists—does not prove stochasticity but shifts the burden of proof: to demonstrate deterministic structure, future work must measure the specific unobserved variables (e.g., gaze, fidgeting, arousal) that could account for the gap.","The H1–H6 gap decreasing across trials within H6 games suggests that the stochastic component may be strategic: it is engaged when exploration is valuable and reduced when remaining opportunities shrink, consistent with a regulated noise process rather than irreducible neural noise.","If the residual variability is genuinely stochastic, then optimal exploration algorithms in AI might benefit from calibrated noise injection rather than relying solely on deterministic uncertainty-driven exploration, mirroring the human strategy.","The paper's reliance on predictive accuracy as the metric may miss deterministic structure that is not linearly separable or that manifests in choice patterns not captured by next-choice prediction (e.g., sequences of choices across trials)."],"forward_implications":["If the central claim is correct, cognitive models of exploration must incorporate a genuinely stochastic component rather than treating apparent randomness as a modeling artifact or hidden deterministic process.","Neuroscientific theories of random exploration should focus on mechanisms that generate irreducible neural noise, such as stochastic neurotransmitter release or noisy decision thresholds, rather than only on deterministic computations.","The findings suggest that behavioral variability in explore-exploit tasks is partly irreducible, which may constrain the precision with which individual choices can be predicted, even with ideal models.","The methodology of using expressive neural networks as a probe for residual stochasticity provides a template for testing whether unexplained variance in other cognitive tasks is genuinely random or reflects unmodeled structure."],"supporting_citations":[{"why":"Introduces the Horizon Task and the baseline logistic model that attributes behavioral variability to decision noise; the paper extends this work by testing whether that noise is genuinely stochastic.","marker":"[1]"},{"why":"Previous work using repeated identical games to separate random and deterministic sources of variability; the paper addresses limitations of that approach by testing additional sources of structure.","marker":"[33]"},{"why":"Shows that behavior changes across repeated games, motivating the paper's analyses of first-game and cross-game history effects.","marker":"[34]"},{"why":"Neuroscientific evidence that neural variability increases during random exploration and choice-predictive signals weaken in prefrontal cortex, providing a biological basis for the claim of genuinely stochastic behavior.","marker":"[27]"},{"why":"Methodological precedent for using neural networks to discover psychological structure from behavior; the paper adapts this approach to test for residual stochasticity.","marker":"[46]"},{"why":"Inspiration for the TinyGRU architecture and the approach of using small recurrent networks to discover compact latent strategies; the paper uses this to establish a lower bound on representational capacity.","marker":"[48]"}],"fun_headline_variants":[],"cache_read_input_tokens":0,"weakest_assumption_plain":"The claim that the residual unpredictability is genuinely stochastic depends on the assumption that the neural network models are sufficiently expressive to capture any deterministic structure present in the measured data, but unmeasured deterministic factors (e.g., eye movements, fidgeting, internal states) could account for the gap.","fun_headline_variants_meta":{"raw":{"variants":[]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":6.5e-05,"raw_usage":{"cost":6.50832e-05,"is_byok":false,"cost_details":{"upstream_inference_cost":6.50832e-05,"upstream_inference_prompt_cost":6.47192e-05,"upstream_inference_completions_cost":3.64e-07},"total_tokens":918,"prompt_tokens":916,"completion_tokens":2,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":256,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":0}},"tokens_in":916,"tokens_out":2,"cost_source":"openrouter_usage","duration_ms":1544,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":0,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-11-13T07:31:47.645025+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"If future studies measuring richer behavioral and physiological signals (e.g., eye tracking, motion tracking, pupil dilation, heart rate) alongside Horizon Task choices were able to eliminate the H1–H6 predictability gap using models that incorporate these signals, the claim that random exploration is substantially random would be falsified.","supporting_citations":[{"cited_title":"Journal of Experimental Psychology: General143, 2074–2081 (2014) https://doi.org/10","cited_arxiv_id":null,"evidence_quote":"Introduces the Horizon Task and the baseline logistic model that attributes behavioral variability to decision noise; the paper extends this work by testing whether that noise is genuinely stochastic."},{"cited_title":"PLOS Computational Biology22, 1014026 (2026) https://doi.org/10.1371/JOURNAL.PCBI.1014026","cited_arxiv_id":null,"evidence_quote":"Previous work using repeated identical games to separate random and deterministic sources of variability; the paper addresses limitations of that approach by testing additional sources of structure."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows that behavior changes across repeated games, motivating the paper's analyses of first-game and cross-game history effects."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Neuroscientific evidence that neural variability increases during random exploration and choice-predictive signals weaken in prefrontal cortex, providing a biological basis for the claim of genuinely stochastic behavior."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Methodological precedent for using neural networks to discover psychological structure from behavior; the paper adapts this approach to test for residual stochasticity."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Inspiration for the TinyGRU architecture and the approach of using small recurrent networks to discover compact latent strategies; the paper uses this to establish a lower bound on representational capacity."}],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}