{"id":"5473123b-4e99-4461-a24e-08287089756d","arxiv_id":"2411.09176","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A reinforcement-learning transformer model reproduces human eye movements and reward-driven foraging choices in hybrid visual search, with generalization to novel targets and values.","lead":"Researchers tracked eye movements and clicks as people searched for targets worth different amounts of reward, then built an AI that learns to search the same way by trial and error. The AI collects rewards at near-human levels, mimics human gaze patterns, and still works on objects and values it never saw during training.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The human-AI comparison mixes a shuffled and a static environment; if full-trial metrics are not restricted to pre-shuffle data, the behavioral-alignment claim is confounded.","rationale":"The reader's CONDITIONAL verdict is well-founded, and the shuffle mismatch is indeed the most load-bearing concern. The central novelty claim rests on human-model equivalence in reward and eye-movement statistics, and the human environment reshuffles item positions every 3 seconds while the AI environment does not. Since the paper's only pre-shuffle restriction appears in the qualitative figure (Appendix S3.9), all full-trial metrics in Figs. 4-5 and Appendix S3.4 may mix a dynamic-environment human strategy with a static-environment AI strategy. This is not an internal inconsistency in the model, but a validity threat to the comparison supporting the strongest claim. If re-analysis restricted to the first 3 seconds reproduces the reported results, the concern is resolved; if not, the claim must be narrowed. Other issues, such as the inconsistent training values (main text says 2,4,8,16; Appendix S1.2 says 2,4,8,12), the beta hyperparameter tuned to human saccade size, VF's lack of fixation-duration modeling, and scanpath similarity only slightly above chance, are real but secondary: they affect the precision or strength of the match, not the fairness of the comparison. The model architecture, ablations, and code release are otherwise reasonably described. Therefore the existing CONDITIONAL verdict stands without adjustment.","tokens_in":23715,"tokens_out":6282,"duration_ms":77893,"concrete_test":"Recompute Fig. 4 Norm.Score curves, Fig. 5A CBR, Fig. 5B saccade size distributions, and Fig. S4 fixation-area rewards using only human fixations and clicks with timestamp less than 3 seconds (before the first shuffle) and the corresponding VF trajectories on the static initial array. If the pre-shuffle patterns match the reported full-trial patterns, the shuffle mismatch is not a confound; if they differ materially, all full-trial human-VF comparisons must be re-run in matched environments or the behavioral claims restricted to pre-shuffle trials.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Human trials shuffle item positions every 3 seconds (Appendix S1.1), but the AI foraging environments (Sec. 3.2 and Appendix S1.2) do not include shuffling. The paper only restricts its qualitative scanpath example to pre-shuffle data (Appendix S3.9), while Figs. 4-5 and Appendix S3.4 report full-trial aggregated metrics for Norm.Score, CBR, saccade size, and fixation-area rewards. After a shuffle, humans must re-localize targets and may change both fixation and click policies; VF never faces this. Unless all quantitative human-VF comparisons are also restricted to the interval before the first shuffle, the reported match between VF and human behavior could reflect two different tasks rather than similar foraging policies. This is the most load-bearing issue because the central claim is precisely that VF approximates human gaze-and-click behavior; a known environmental difference in the only human-model comparison makes that claim unestablished as reported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a hybrid visual foraging task with varying target values and prevalence, reports a human eye-tracking experiment, and proposes Visual Forager (VF), a transformer-based reinforcement-learning agent that sequentially produces fixation locations and click decisions. The authors claim that VF outperforms all baselines, achieves cumulative rewards comparable to humans, approximates human eye-movement statistics and click biases without supervised training on human data, and generalizes to out-of-distribution conditions. The supplementary material contains implementation details, ablations, and additional analyses.","tokens_in":23898,"tokens_out":2837,"duration_ms":34342,"significance":"If the central claims hold, the paper makes a valuable contribution: it provides a large human eye-tracking dataset for hybrid visual foraging and demonstrates that an RL agent, not trained on human gaze data, can reproduce several qualitative features of human foraging, including value-guided click biases and approximate saccade-size distributions. The central Norm.Score results are clearly presented, VF consistently outperforms the considered baselines, and the authors provide code, data, and models. The generalization tests to unseen targets, values, set sizes, and item numbers are a useful stress test. However, three load-bearing issues—the shuffled versus static environment mismatch, the unsupported fixation-duration significance claim, and the calibrated saccade-size comparison—need to be resolved before the behavioral-alignment claim is fully established.","major_comments":[{"comment":"The human and AI foraging environments differ in a way that confounds the quantitative human–VF comparisons. Appendix S1.1 states that item positions in human trials are shuffled every 3 seconds, while the AI environments described in §3.2 and Appendix S1.2 do not include any shuffling. The paper restricts its qualitative scanpath example to pre-shuffle data only in Appendix S3.9, but Figures 4 and 5, Table S4, and Figures S4 and S5 report full-trial aggregated metrics such as Norm.Score, CBR, saccade size, and fixation-area rewards. After a shuffle, humans must re-localize targets and may change both fixation and click policies, whereas VF never faces this change. Unless all quantitative human–VF comparisons are also restricted to the interval before the first shuffle, or the AI environment is modified to include shuffling, the reported behavioral alignment may reflect two different tasks rather than similar foraging policies. This issue is load-bearing because the central claim is precisely that VF approximates human gaze-and-click behavior.","section":"§3.2, Appendix S1.1, Appendix S3.9"},{"comment":"The fixation-duration claim is not supported by the reported statistics. Appendix S3.3 reports p = 0.13 for UnValEqPre and p = 0.12 for UnValUnPre, yet the text states that \"fixation durations are significantly different for targets with different values\" in those conditions. With conventional thresholds, p > 0.05 indicates no significant difference. The Abstract and §5.2 rely on this claim when stating that fixation durations are longer on more valuable targets. The authors should either report a properly powered and correctly specified statistical test that supports the claim, or reframe the finding as a descriptive trend and remove it from the paper's central contributions.","section":"Appendix S3.3, §5.2, Abstract"},{"comment":"The saccade-size match is partly calibrated, so the claim that VF \"approximates the saccade size distributions of humans, without training on human eye movements\" needs qualification. Table S1 shows that the eccentricity-dependent pooling scaling factor β was varied and set to β = 1 because it yields a mean saccade size of 4.06 dva, closely matching the human value of 4.05 dva. Because β was selected on the basis of human saccade data, the favorable saccade-size comparison is not an emergent property of the RL-trained policy alone. The authors should acknowledge this parameter selection explicitly, report results across β values, or provide a principled a priori justification for β = 1.","section":"Appendix S1.4, Table S1, §5.3"}],"minor_comments":[{"comment":"There is an inconsistency in the target values used for training: §3.2 states that values are set at 2, 4, 8, and 16, while Appendix S1.2 says values are consistently set at 2, 4, 8, and 12. Please reconcile these statements.","section":"§3.2 vs Appendix S1.2"},{"comment":"The heading \"Eye movements are effected by target values\" contains a typo; it should be \"affected.\"","section":"§5.2 heading"},{"comment":"The p-values in Figure S3 are reported without specification of the statistical test used; please state the test (e.g., ANOVA, Kruskal-Wallis, mixed-effects model) and the associated sample sizes or degrees of freedom.","section":"Appendix S3.3"},{"comment":"The description of OOD - USetSize says \"the set size of target objects was manipulated to include either one or two,\" but the name and surrounding text are ambiguous about whether this refers to the number of target types or the number of instances per type. Please clarify.","section":"Appendix S1.3"},{"comment":"The intrinsic blank-click penalty is introduced with a single value of -0.01, and Table S5 shows that the Norm.Score with this penalty is 72.6 while the model with no penalty achieves 79.9. The text says the penalty \"has no significant effect,\" but the difference of 7.3 percentage points is not negligible; please provide a significance test or a discussion of why this difference is considered non-significant.","section":"§4.4 and Appendix S3.7"}],"recommendation":"major_revision","confidential_remarks":"The manuscript has a concrete and potentially valuable core—the RL-based Visual Forager and its systematic comparison against human data—but the current version contains three issues that affect the main claim: the shuffled-versus-static environment mismatch, the misinterpreted fixation-duration p-values, and the calibration of β on human saccade data. These are fixable within the manuscript's scope, so I recommend major revision rather than rejection. I would also encourage the authors to double-check whether all statements in the abstract and main text are consistent with the supplementary statistics, as the current text overstates the fixation-duration finding."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core contribution here is real: a new human eye-tracking dataset for hybrid visual foraging with varying target values and prevalence, plus a transformer-based RL model, VF, that outputs both fixation and click policies without any supervised human data. The Norm.Score results are clearly presented and VF consistently beats the baselines and approaches human cumulative reward. The OOD generalization tests are a nice extra, and the paper is honest about several limitations in the appendix.\n\nThe soft spots are real but mostly fixable. The most serious is the shuffle mismatch. Human trials reshuffle item positions every three seconds; the AI environments never shuffle. Only the qualitative scanpath example is restricted to pre-shuffle data. The full-trial aggregated comparisons in Figures 4 and 5 and Appendix S3.4 are not. If humans change their fixation and click strategies after a shuffle, the quantitative human-VF match could reflect two different tasks. This needs to be addressed directly, either by restricting all behavioral comparisons to the pre-shuffle window or by adding shuffling to the AI environments.\n\nThe saccade-size match is also partly calibrated. Appendix Table S1 shows beta, the eccentricity-dependent pooling scaling factor, was varied and set to 1 because it best matched human saccade sizes. That means the match is not a pure prediction. The paper should be upfront that this one parameter is fit, not free.\n\nThe fixation-duration claim is overstated. Appendix Figure S3 reports p = 0.13 and p = 0.12 for the two conditions, yet the text calls the differences significant. They are not. Either the analysis or the wording needs correction. There is also an inconsistency: the main text says training values are 2, 4, 8, 16, while Appendix S1.2 says 2, 4, 8, 12. And several key behavioral plots appear to lack error bars.\n\nNone of this breaks the central model claim. VF is a serious new benchmark tool and the dataset is a genuine resource. The paper deserves a proper peer review, but it needs revision on the points above before the behavioral-alignment claims can be taken at face value. I would bring it to a reading group and would cite it if I worked on visual search or value-guided attention.","headline":"A valuable new dataset and a credible RL model for hybrid visual foraging, but the behavioral-alignment claims are weakened by a shuffled-versus-static comparison, a tuned saccade parameter, and a mislabeled p-value.","tokens_in":24446,"tokens_out":2406,"would_cite":true,"duration_ms":29531,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A reinforcement-learning agent reproduces human gaze and click decisions in hybrid visual foraging without any training on human data.","keywords":["hybrid visual foraging","eye movements","reinforcement learning","value-based decision making","foveated vision","visual search","transformer","scanpath prediction"],"falsifier":"Run VF in a foraging environment where item positions are reshuffled every three seconds, exactly as in the human experiment, keeping all reward and time-cost parameters equal. If VF's normalized scores, click-bias ratios, or saccade-size distributions no longer track the human data under this shuffling, the central claim that VF approximates human foraging behavior without human data is falsified.","tokens_in":23510,"feed_emoji":"👀","tokens_out":5417,"duration_ms":147852,"temperature":0.7,"pith_summary":"In hybrid visual foraging—searching a cluttered display for multiple instances of several target types that carry different rewards—humans behave as efficient reward foragers: their fixations land in high-reward regions, they linger longer on more valuable targets, and their cumulative rewards approach an optimal upper bound. This paper asks whether an artificial agent can acquire the same gaze-and-click policy purely from the reward structure of the task, without any supervised training on human eye or mouse data. The authors introduce Visual Forager (VF), a transformer-based reinforcement-learning agent with foveated vision, and report that it outperforms all tested baselines, achieves rewards comparable to human participants, and approximates human click biases and saccade-size distributions. If the claim holds, VF is the first model to reproduce human hybrid-foraging behavior—including where the eyes go—from reward maximization alone, and it offers a computational tool for probing the link between oculomotor decisions and value-based choice.","feed_headline":"Reward-trained AI matches human gaze in hybrid foraging","feed_subtitle":"A value-driven transformer learns to fixate and click like a human forager, with no training on human gaze data.","key_machinery":"The load-bearing mechanism is the Visual Forager's value-modulated, foveated policy network. A VGG16 backbone with eccentricity-dependent pooling (replicated from macaque neurophysiology) converts the search image into fixation-dependent feature maps; target features are matched to those maps by convolution to form similarity maps; a learnable value encoder's embedding is added at every spatial location, so that the value of each target type directly reshapes the priority map; and a vision transformer over these value-modulated maps outputs a softmax fixation map plus a click probability, trained jointly by PPO with a two-stage curriculum (first fixations with full resolution and infinite inhibition of return, then clicks with eccentricity-dependent vision and a decaying memory of visited locations). This design lets value and visual similarity interact inside the policy rather than being combined heuristically.","core_discovery":"The central claim is that a value-guided reinforcement learner, with no human behavioral data, can converge on the same foraging policy that humans use when target values and prevalence vary. Specifically, VF reads in a search image, the target templates, and their monetary values; a foveated feature extractor, with eccentricity-dependent pooling modelled on the primate visual system, produces fixation-dependent similarity maps; a value encoder adds target values into those maps; and a transformer actor-critic decides both where to fixate next and whether to click the currently fixated item. Trained with PPO under a curriculum, VF is reported to reach 72.6%, 67.1%, and 81.6% normalized scores in the three in-domain conditions (humans: 87.4%, 84.1%, 93.1%), to share the sign of human click-bias ratios for high- and low-valued targets, and to produce a mean saccade size of 4.06 degrees versus 4.05 degrees for humans, despite zero training on human eye movements. The paper interprets this as evidence that eye-movement policies in hybrid foraging are largely learned from reward structure, and that fixation decisions are a reliable lens on value-based decision-making.","pith_inferences":["A direct test of the model's causal claim would be to remove target values from the input: the paper's analysis predicts fixations and click biases should then flatten toward chance, offering a falsifiable dissociation between feature-driven and value-driven guidance.","The human experiment shuffled item positions every three seconds while the AI environments did not, so adding the same periodic shuffle to VF's environment would test whether the reported human–AI alignment persists under working-memory load; this is an extension the paper does not run.","The VF architecture effectively produces a dynamic priority map that is updated by value after each fixation; that map could be repurposed as a generative model of scanpaths in other multi-target tasks, or as a synthetic data source for training models of attention.","The paper's finding that fixation durations scale with target value suggests a natural extension: an augmented VF that predicts fixation durations, rather than only locations, would let the model speak directly to the time-cost trade-offs humans face in foraging."],"forward_implications":["VF achieves human-comparable cumulative rewards and closer-to-human scanpaths than all baselines in time-limited hybrid foraging, suggesting that reward maximization alone can induce human-like search priorities.","VF reproduces the human overpicking of high-value targets and underpicking of low-value targets, so value-driven click biases need not be trained from human examples.","The saccade-size match implies that human-like fixation step lengths emerge partly from foveated processing rather than from explicit oculomotor constraints.","VF generalizes to out-of-distribution conditions—unseen target objects, unseen value ranges, different item counts and set sizes—better than the strongest baseline, indicating that the learned value-modulation policy is transferable.","Eye fixation locations and durations carry measurable signatures of reward, giving a non-invasive readout of subjective value during search."],"supporting_citations":[{"why":"Supplies the pool of 2,400 unique item images used to build the human and AI foraging arrays.","marker":"[14]"},{"why":"Provides the eccentricity-dependent pooling and feature-similarity mechanism that gives VF its foveated, fixation-dependent vision.","marker":"[51]"},{"why":"The PPO algorithm used to train the fixation and click policies.","marker":"[109]"},{"why":"The DQN baseline and the deep-Q-learning setup that VF is compared against.","marker":"[92]"},{"why":"Source of the human hybrid-value foraging finding that people overpick high-value targets, used to define the CBR comparison.","marker":"[138]"},{"why":"Supplies the GazeFormer baseline and the FED scanpath-similarity metric used to compare fixation sequences.","marker":"[93]"},{"why":"Source of the zero-shot invariant visual search model whose similarity-map computation and IVSN baseline VF extends.","marker":"[145]"},{"why":"The vision-transformer architecture used as the actor-critic's decision backbone.","marker":"[33]"}],"fun_headline_variants":["Reward-trained AI matches human gaze in foraging","AI learns to forage by reward alone, eye movements follow","Value-driven transformer mimics human visual foraging","No gaze data, yet AI fixates like a human forager","Reward-based AI learns human-like eye movement strategy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The match between humans and VF assumes that the simplified grid world with fixed item positions and constant per-click and per-fixation time costs faithfully captures human foraging strategy, even though the human experiment shuffled item positions every three seconds and the AI environments did not.","fun_headline_variants_meta":{"raw":{"variants":["Reward-trained AI matches human gaze in foraging","AI learns to forage by reward alone, eye movements follow","Value-driven transformer mimics human visual foraging","No gaze data, yet AI fixates like a human forager","Reward-based AI learns human-like eye movement strategy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000325,"raw_usage":{"total_tokens":1908,"prompt_tokens":1115,"completion_tokens":793,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":731,"completion_tokens_details":{"reasoning_tokens":717}},"tokens_in":731,"tokens_out":793,"duration_ms":9977,"temperature":1.0,"reasoning_tokens":717,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:56:08.321324+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run VF in a foraging environment where item positions are reshuffled every three seconds, exactly as in the human experiment, keeping all reward and time-cost parameters equal. If VF's normalized scores, click-bias ratios, or saccade-size distributions no longer track the human data under this shuffling, the central claim that VF approximates human foraging behavior without human data is falsified.","supporting_citations":[{"cited_title":"Human-level control through deep reinforcement learning","cited_arxiv_id":null,"evidence_quote":"The DQN baseline and the deep-Q-learning setup that VF is compared against."},{"cited_title":"Hybrid value foraging: How the value of targets shapes human foraging behavior","cited_arxiv_id":null,"evidence_quote":"Source of the human hybrid-value foraging finding that people overpick high-value targets, used to define the CBR comparison."},{"cited_title":"Gazeformer: Scalable, effective and fast prediction of goal-directed human attention","cited_arxiv_id":null,"evidence_quote":"Supplies the GazeFormer baseline and the FED scanpath-similarity metric used to compare fixation sequences."},{"cited_title":"Finding any waldo with zero-shot invariant and efficient visual search","cited_arxiv_id":null,"evidence_quote":"Source of the zero-shot invariant visual search model whose similarity-map computation and IVSN baseline VF extends."}],"review_version":1}