{"id":"a682b796-5c63-4071-bc6b-954e16bada34","arxiv_id":"2507.23021","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"ScanDiff generates diverse, text-conditioned gaze scanpaths with a diffusion-ViT architecture and reports state-of-the-art results on three benchmarks.","lead":"ScanDiff is a new gaze-prediction model that combines diffusion models with vision transformers to generate multiple plausible scanpaths from an image and an optional text instruction. The authors report better accuracy and diversity than prior methods on free-viewing and visual search benchmarks, and they introduce a new diversity metric.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Diversity comparison in Table 5 does not control for the number of generated scanpaths per image, so the reported RSS/DSS gains may reflect sampling effort rather than better modeling of variability.","rationale":"The reader's verdict of CONDITIONAL is appropriate. The reader flagged the self-defined DSS metric and the TPP-Gaze KL protocol as the weakest assumption. My review identifies a more concrete and more general confound: the diversity evaluation in Table 5 does not control for the number of generated scanpaths per image. RSS is monotonically non-decreasing with sample count, and DSS is also sensitive to it, so the reported diversity advantage may not reflect genuine modeling of human variability. This concern can be settled by a simple controlled experiment. I also note a secondary overclaim: the text in Sec. 4.2 states that ScanDiff achieves state-of-the-art performance across all metrics in the target-present condition of COCO-Search18, but in Table 2 GazeXplain (retrained, gray row) has a better SM w/ Dur value (0.010 vs 0.037). This does not invalidate the main claim, but it should be corrected. The diversity confound, however, is the load-bearing issue because it directly targets the claimed novelty. The recommended verdict remains CONDITIONAL, as the paper should be accepted only after the sample-count issue is addressed and the results are shown to be robust at matched sample sizes.","tokens_in":19497,"tokens_out":8140,"duration_ms":83794,"concrete_test":"Rerun the variability analysis of Table 5 using exactly K generated scanpaths per image for every model, with K equal to the number of human observers (or another fixed constant), and report the RSS threshold used. If ScanDiff remains best at matched K, the diversity claim survives; if its advantage shrinks, vanishes, or reverses, the reported diversity gains are an artifact of generating more trajectories. In addition, report the number of generated scanpaths used for the KL-based comparisons in Tables 1 and 2.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central novelty is that ScanDiff generates more diverse scanpaths (Abstract, Sec. 4.4). This claim rests on Table 5, which reports RSS and DSS. RSS, adopted from HAT [69], counts a human scanpath as covered if its SS with at least one generated scanpath exceeds a threshold; RSS is therefore non-decreasing in the number of generated scanpaths per stimulus. DSS (Eq. 8) divides SS(sg,sh) by 1+|SS(sg,sg)-SS(sh,sh)|; as the number of generated samples grows, the intra-set similarity SS(sg,sg) typically decreases, which reduces the denominator when human scanpaths are themselves diverse and tends to inflate DSS. The manuscript never states how many scanpaths were generated per image for any model, either in Table 5 or in the KL-protocol results of Tables 1-2. ScanDiff is a stochastic diffusion model that can produce arbitrarily many samples, while several baselines (e.g., Gazeformer, HAT) are deterministic or near-deterministic. Without matching the number of generated samples across models, the large RSS/DSS improvements reported in Table 5 could be an artifact of sampling effort rather than evidence of better modeling of human gaze variability. This confound affects the established RSS metric as well as the newly introduced DSS, making it more load-bearing than the lack of validation of DSS alone.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ScanDiff, an encoder-only Transformer combined with a diffusion process for scanpath generation. Fixations are embedded, corrupted by Gaussian noise, and denoised under conditioning from a DINOv2 visual backbone and a CLIP text encoder that encodes the viewing task; a length-prediction head allows variable-length outputs. The training loss combines a simplified variational lower bound, reconstruction, validity prediction, and a prior-alignment term. The authors evaluate on COCO-FreeView, MIT1003, COCO-Search18, and zero-shot OSIE, using the KL-divergence protocol of TPP-Gaze with MultiMatch, ScanMatch, Sequence Score, and Semantic Sequence Score, and add a scanpath variability analysis based on Recall Sequence Score (RSS) and a newly introduced Diversity-aware Sequence Score (DSS). The paper claims state-of-the-art accuracy on several metrics and, centrally, that the stochastic diffusion formulation produces more diverse scanpaths than prior models. The central diversity claim relies on Table 5 and the new DSS metric, and the paper acknowledges in Sec. 4.4 that this variability analysis is a first attempt.","tokens_in":19788,"tokens_out":7865,"duration_ms":101031,"significance":"If the diversity claim were fully supported, ScanDiff would be a useful contribution: it offers a unified free-viewing/visual-search model with variable-length generation, uses a reasonable multimodal conditioning design, retrains several baselines under identical settings, reports zero-shot results on OSIE, and provides a public project page. The traditional metric results, especially on COCO-Search18, are strong and well aligned with the main architecture story. However, the load-bearing 'more diverse' claim currently rests on a metric introduced in this paper (DSS) and on RSS, and neither is shown to be invariant to the number of generated scanpaths per stimulus. Because the paper itself says that conventional metrics reward averaged behavior, the diversity analysis is central to the contribution, and the current evidence is not yet conclusive. The self-evaluation protocol from the authors' TPP-Gaze is a further reason to require a robustness analysis, though it is not by itself an error.","major_comments":[{"comment":"The central diversity claim is confounded by the uncontrolled number of generated scanpaths per stimulus. As defined in Sec. 4.1, RSS counts a human scanpath as covered if its SS with at least one generated scanpath exceeds a threshold, so RSS is non-decreasing when more samples are drawn. DSS in Eq. 8 also depends on the size of the generated set through SS(sg, sg), an average over within-set pairs that generally changes with sample count. The manuscript does not state how many scanpaths were generated per image for ScanDiff or for any baseline in Table 5, nor for the KL-protocol results in Tables 1-2. Since ScanDiff is stochastic and can produce an arbitrary number of samples, while several baselines (e.g., Gazeformer, HAT) are deterministic, the large RSS and DSS gains could reflect sampling effort rather than better modeling of human gaze variability. The authors should match the number of samples across methods, report RSS/DSS as a function of the number of samples, or use a sample-size-invariant diversity measure.","section":"Sec. 4.4, Table 5; Sec. 3.3"},{"comment":"Eq. (5) is presented as a simplification of the variational lower bound, citing [8], but no derivation is given and the form is not standard: the loss sums the squared error between z0 and the model prediction at every timestep with uniform weight, with no explicit dependence on the noise level. Because this objective is the main training signal for the denoising network, readers cannot verify whether the simplified loss corresponds to the stated VLB or whether the reported performance depends on an ad hoc reweighting. Please include the derivation (or a precise reference with equation numbers) and state the role of the importance-sampling scheme.","section":"Sec. 3.3, Eq. (5)"},{"comment":"The newly introduced DSS metric is used as the main quantitative evidence for the diversity claim, but it is not validated against human judgments or against a synthetic set with known diversity. As written, DSS rewards a generated set whose within-set similarity matches the human within-set similarity, but it is not shown that this criterion corresponds to human-perceived scanpath variability, and the metric's behavior as a function of the number of generated scanpaths is not analyzed. At minimum, the authors should provide a sensitivity analysis for both RSS and DSS with respect to the sample count and a validation on cases where diversity is controlled (e.g., matching or over-matching human intra-set similarity).","section":"Sec. 4.1, Eq. (8)"}],"minor_comments":[{"comment":"The threshold used for RSS is never specified; please report the exact threshold value and justify its choice.","section":"Sec. 4.1"},{"comment":"The notation ||s - \\tilde{s}|| is used for a mean absolute error; please use an explicit L1 norm and define \\tilde{m}_i as a duration value with its units or normalization clearly stated.","section":"Eq. (6)"},{"comment":"The length prediction module is described only as predicting the validity of each token; please explain how the final length N is derived from consecutive valid tokens and how this interacts with the masked padding during training.","section":"Sec. 3.2.2"},{"comment":"The captions of Tables 3 and 4 cite COCO-FreeView as [66]; the correct reference for COCO-FreeView is [68]. Table 4 also contains inconsistent decimal separators ('0,131', '0,122').","section":"Tables 3 and 4"},{"comment":"Table 5 omits several baselines from Tables 1-2 (e.g., PathGAN, DeepGazeIII, G-Eymol, Itti-Koch); either add them or state why they are excluded from the variability analysis.","section":"Table 5"},{"comment":"Report confidence intervals or significance tests for the KL-divergence values; some differences are small (e.g., several MM values on COCO-FreeView), and the 'best' claims would be more reliable with error bars or paired tests.","section":"Tables 1 and 2"}],"recommendation":"major_revision","confidential_remarks":"The evaluation protocol in Tables 1-2 comes from the authors' own TPP-Gaze paper, and the main diversity metric is introduced in this manuscript. This is not disqualifying, but it increases the need for an independent validation of DSS and for a sampling-effort control in Table 5. If the authors address those points convincingly, the paper would be suitable for publication; the architecture and the traditional metric results are otherwise plausible."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about arXiv:2507.23021 before you invest time. First, it's a genuine first: a diffusion model applied to natural-scene scanpath prediction, with CLIP text conditioning for task-driven viewing and a module that predicts variable-length scanpaths. Prior diffusion work stopped at 360-degree images and text reading. Second, the paper's headline claim—\"more diverse and accurate scanpaths\"—is only half-supported. The accuracy side is credible; the diversity side rests on Table 5, which has a sampling-effort confound the authors never address.\n\nWhat it does well: the evaluation is unusually thorough. They retrained a long list of baselines under identical settings, ran ablations on backbones, diffusion timesteps, the extra alignment loss, and cross-attention vs. input concatenation, and added a zero-shot OSIE experiment in the supplement. That is real work, and the architecture is conventional enough to be plausible.\n\nThe soft spots, in order of size. The diversity comparison in Table 5 does not control for the number of generated scanpaths per stimulus, for any model. RSS counts a human scanpath as covered if any generated scanpath matches it, so it is non-decreasing in the number of samples drawn. The new DSS (Eq. 8) divides by a term involving intra-set generated similarity, which typically falls as you draw more samples, so DSS also shifts with sampling count. ScanDiff is a stochastic diffusion model that can sample arbitrarily many times at inference; Gazeformer and HAT are near-deterministic. The paper never states how many scanpaths each model generated per image, in the main text or the supplement. So the large RSS/DSS margins could reflect how many times you asked each model to run, and the central diversity claim is under-supported as written. The fix is easy: match sample counts across models, report them, and validate DSS against human judgments.\n\nTwo smaller issues. There are no error bars or significance tests anywhere; on COCO-FreeView the headline differences are hundredths of a KL value (0.015 vs. 0.027), and the abstract overclaims relative to Table 1, where free-viewing results are competitive rather than uniformly best. And Eq. 5 states the LVLB simplification without derivation—that deserves a page of algebra or a precise citation. Reusing the TPP-Gaze KL protocol is a minor concern at most; it's a reasonable protocol, just one the authors wrote.\n\nWho it's for: anyone working on scanpath or attention prediction, and anyone building generative sequence models who wants a cautionary example about diversity metrics that don't control for sampling. It deserves a serious referee—the first-application contribution and the comparison workload justify referee time. I'd send it out and make the sample-count issue the centerpiece of the revision requests. The core architecture stands up; the diversity claim needs to be earned.","headline":"A credible first application of diffusion to natural-scene scanpath prediction, but the headline diversity claim rests on a sampling-count confound the authors never address.","tokens_in":20319,"tokens_out":6004,"would_cite":true,"duration_ms":70355,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ScanDiff is a diffusion-plus-Transformer scanpath model that conditions on an image and a text-described viewing task, generates variable-length and diverse fixation sequences, and is reported to outperform existing predictors on…","keywords":["scanpath prediction","diffusion models","visual attention","gaze trajectory generation","visual search","text conditioning","scanpath diversity","transformer"],"falsifier":"Run a human study in which raters choose which of two generated scanpaths looks more like a person's exploration of a scene, then correlate those choices with the DSS and KL-divergence scores; a near-zero or negative correlation on a held-out set of images would falsify the claim that ScanDiff's diversity and accuracy advantages are real rather than artifacts of the metrics.","tokens_in":19326,"feed_emoji":"👀","tokens_out":6858,"duration_ms":75394,"temperature":0.7,"pith_summary":"The paper tries to establish that stochastic, generative modeling of gaze is the right way to build a single scanpath predictor for both free viewing and task-driven search. It proposes ScanDiff, a diffusion model conditioned on an image and a text label of the viewing task, and claims that this unified model produces scanpaths that are quantitatively closer to human scanpaths and more varied across repeated samples than prior autoregressive or GAN-based predictors. A sympathetic reading of the experiments is that the model consistently ranks first or tied-first across datasets and metrics, with the largest margins on the visual-search benchmark, and that its output diversity is measurable and not obtained by sacrificing accuracy. This would matter because applications that simulate human attention need the range of human behavior, not a single typical trajectory.","feed_headline":"Diffusion model beats prior gaze predictors on two visual tasks","feed_subtitle":"One architecture handles free viewing and text-prompted search, producing diverse, variable-length scanpaths.","key_machinery":"The central machinery is a non-autoregressive denoising diffusion Transformer. A scanpath of N fixations, each represented by a 2D position and a duration, is linearly projected into N tokens, noised, and then denoised by an encoder-only Transformer whose extra cross-attention layer receives a joint multimodal embedding built from DINOv2 image features and CLIP text features mapped to a shared space. A linear head predicts the validity of each token, so the final scanpath length is determined dynamically rather than fixed in advance, and sampling from an isotropic Gaussian at inference time produces diverse trajectories. The paper also introduces the Diversity-aware Sequence Score (DSS), which divides the Sequence Score by a term that penalizes generated scanpaths for being more similar to each other than human scanpaths are, and thereby rewards output variability.","core_discovery":"ScanDiff is a scanpath predictor that treats a gaze trajectory as a latent sequence, corrupts it with Gaussian noise over 1000 steps, and learns a conditional denoising Transformer that reconstructs it from a visual stimulus and a textual viewing task. The distinctive choices are conditioning via cross-attention on a joint image-text embedding rather than rigid concatenation, a length-prediction head that lets the model output variable-length scanpaths instead of a fixed number of fixations, and an extra loss term that pins the final latent to an isotropic Gaussian. The paper reports that on COCO-FreeView, MIT1003, and COCO-Search18 the model obtains the best or tied-best scores on most similarity metrics, notably duration-aware SM and SS on free-viewing data and the main metrics on visual search, and that it also scores highest on the recall-based RSS metric and on the newly proposed Diversity-aware Sequence Score. These results are presented as evidence that the generated trajectories are not only accurate but diverse.","pith_inferences":["Beyond the paper, text conditioning could be extended from single object labels to richer instructions such as 'find the red mug on the table,' turning the model into a controllable attention simulator.","Beyond the paper, the Diversity-aware Sequence Score could be reused as a generic diversity penalty in other generative sequence tasks that collapse to deterministic outputs.","Beyond the paper, sampling many diverse scanpaths offers a cheap way to synthesize training data for downstream agents that need varied exploration trajectories rather than one average fixation path."],"forward_implications":["A single model can now serve both free-viewing and goal-directed scanpath prediction, with no architectural change between tasks.","Repeated sampling from the same image and task yields multiple plausible scanpaths rather than a single averaged trajectory.","The duration-aware similarity scores on COCO-FreeView and MIT1003 improve, which the paper attributes to explicit modeling of fixation durations.","On COCO-Search18, the reported score gaps over the next-best model are largest in MultiMatch distributions in both target-present and target-absent conditions.","Zero-shot evaluation on OSIE indicates that the trained model generalizes to a dataset it never saw."],"supporting_citations":[{"why":"Supplies the KL-divergence evaluation protocol that compares human-versus-generated metric distributions with human-versus-human consistency distributions, and serves as a main recent baseline.","marker":"[23]"},{"why":"Establishes the diffusion formulation for scanpath generation and the variational-lower-bound objective that ScanDiff simplifies.","marker":"[8]"},{"why":"Prior diffusion scanpath model for 360-degree images whose fixed-length design ScanDiff extends with variable-length prediction.","marker":"[36]"},{"why":"Gazeformer provides the Transformer conditioning design with image and task features in a joint embedding space that ScanDiff adapts.","marker":"[48]"},{"why":"DINOv2 is the visual backbone used to encode the stimulus.","marker":"[51]"},{"why":"CLIP text encoder provides the task-conditioning features.","marker":"[55]"},{"why":"COCO-Search18 supplies the visual-search dataset and its train-test splits.","marker":"[16]"},{"why":"COCO-FreeView is the free-viewing dataset used for training and evaluation.","marker":"[68]"}],"fun_headline_variants":["Text-conditioned diffusion generates diverse, accurate scanpaths","Unified diffusion model for free-viewing and task-driven gaze","Diffusion model predicts scanpaths from image and text prompt","ScanDiff: diffusion model yields diverse, text-adaptive gaze paths","One diffusion model beats prior methods on gaze prediction"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the KL-divergence comparison protocol and the newly proposed DSS metric reflect human judgment of scanpath quality; if either favors spread-out outputs without correlating with human preferences, the reported accuracy-plus-diversity advantage would not be established.","fun_headline_variants_meta":{"raw":{"variants":["Text-conditioned diffusion generates diverse, accurate scanpaths","Unified diffusion model for free-viewing and task-driven gaze","Diffusion model predicts scanpaths from image and text prompt","ScanDiff: diffusion model yields diverse, text-adaptive gaze paths","One diffusion model beats prior methods on gaze prediction"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000396,"raw_usage":{"total_tokens":2062,"prompt_tokens":919,"completion_tokens":1143,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":535,"completion_tokens_details":{"reasoning_tokens":1062}},"tokens_in":535,"tokens_out":1143,"duration_ms":11614,"temperature":1.0,"reasoning_tokens":1062,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T11:07:35.885012+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a human study in which raters choose which of two generated scanpaths looks more like a person's exploration of a scene, then correlate those choices with the DSS and KL-divergence scores; a near-zero or negative correlation on a held-out set of images would falsify the claim that ScanDiff's diversity and accuracy advantages are real rather than artifacts of the metrics.","supporting_citations":[{"cited_title":"TPP-Gaze: Modelling Gaze Dynam- ics in Space and Time with Neural Temporal Point Processes","cited_arxiv_id":null,"evidence_quote":"Supplies the KL-divergence evaluation protocol that compares human-versus-generated metric distributions with human-versus-human consistency distributions, and serves as a main recent baseline."},{"cited_title":"ScanDL: A diffu- sion model for generating synthetic scanpaths on texts","cited_arxiv_id":null,"evidence_quote":"Establishes the diffusion formulation for scanpath generation and the variational-lower-bound objective that ScanDiff simplifies."},{"cited_title":"DiffGaze: A Diffusion Model for Continuous Gaze Sequence Generation on 360{\\deg} Images","cited_arxiv_id":"2403.17477","evidence_quote":"Prior diffusion scanpath model for 360-degree images whose fixed-length design ScanDiff extends with variable-length prediction."},{"cited_title":"Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Hu- man Attention","cited_arxiv_id":null,"evidence_quote":"Gazeformer provides the Transformer conditioning design with image and task features in a joint embedding space that ScanDiff adapts."},{"cited_title":"COCO-Search18 fixation dataset for predicting goal-directed attention control","cited_arxiv_id":null,"evidence_quote":"COCO-Search18 supplies the visual-search dataset and its train-test splits."}],"review_version":1}