Pith. sign in

REVIEW 3 major objections 6 minor 76 references

Modeling Human Gaze Behavior with Diffusion Models for Unified Scanpath Prediction

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ScanDiff is a diffusion-plus-Transformer scanpath model that conditions on an image and a text-described viewing task, generates variable-length and diverse fixation sequences, and is reported to outperform existing predictors on…

desk verdict A credible first application of diffusion to natural-scene scanpath prediction, but the headline diversity claim rests on a sampling-count confound the authors never address. read the letter →

arxiv 2507.23021 v1 pith:IFBPG3AY submitted 2025-07-30 cs.CV cs.AI

Giuseppe Cartella , Vittorio Cuculo , Alessandro D'Amelio , Marcella Cornia , Giuseppe Boccignone , Rita Cucchiara This is my paper
classification cs.CVcs.AI
keywords scanpathpredictiondiffusionmodelsvisualattentiongazetrajectorygenerationsearchtextconditioningdiversitytransformer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that stochastic, generative modeling of gaze is the right way to build a single scanpath predictor for both free viewing and task-driven search. It proposes ScanDiff, a diffusion model conditioned on an image and a text label of the viewing task, and claims that this unified model produces scanpaths that are quantitatively closer to human scanpaths and more varied across repeated samples than prior autoregressive or GAN-based predictors. A sympathetic reading of the experiments is that the model consistently ranks first or tied-first across datasets and metrics, with the largest margins on the visual-search benchmark, and that its output diversity is measurable and not obtained by sacrificing accuracy. This would matter because applications that simulate human attention need the range of human behavior, not a single typical trajectory.

What carries the argument

The central machinery is a non-autoregressive denoising diffusion Transformer. A scanpath of N fixations, each represented by a 2D position and a duration, is linearly projected into N tokens, noised, and then denoised by an encoder-only Transformer whose extra cross-attention layer receives a joint multimodal embedding built from DINOv2 image features and CLIP text features mapped to a shared space. A linear head predicts the validity of each token, so the final scanpath length is determined dynamically rather than fixed in advance, and sampling from an isotropic Gaussian at inference time produces diverse trajectories. The paper also introduces the Diversity-aware Sequence Score (DSS), which divides the Sequence Score by a term that penalizes generated scanpaths for being more similar to each other than human scanpaths are, and thereby rewards output variability.

What would settle it

Run a human study in which raters choose which of two generated scanpaths looks more like a person's exploration of a scene, then correlate those choices with the DSS and KL-divergence scores; a near-zero or negative correlation on a held-out set of images would falsify the claim that ScanDiff's diversity and accuracy advantages are real rather than artifacts of the metrics.

Watch

Extended reading notes

Core claim

ScanDiff is a scanpath predictor that treats a gaze trajectory as a latent sequence, corrupts it with Gaussian noise over 1000 steps, and learns a conditional denoising Transformer that reconstructs it from a visual stimulus and a textual viewing task. The distinctive choices are conditioning via cross-attention on a joint image-text embedding rather than rigid concatenation, a length-prediction head that lets the model output variable-length scanpaths instead of a fixed number of fixations, and an extra loss term that pins the final latent to an isotropic Gaussian. The paper reports that on COCO-FreeView, MIT1003, and COCO-Search18 the model obtains the best or tied-best scores on most similarity metrics, notably duration-aware SM and SS on free-viewing data and the main metrics on visual search, and that it also scores highest on the recall-based RSS metric and on the newly proposed Diversity-aware Sequence Score. These results are presented as evidence that the generated trajectories are not only accurate but diverse.

Load-bearing premise

The load-bearing premise is that the KL-divergence comparison protocol and the newly proposed DSS metric reflect human judgment of scanpath quality; if either favors spread-out outputs without correlating with human preferences, the reported accuracy-plus-diversity advantage would not be established.

Editorial extensions

If this is right

  • A single model can now serve both free-viewing and goal-directed scanpath prediction, with no architectural change between tasks.
  • Repeated sampling from the same image and task yields multiple plausible scanpaths rather than a single averaged trajectory.
  • The duration-aware similarity scores on COCO-FreeView and MIT1003 improve, which the paper attributes to explicit modeling of fixation durations.
  • On COCO-Search18, the reported score gaps over the next-best model are largest in MultiMatch distributions in both target-present and target-absent conditions.
  • Zero-shot evaluation on OSIE indicates that the trained model generalizes to a dataset it never saw.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, text conditioning could be extended from single object labels to richer instructions such as 'find the red mug on the table,' turning the model into a controllable attention simulator.
  • Beyond the paper, the Diversity-aware Sequence Score could be reused as a generic diversity penalty in other generative sequence tasks that collapse to deterministic outputs.
  • Beyond the paper, sampling many diverse scanpaths offers a cheap way to synthesize training data for downstream agents that need varied exploration trajectories rather than one average fixation path.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes ScanDiff, an encoder-only Transformer combined with a diffusion process for scanpath generation. Fixations are embedded, corrupted by Gaussian noise, and denoised under conditioning from a DINOv2 visual backbone and a CLIP text encoder that encodes the viewing task; a length-prediction head allows variable-length outputs. The training loss combines a simplified variational lower bound, reconstruction, validity prediction, and a prior-alignment term. The authors evaluate on COCO-FreeView, MIT1003, COCO-Search18, and zero-shot OSIE, using the KL-divergence protocol of TPP-Gaze with MultiMatch, ScanMatch, Sequence Score, and Semantic Sequence Score, and add a scanpath variability analysis based on Recall Sequence Score (RSS) and a newly introduced Diversity-aware Sequence Score (DSS). The paper claims state-of-the-art accuracy on several metrics and, centrally, that the stochastic diffusion formulation produces more diverse scanpaths than prior models. The central diversity claim relies on Table 5 and the new DSS metric, and the paper acknowledges in Sec. 4.4 that this variability analysis is a first attempt.

Significance. If the diversity claim were fully supported, ScanDiff would be a useful contribution: it offers a unified free-viewing/visual-search model with variable-length generation, uses a reasonable multimodal conditioning design, retrains several baselines under identical settings, reports zero-shot results on OSIE, and provides a public project page. The traditional metric results, especially on COCO-Search18, are strong and well aligned with the main architecture story. However, the load-bearing 'more diverse' claim currently rests on a metric introduced in this paper (DSS) and on RSS, and neither is shown to be invariant to the number of generated scanpaths per stimulus. Because the paper itself says that conventional metrics reward averaged behavior, the diversity analysis is central to the contribution, and the current evidence is not yet conclusive. The self-evaluation protocol from the authors' TPP-Gaze is a further reason to require a robustness analysis, though it is not by itself an error.

major comments (3)
  1. [Sec. 4.4, Table 5; Sec. 3.3] The central diversity claim is confounded by the uncontrolled number of generated scanpaths per stimulus. As defined in Sec. 4.1, RSS counts a human scanpath as covered if its SS with at least one generated scanpath exceeds a threshold, so RSS is non-decreasing when more samples are drawn. DSS in Eq. 8 also depends on the size of the generated set through SS(sg, sg), an average over within-set pairs that generally changes with sample count. The manuscript does not state how many scanpaths were generated per image for ScanDiff or for any baseline in Table 5, nor for the KL-protocol results in Tables 1-2. Since ScanDiff is stochastic and can produce an arbitrary number of samples, while several baselines (e.g., Gazeformer, HAT) are deterministic, the large RSS and DSS gains could reflect sampling effort rather than better modeling of human gaze variability. The authors should match the number of samples across methods, report RSS/DSS as a function of the number of samples, or use a sample-size-invariant diversity measure.
  2. [Sec. 3.3, Eq. (5)] Eq. (5) is presented as a simplification of the variational lower bound, citing [8], but no derivation is given and the form is not standard: the loss sums the squared error between z0 and the model prediction at every timestep with uniform weight, with no explicit dependence on the noise level. Because this objective is the main training signal for the denoising network, readers cannot verify whether the simplified loss corresponds to the stated VLB or whether the reported performance depends on an ad hoc reweighting. Please include the derivation (or a precise reference with equation numbers) and state the role of the importance-sampling scheme.
  3. [Sec. 4.1, Eq. (8)] The newly introduced DSS metric is used as the main quantitative evidence for the diversity claim, but it is not validated against human judgments or against a synthetic set with known diversity. As written, DSS rewards a generated set whose within-set similarity matches the human within-set similarity, but it is not shown that this criterion corresponds to human-perceived scanpath variability, and the metric's behavior as a function of the number of generated scanpaths is not analyzed. At minimum, the authors should provide a sensitivity analysis for both RSS and DSS with respect to the sample count and a validation on cases where diversity is controlled (e.g., matching or over-matching human intra-set similarity).
minor comments (6)
  1. [Sec. 4.1] The threshold used for RSS is never specified; please report the exact threshold value and justify its choice.
  2. [Eq. (6)] The notation ||s - \tilde{s}|| is used for a mean absolute error; please use an explicit L1 norm and define \tilde{m}_i as a duration value with its units or normalization clearly stated.
  3. [Sec. 3.2.2] The length prediction module is described only as predicting the validity of each token; please explain how the final length N is derived from consecutive valid tokens and how this interacts with the masked padding during training.
  4. [Tables 3 and 4] The captions of Tables 3 and 4 cite COCO-FreeView as [66]; the correct reference for COCO-FreeView is [68]. Table 4 also contains inconsistent decimal separators ('0,131', '0,122').
  5. [Table 5] Table 5 omits several baselines from Tables 1-2 (e.g., PathGAN, DeepGazeIII, G-Eymol, Itti-Koch); either add them or state why they are excluded from the variability analysis.
  6. [Tables 1 and 2] Report confidence intervals or significance tests for the KL-divergence values; some differences are small (e.g., several MM values on COCO-FreeView), and the 'best' claims would be more reliable with error bars or paired tests.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; the only self-referential element is a minor non-load-bearing citation to the authors' prior evaluation protocol.

full rationale

ScanDiff's contributions are an architecture and a training objective; its headline claims are empirical benchmark comparisons on held-out test splits against external and re-trained baselines. The model does not fit constants to test data, and no predicted scanpath is defined in terms of the evaluation metrics. The KL-divergence protocol is taken from TPP-Gaze [23], the authors' own prior work, but it is a distribution-comparison procedure applied identically to every model and does not encode ScanDiff's outputs, so it is not load-bearing circularity. The new DSS metric (Eq. 8) rewards low intra-generated similarity when human scanpaths are also diverse; this may raise validity or fairness questions, but it is not a case of deriving a result from its own definition: DSS is an external measurement, not a fitted parameter or an identity that forces ScanDiff's scores. RSS is explicitly attributed to HAT and only renamed, so the renaming is not presented as a new result. No uniqueness theorem, ansatz-smuggling citation, or fitted-input-called-prediction appears. The unstated number of generated scanpaths per image in Table 5 is a potential confound for the diversity comparison, but that is a benchmarking caveat, not circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim is empirical, so the main free choices are hyperparameters and the evaluation protocol. No novel entities are introduced; the DSS metric is a measurement, not an entity.

free parameters (4)
  • Diffusion timesteps T = 1000
    Chosen by ablation (Section 4.3, Table 4); not derived from theory.
  • Maximum scanpath length L = 16
    Set to match the median across datasets and prior work [13,15], used to pad/truncate sequences (Section 3.3).
  • Loss weighting = implicitly equal
    Eq. 4 sums four loss terms without stated weights; this choice affects training and is not justified.
  • Number of generated scanpaths per stimulus at inference = not reported
    The diversity metrics depend on sampling multiple trajectories, but the paper does not state how many are generated; this affects DSS and RSS scores.
assumptions (5)
  • standard math Standard Gaussian diffusion theory and variational bounds apply to the learned embedding space
    Used in Section 3.1 and 3.2; the simplification of LVLB (Eq. 5) is claimed to follow from [8].
  • domain assumption Scanpaths can be represented as fixed-length sequences of (x,y,duration) tokens after padding/truncation to length 16
    Section 3.3 pads or truncates ground truth to L=16; the model predicts a validity mask to recover variable length.
  • domain assumption CLIP text embeddings encode viewing task semantics well enough to condition attention
    Section 3.2.2 uses CLIP ViT-B/32 for task encoding; ablation with RoBERTa degrades performance, supporting the choice empirically.
  • domain assumption The KL-divergence evaluation protocol from TPP-Gaze is a valid measure of scanpath quality
    Section 4.1 adopts this protocol from [23], which shares authors with this paper; no independent validation is offered.
  • domain assumption DINOv2 features provide a rich visual representation for attention prediction
    Section 4.1 uses DINOv2 ViT-B/14; ablation shows it outperforms CLIP and ResNet on most metrics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Modeling Human Gaze Behavior with Diffusion Models for Unified Scanpath Prediction." pith.science (2026). https://pith.science/paper/IFBPG3AY

@misc{pith2026250723021,
  author       = {Pith},
  title        = {Pith review of: Modeling Human Gaze Behavior with Diffusion Models for Unified Scanpath Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IFBPG3AY}},
  note         = {Machine review of arXiv:2507.23021}
}
read the original abstract

Predicting human gaze scanpaths is crucial for understanding visual attention, with applications in human-computer interaction, autonomous systems, and cognitive robotics. While deep learning models have advanced scanpath prediction, most existing approaches generate averaged behaviors, failing to capture the variability of human visual exploration. In this work, we present ScanDiff, a novel architecture that combines diffusion models with Vision Transformers to generate diverse and realistic scanpaths. Our method explicitly models scanpath variability by leveraging the stochastic nature of diffusion models, producing a wide range of plausible gaze trajectories. Additionally, we introduce textual conditioning to enable task-driven scanpath generation, allowing the model to adapt to different visual search objectives. Experiments on benchmark datasets show that ScanDiff surpasses state-of-the-art methods in both free-viewing and task-driven scenarios, producing more diverse and accurate scanpaths. These results highlight its ability to better capture the complexity of human visual behavior, pushing forward gaze prediction research. Source code and models are publicly available at https://aimagelab.github.io/ScanDiff.

Figures

Figures reproduced from arXiv: 2507.23021 by the authors.

Figure 1
Figure 1. The diffusion process of ScanDiff that generates re￾alistic scanpaths through learned transitions conditioned on image I and viewing task c. Models based on deep convolutional [40] and recurrent architectures [17], as well as more recent Transformer￾based methods [48, 69], have significantly improved the ability to predict eye movements. These models are effec￾tive in both free-viewing scenarios, where observers exp… view at source ↗
Figure 2
Figure 2. Overview of ScanDiff. Given a stimulus I and a viewing task c, a scanpath s˜ is generated through a diffusion process. patterns (see [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparison of simulated and human scanpaths across different datasets for both free-viewing and visual search tasks. From top [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison of simulated and human scanpaths on the COCO-FreeView dataset. [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparison of simulated and human scanpaths on the MIT1003 dataset. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison of simulated and human scanpaths on the OSIE dataset. [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Qualitative comparison of simulated and human scanpaths on the COCO-Search18 (TP) dataset for the visual search task. [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Qualitative comparison of simulated and human scanpaths on the COCO-Search18 (TA) dataset for the visual search task. [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Qualitative comparison of the variability in simulated and human scanpaths on the COCOFreeView dataset. Each row corresponds [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: Qualitative comparison of the variability in simulated and human scanpaths on the MIT1003 dataset. Each row corresponds to [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: Qualitative comparison of the variability of simulated and human scanpaths on the COCO-Search18 (TP) dataset for the viewing [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 12
Figure 12. Figure 12: Qualitative comparison of the variability of simulated and human scanpaths on the COCO-Search18 (TA) dataset for the viewing [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

76 extracted references · 67 canonical work pages

  1. [8]

    ScanDL: A diffu- sion model for generating synthetic scanpaths on texts

    Lena S Bolliger, David R Reich, Patrick Haller, Deborah N Jakobi, Paul Prasse, and Lena A J ¨ager. ScanDL: A diffu- sion model for generating synthetic scanpaths on texts. In EMNLP, 2023. 2, 4

    work page 2023

  2. [1]

    Active Vision

    J Aloimonos, Isaac Weiss, and Amit Bandyopadhyay. Active Vision. In ICCV, 1987. 2

    work page 1987

  3. [2]

    Pathgan: Visual scanpath prediction with generative adversarial networks

    Marc Assens, Xavier Giro-i Nieto, Kevin McGuinness, and Noel E O’Connor. Pathgan: Visual scanpath prediction with generative adversarial networks. In ECCV Workshops, 2018. 2, 5, 6, 7, 12

    work page 2018

  4. [3]

    Active and Exploratory Perception

    Ruzena Bajcsy and Mario Campos. Active and Exploratory Perception. CVGIP: Image Understanding , 56(1):31–40,

    work page

  5. [4]

    Re- visiting active perception

    Ruzena Bajcsy, Yiannis Aloimonos, and John K Tsotsos. Re- visiting active perception. Autonomous Robots, 42:177–196,

    work page

  6. [5]

    Animate Vision

    Dana H Ballard. Animate Vision. Artificial Intelligence, 48 (1):57–86, 1991. 2

    work page 1991

  7. [6]

    The neural basis of visual attention

    James W Bisley. The neural basis of visual attention. The Journal of physiology, 589(1):49–57, 2011. 7

    work page 2011

  8. [7]

    Modelling gaze shift as a constrained random walk

    Giuseppe Boccignone and Mario Ferraro. Modelling gaze shift as a constrained random walk. Physica A: Statistical Mechanics and its Applications, 331(1-2):207–218, 2004. 2, 5, 6, 12

    work page 2004

Show all 76 references
  1. [9]

    Real-world vision: Selective perception and task

    Roxanne L Canosa. Real-world vision: Selective perception and task. ACM Transactions on Applied Perception , 6(2): 1–34, 2009. 1

    2009

  2. [10]

    Trends, Applications, and Challenges in Human Attention Modelling

    Giuseppe Cartella, Marcella Cornia, Vittorio Cuculo, Alessandro D’Amelio, Dario Zanca, Giuseppe Boccignone, and Rita Cucchiara. Trends, Applications, and Challenges in Human Attention Modelling. In IJCAI, 2024. 1, 2

    2024

  3. [11]

    Unveiling the Truth: Exploring Human Gaze Patterns in Fake Images.IEEE Signal Processing Letters, 31: 820–824, 2024

    Giuseppe Cartella, Vittorio Cuculo, Marcella Cornia, and Rita Cucchiara. Unveiling the Truth: Exploring Human Gaze Patterns in Fake Images.IEEE Signal Processing Letters, 31: 820–824, 2024. 1

    2024

  4. [12]

    Viewing task influences eye movement control during active scene perception

    Monica S Castelhano, Michael L Mack, and John M Hender- son. Viewing task influences eye movement control during active scene perception. Journal of vision, 9(3):6–6, 2009. 8

    2009

  5. [13]

    Predicting Human Scanpaths in Visual Question Answering

    Xianyu Chen, Ming Jiang, and Qi Zhao. Predicting Human Scanpaths in Visual Question Answering. InCVPR, 2021. 2, 5, 6, 7, 8, 12, 14, 15, 16, 17, 18, 19, 20

    2021

  6. [14]

    Beyond average: Individualized visual scanpath prediction

    Xianyu Chen, Ming Jiang, and Qi Zhao. Beyond average: Individualized visual scanpath prediction. In CVPR, 2024. 2, 5, 6, 7, 8, 12

    2024

  7. [15]

    GazeXplain: Learning to Predict Natural Language Explanations of Vi- sual Scanpaths

    Xianyu Chen, Ming Jiang, and Qi Zhao. GazeXplain: Learning to Predict Natural Language Explanations of Vi- sual Scanpaths. In ECCV, 2024. 1, 2, 5, 6, 7, 8, 12, 14, 15, 16, 17, 18, 19, 20

    2024

  8. [16]

    COCO-Search18 fixation dataset for predicting goal-directed attention control

    Yupei Chen, Zhibo Yang, Seoyoung Ahn, Dimitris Samaras, Minh Hoai, and Gregory Zelinsky. COCO-Search18 fixation dataset for predicting goal-directed attention control. Scien- tific Reports, 11(1):1–11, 2021. 2, 5, 7, 8, 12, 13

    2021

  9. [17]

    Scanpath Prediction for Visual Attention using IOR-ROI LSTM

    Zhenzhong Chen and Wanjie Sun. Scanpath Prediction for Visual Attention using IOR-ROI LSTM. In IJCAI, 2018. 1, 2, 5, 6, 8, 14, 15, 16, 19

    2018

  10. [18]

    Simulating human mobility with a trajectory generation framework based on diffusion model

    Chen Chu, Hengcai Zhang, Peixiao Wang, and Feng Lu. Simulating human mobility with a trajectory generation framework based on diffusion model. Int. J. of Geograph- ical Information Science, 38(5):847–878, 2024. 2

    2024

  11. [19]

    On the constrained time-series genera- tion problem

    Andrea Coletta, Sriram Gopalakrishnan, Daniel Borrajo, and Svitlana Vyetrenko. On the constrained time-series genera- tion problem. In NeurIPS, 2023. 2

    2023

  12. [20]

    Time series diffusion in the frequency do- main

    Jonathan Crabb ´e, Nicolas Huynh, Jan Stanczuk, and Mihaela Van Der Schaar. Time series diffusion in the frequency do- main. In ICML, 2024. 2

    2024

  13. [21]

    ScanMatch: A novel method for compar- ing fixation sequences

    Filipe Cristino, Sebastiaan Math ˆot, Jan Theeuwes, and Iain D Gilchrist. ScanMatch: A novel method for compar- ing fixation sequences. Behavior Research Methods, 42(3): 692–700, 2010. 5

    2010

  14. [22]

    Diffusion models in vision: A survey

    Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah. Diffusion models in vision: A survey. IEEE Trans. PAMI, 45(9):10850–10869, 2023. 2

    2023

  15. [23]

    TPP-Gaze: Modelling Gaze Dynam- ics in Space and Time with Neural Temporal Point Processes

    Alessandro D’Amelio, Giuseppe Cartella, Vittorio Cuculo, Manuele Lucchi, Marcella Cornia, Rita Cucchiara, and Giuseppe Boccignone. TPP-Gaze: Modelling Gaze Dynam- ics in Space and Time with Neural Temporal Point Processes. In WACV, 2025. 2, 5, 6, 7, 8, 12, 14, 15, 16, 17, 18, 19, 20

    2025

  16. [24]

    Vision transformers need registers

    Timoth ´ee Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. In ICLR,

  17. [25]

    It de- pends on how you look at it: Scanpath comparison in mul- tiple dimensions with MultiMatch, a vector-based approach

    Richard Dewhurst, Marcus Nystr ¨om, Halszka Jarodzka, Tom Foulsham, Roger Johansson, and Kenneth Holmqvist. It de- pends on how you look at it: Scanpath comparison in mul- tiple dimensions with MultiMatch, a vector-based approach. Behavior Research Methods, 44(4):1079–1100, 2012. 5

    2012

  18. [26]

    Diffusion Models Beat GANs on Image Synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion Models Beat GANs on Image Synthesis. In NeurIPS, 2021. 2

    2021

  19. [27]

    An Image is Worth 16x16 Words: Trans- formers for Image Recognition at Scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An Image is Worth 16x16 Words: Trans- formers for Image Recognition at Scale. In ICLR, 2021. 2

    2021

  20. [28]

    DiffuSeq: Sequence to Sequence Text Gen- eration with Diffusion Models

    Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and LingPeng Kong. DiffuSeq: Sequence to Sequence Text Gen- eration with Diffusion Models. In ICLR, 2022. 2

    2022

  21. [29]

    Mask R-CNN

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask R-CNN. In ICCV, 2017. 7

    2017

  22. [30]

    Human attention in image captioning: Dataset and analysis

    Sen He, Hamed R Tavakoli, Ali Borji, and Nicolas Pugeault. Human attention in image captioning: Dataset and analysis. In ICCV, 2019. 2

    2019

  23. [31]

    Human gaze control during real-world scene perception

    John M Henderson. Human gaze control during real-world scene perception. Trends in cognitive sciences , 7(11):498– 504, 2003. 7 9

    2003

  24. [32]

    Denoising Dif- fusion Probabilistic Models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Dif- fusion Probabilistic Models. In NeurIPS, 2020. 2

    2020

  25. [33]

    L. Itti, C. Koch, and E. Niebur. A Model of Saliency-based Visual Attention for Rapid Scene Analysis. IEEE Trans. PAMI, 20:1254–1259, 1998. 2, 5, 6, 12

    1998

  26. [34]

    A Vector-based, Multidimensional Scanpath Sim- ilarity Measure

    Halszka Jarodzka, Kenneth Holmqvist, and Marcus Nystr¨om. A Vector-based, Multidimensional Scanpath Sim- ilarity Measure. In ETRA, 2010. 5

    2010

  27. [35]

    UEyes: Understanding Visual Saliency across User Interface Types

    Yue Jiang, Luis A Leiva, Hamed Rezazadegan Tavakoli, Paul RB Houssel, Julia Kylm ¨al¨a, and Antti Oulasvirta. UEyes: Understanding Visual Saliency across User Interface Types. In ACM CHI, 2023. 1

    2023

  28. [36]

    DiffGaze: A Diffusion Model for Continuous Gaze Sequence Generation on 360° Images

    Chuhan Jiao, Yao Wang, Guanhua Zhang, Mihai B ˆace, Zhiming Hu, and Andreas Bulling. DiffGaze: A Diffusion Model for Continuous Gaze Sequence Generation on 360° Images. arXiv preprint arXiv:2403.17477, 2024. 2, 4, 12

    2024 arXiv

  29. [37]

    Learning to predict where humans look

    Tilke Judd, Krista Ehinger, Fr ´edo Durand, and Antonio Tor- ralba. Learning to predict where humans look. In ICCV,

  30. [38]

    DiffWave: A Versatile Diffusion Model for Audio Synthesis

    Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. DiffWave: A Versatile Diffusion Model for Audio Synthesis. In ICLR, 2021. 2

    2021

  31. [39]

    State-of- the-art in human scanpath prediction

    Matthias K ¨ummerer and Matthias Bethge. State-of- the-art in human scanpath prediction. arXiv preprint arXiv:2102.12239, 2021. 8

    2021 arXiv

  32. [40]

    Deep Gaze I: Boosting saliency prediction with feature maps trained on ImageNet

    Matthias K ¨ummerer, Lucas Theis, and Matthias Bethge. Deep Gaze I: Boosting saliency prediction with feature maps trained on ImageNet. arXiv preprint arXiv:1411.1045, 2014. 1, 6, 12

    2014 arXiv

  33. [41]

    DeepGaze III: Modeling free-viewing human scanpaths with deep learning

    Matthias K ¨ummerer, Matthias Bethge, and Thomas SA Wal- lis. DeepGaze III: Modeling free-viewing human scanpaths with deep learning. J. of Vision, 22(5):7–7, 2022. 1, 2, 5, 6, 12, 14

    2022

  34. [42]

    The way we look at an image or a webpage can reveal per- sonality traits

    Thomas Le Bras, Benoit Allibe, and Karine Dor ´e-Mazars. The way we look at an image or a webpage can reveal per- sonality traits. Scientific Reports, 14(1):15488, 2024. 1

    2024

  35. [43]

    UniAR: A Uni- fied model for predicting human Attention and Responses on visual content

    Peizhao Li, Junfeng He, Gang Li, Rachit Bhargava, Shaolei Shen, Nachiappan Valliappan, Youwei Liang, Hongxiang Gu, Venky Ramachandran, Yang Li, et al. UniAR: A Uni- fied model for predicting human Attention and Responses on visual content. In NeurIPS, 2024. 2

    2024

  36. [44]

    Diffusion-LM Improves Con- trollable Text Generation

    Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto. Diffusion-LM Improves Con- trollable Text Generation. In NeurIPS, 2022. 2

    2022

  37. [45]

    Regular Time-series Generation using SGM

    Haksoo Lim, Minjung Kim, Sewon Park, and Noseong Park. Regular Time-series Generation using SGM. arXiv preprint arXiv:2301.08518, 2023. 2

    2023 arXiv

  38. [46]

    Eyear: Learning audio synchronized human gaze trajectory based on physics- informed dynamics

    Xiaochuan Liu, Xin Cheng, Yuchong Sun, Xiaoxue Wu, Rui- hua Song, Hao Sun, and Denghao Zhang. Eyear: Learning audio synchronized human gaze trajectory based on physics- informed dynamics. arXiv preprint arXiv:2502.20858, 2025. 2

    2025 arXiv

  39. [47]

    RoBERTa: A Robustly Optimized BERT Pretraining Approach

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv preprint arXiv:1907.11692, 2019. 7

    1907 arXiv

  40. [48]

    Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Hu- man Attention

    Sounak Mondal, Zhibo Yang, Seoyoung Ahn, Dimitris Samaras, Gregory Zelinsky, and Minh Hoai. Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Hu- man Attention. In CVPR, 2023. 1, 2, 4, 5, 7, 8, 12, 17, 18, 20

    2023

  41. [49]

    Look Hear: Gaze Prediction for Speech-directed Hu- man Attention

    Sounak Mondal, Seoyoung Ahn, Zhibo Yang, Niranjan Bala- subramanian, Dimitris Samaras, Gregory Zelinsky, and Minh Hoai. Look Hear: Gaze Prediction for Speech-directed Hu- man Attention. In ECCV, 2024. 2

    2024

  42. [50]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In ICML, 2021. 4

    2021

  43. [51]

    DINOv2: Learning Robust Visual Features without Supervi- sion

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. DINOv2: Learning Robust Visual Features without Supervi- sion. arXiv preprint arXiv:2304.07193, 2023. 2, 5

    2023 arXiv

  44. [52]

    Looking at the Right Stuff - Guided Semantic-Gaze for Au- tonomous Driving

    Anwesan Pal, Sayan Mondal, and Henrik I Christensen. Looking at the Right Stuff - Guided Semantic-Gaze for Au- tonomous Driving. In CVPR, 2020. 1

    2020

  45. [53]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, 2023. 4

    2023

  46. [54]

    Gvgnet: Gaze-directed visual grounding for learning under- specified object referring intention

    Kun Qian, Zhuoyang Zhang, Wei Song, and Jianfeng Liao. Gvgnet: Gaze-directed visual grounding for learning under- specified object referring intention. IEEE RA-L, 8(9), 2023. 1

    2023

  47. [55]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In ICML, 2021. 5

    2021

  48. [56]

    Curious eyes: Individual differences in per- sonality predict eye movement behavior in scene-viewing

    Evan F Risko, Nicola C Anderson, Sophie Lanthier, and Alan Kingstone. Curious eyes: Individual differences in per- sonality predict eye movement behavior in scene-viewing. Cognition, 122(1):86–90, 2012. 1

    2012

  49. [57]

    Controllable human trajectory generation using profile-guided latent diffusion

    Yiwen Song, Jingtao Ding, Jian Yuan, Qingmin Liao, and Yong Li. Controllable human trajectory generation using profile-guided latent diffusion. ACM Trans. KDD, 19(1):1– 25, 2024. 2

    2024

  50. [58]

    Perception and navigation in autonomous systems in the era of learning: A survey

    Yang Tang, Chaoqiang Zhao, Jianrui Wang, Chongzhen Zhang, Qiyu Sun, Wei Xing Zheng, Wenli Du, Feng Qian, and J¨urgen Kurths. Perception and navigation in autonomous systems in the era of learning: A survey. IEEE Transactions on Neural Networks and Learning Systems , 34(12):960...

    2022

  51. [59]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, 2017. 2, 3

    2017

  52. [60]

    ScanTD: 360° Scanpath Prediction based on Time-Series Diffusion

    Yujia Wang, Fang-Lue Zhang, and Neil A Dodgson. ScanTD: 360° Scanpath Prediction based on Time-Series Diffusion. In ACM Multimedia, 2024. 2, 4, 12

    2024

  53. [61]

    Diff-RNTraj: A Structure-Aware Diffusion Model for Road Network-Constrained Trajectory Generation

    Tonglong Wei, Youfang Lin, Shengnan Guo, Yan Lin, Yi- heng Huang, Chenyang Xiang, Yuqing Bai, and Huaiyu Wan. Diff-RNTraj: A Structure-Aware Diffusion Model for Road Network-Constrained Trajectory Generation. IEEE Trans. KDD, 36(12):7940–7953, 2024. 2 10

    2024

  54. [62]

    Predicting human gaze beyond pixels

    Juan Xu, Ming Jiang, Shuo Wang, Mohan S Kankanhalli, and Qi Zhao. Predicting human gaze beyond pixels. J. of Vision, 14(1):28–28, 2014. 12, 13

    2014

  55. [63]

    How do personality traits modu- late real-world gaze behavior? generated gaze data shows situation-dependent modulations

    Jumpei Yamashita, Yoshiaki Takimoto, Haruo Oishi, and Takatsune Kumada. How do personality traits modu- late real-world gaze behavior? generated gaze data shows situation-dependent modulations. Frontiers in Psychology, 14:1144048, 2024. 1

    2024

  56. [64]

    A survey on diffusion models for time series and spatio-temporal data

    Yiyuan Yang, Ming Jin, Haomin Wen, Chaoli Zhang, Yux- uan Liang, Lintao Ma, Yi Wang, Chenghao Liu, Bin Yang, Zenglin Xu, et al. A survey on diffusion models for time series and spatio-temporal data. arXiv preprint arXiv:2404.18886, 2024. 2

    2024

  57. [65]

    Predicting Goal-Directed Human Attention Using In- verse Reinforcement Learning

    Zhibo Yang, Lihan Huang, Yupei Chen, Zijun Wei, Seoy- oung Ahn, Gregory Zelinsky, Dimitris Samaras, and Minh Hoai. Predicting Goal-Directed Human Attention Using In- verse Reinforcement Learning. In CVPR, 2020. 2, 5, 7

    2020

  58. [66]

    Predicting Goal-Directed Human Attention Using In- verse Reinforcement Learning

    Zhibo Yang, Lihan Huang, Yupei Chen, Zijun Wei, Seoy- oung Ahn, Gregory Zelinsky, Dimitris Samaras, and Minh Hoai. Predicting Goal-Directed Human Attention Using In- verse Reinforcement Learning. In CVPR, 2020. 2, 7, 8, 13

    2020

  59. [67]

    Target-absent human attention

    Zhibo Yang, Sounak Mondal, Seoyoung Ahn, Gregory Zelinsky, Minh Hoai, and Dimitris Samaras. Target-absent human attention. In ECCV, 2022. 2, 5, 8

    2022

  60. [68]

    Predicting Human Attention using Computational Attention

    Zhibo Yang, Sounak Mondal, Seoyoung Ahn, Gregory Zelinsky, Minh Hoai, and Dimitris Samaras. Predicting Human Attention using Computational Attention. arXiv preprint arXiv:2303.09383, 2023. 2, 5, 6

    2023 arXiv

  61. [69]

    Unify- ing top-down and bottom-up scanpath prediction using trans- formers

    Zhibo Yang, Sounak Mondal, Seoyoung Ahn, Ruoyu Xue, Gregory Zelinsky, Minh Hoai, and Dimitris Samaras. Unify- ing top-down and bottom-up scanpath prediction using trans- formers. In CVPR, 2024. 1, 2, 5, 6, 7, 8, 12

    2024

  62. [70]

    Gravita- tional laws of focus of attention

    Dario Zanca, Stefano Melacci, and Marco Gori. Gravita- tional laws of focus of attention. IEEE Trans. PAMI, 42(12): 2983–2995, 2020. 5, 6, 12

    2020

  63. [71]

    A theory of eye movements during target acquisition

    Gregory J Zelinsky. A theory of eye movements during target acquisition. Psychological review, 115(4):787, 2008. 8

    2008

  64. [72]

    Motiondif- fuse: Text-driven human motion generation with diffusion model

    Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, and Ziwei Liu. Motiondif- fuse: Text-driven human motion generation with diffusion model. IEEE Trans. PAMI, 46(6):4115–4128, 2024. 2

    2024

  65. [73]

    From cognition to computation: A comparative review of human attention and transformer architectures

    Minglu Zhao, Dehong Xu, and Tao Gao. From cognition to computation: A comparative review of human attention and transformer architectures. arXiv preprint arXiv:2407.01548,

    arXiv

  66. [74]

    DiffTraj: Generating GPS Trajectory with Diffusion Probabilistic Model

    Yuanshao Zhu, Yongchao Ye, Shiyao Zhang, Xiangyu Zhao, and James Yu. DiffTraj: Generating GPS Trajectory with Diffusion Probabilistic Model. In NeurIPS, 2023. 2

    2023

  67. [75]

    Controltraj: Controllable trajectory gen- eration with topology-constrained diffusion model

    Yuanshao Zhu, James Jianqiao Yu, Xiangyu Zhao, Qidong Liu, Yongchao Ye, Wei Chen, Zijian Zhang, Xuetao Wei, and Yuxuan Liang. Controltraj: Controllable trajectory gen- eration with topology-constrained diffusion model. In KDD,

  68. [2024]

    2 11 Modeling Human Gaze Behavior with Diffusion Models for Unified Scanpath Prediction Supplementary Material In the following, we provide additional results on the analysis of scanpath variability, demonstrating that ScanDiff outperforms existing methods in capturing the div...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.