Pith. sign in

REVIEW 3 major objections 2 minor 50 references

STARE: Predicting Decision Making Based on Spatio-Temporal Eye Movements

T0 review · 3 major / 2 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read STARE predicts consumer choices from raw eye-movement time series by mapping gaze onto image regions and processing the resulting token sequences with a time-series foundation model augmented by attention to gaze direction and binocular coo

desk verdict A plausible architecture for eye-tracking choice prediction, but the abstract alone can't show whether the ROI tokenization destroys the signal. read the letter →

arxiv 2508.04148 v1 pith:CF2YOBYX submitted 2025-08-06 cs.NE

classification cs.NE
keywords eyetrackingconsumerchoicepredictiontimeseriesfoundationmodelspatio-temporalattentiontokenizationgazedynamicsdeeplearningmechanism
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes STARE, a deep learning architecture that predicts consumer choices directly from raw eye-movement time series recorded while people view images of the decision environment. The core move is a tokenization that maps gaze positions onto predefined contiguous regions of interest, turning each moment of looking into a discrete token so the data can be processed by a time-series foundation model based on the transformer architecture. STARE adds co-attention and cross-attention to capture directional influences and interocular (two-eye) influences during viewing. The paper reports that STARE outperforms several state-of-the-art methods across multiple datasets. If correct, this is a first step toward foundation models for eye tracking and a way to turn gaze signals into predictions of real-world choices.

What carries the argument

ROI-based tokenization: raw eye-tracking coordinates at each time step are mapped to the predefined contiguous region of interest containing them, producing a discrete token per time step. This is what makes eye-movement time series compatible with a transformer-based time-series foundation model. Co-attention layers capture directional gaze dynamics—how the sequence of regions unfolds over time—and cross-attention layers capture interocular influences, i.e., the relationship between the two eyes' movements.

What would settle it

Compare STARE's ROI-tokenized input against a version that keeps continuous gaze coordinates on a task where choices depend on exact fixation position within a product image; if the continuous version predicts better, the ROI tokenization is discarding decision-relevant signal. Also, test a dataset with identical ROI sequences but different raw trajectories—if choices differ, ROI tokens alone cannot carry the deciding information.

Watch

Extended reading notes

Core claim

The paper claims that spatio-temporal eye movements contain enough signal to predict consumer choices and that this signal can be extracted by treating gaze as a sequence of region tokens. The proposed STARE architecture maps raw x-y pixel coordinates of gaze or fixations onto predefined contiguous Regions of Interest; the resulting token sequence is passed through a time-series foundation model built on the transformer architecture, with additional co-attention and cross-attention layers that model directional transitions of gaze and interocular influences. On multiple datasets, STARE is reported to outperform state-of-the-art alternatives at predicting consumer choice behaviors. The claim

Load-bearing premise

The method depends on the assumption that collapsing gaze positions into predefined contiguous regions of interest preserves the information needed to predict the choice; if exactly where inside a region a person looks matters, that signal is lost.

Editorial extensions

If this is right

  • Eye-movement time series can be treated as token sequences suitable for transformer-based foundation models, opening a route to pretrained models for gaze data.
  • Co-attention and cross-attention allow the model to capture directional and interocular gaze influences that standard single-stream models ignore.
  • Choice prediction accuracy improves across multiple datasets relative to state-of-the-art alternatives, suggesting generalizability beyond any single stimulus set.
  • The approach works directly on raw gaze or fixation time series with images of the decision environment, without hand-crafted features.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, a natural extension is to test a hybrid tokenization that concatenates ROI tokens with coarse continuous coordinates; this would reveal whether within-region gaze position carries residual predictive signal that the current discretization discards.
  • Implicit extension: the same ROI-tokenization pipeline could transfer to other gaze-based prediction problems, such as visual search, reading comprehension, or driver attention, because the architecture makes no consumer-choice-specific assumptions beyond the image stimulus.
  • Implicit extension: pretraining the time-series foundation model on large-scale eye-tracking data rather than generic time series would directly test how much of the gap to task-specific models is due to domain shift.
  • The cross-attention between eyes may yield a testable proxy for binocular coordination that can be compared with known neurophysiological signatures of vergence and saccade programming.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 2 minor

Summary. The manuscript proposes STARE (Spatio-Temporal Attention Representation for Eye Tracking), a deep-learning architecture for predicting consumer choice behaviors from eye-movement time series. The method tokenizes raw gaze coordinates by mapping them onto predefined, contiguous Regions of Interest (ROIs), then feeds these tokens to Chronos, a T5-based time-series foundation model, augmented with co-attention and/or cross-attention to capture directional and interocular influences. The abstract claims that STARE is compared with several state-of-the-art alternatives on multiple datasets, presumably outperforming them. Only the abstract is available; no methods details, experimental results, dataset descriptions, baseline numbers, error bars, or statistical analyses are presented.

Significance. If the claimed results are correct, STARE would represent a first foundational-model approach for eye-tracking-based choice prediction, with potential applications in consumer neuroscience and decision-making research. The architecture is well motivated: leveraging a pretrained time-series foundation model and a spatially meaningful tokenization is a reasonable design. The central claim is also falsifiable, which is a strength. However, because the submission contains no experimental evidence or technical detail, the significance cannot currently be assessed beyond the plausibility of the idea.

major comments (3)
  1. [Abstract] The central claim that STARE 'predicts various consumer choice behaviors' and 'compare[s] STARE with several state-of-the art alternatives on multiple datasets' is unsupported by any reported results. No dataset names, preprocessing steps, baseline descriptions, performance metrics, error bars, significance tests, or ablations are provided. This is the load-bearing empirical assertion of the paper and must be fully documented before the claim can be evaluated.
  2. [Abstract (tokenization)] The tokenization strategy maps continuous x-y pixel coordinates onto predefined, contiguous ROIs. The abstract does not state how these ROIs are defined. If ROIs are derived using outcome information or test-set gaze data, this would introduce target leakage. Even if ROIs are defined a priori, the mapping discards all within-ROI spatial information; the paper must justify that sub-ROI gaze position carries no decision-relevant signal, or provide an ablation (e.g., adding coordinate residuals) demonstrating that ROI granularity is sufficient.
  3. [Abstract (Chronos adaptation)] Chronos is a foundation model pretrained on univariate scalar time series, whereas STARE feeds it categorical ROI tokens. The abstract does not explain how this domain shift is handled, e.g., whether the token embeddings are initialized from Chronos's vocabulary or learned from scratch, and whether pretrained weights are fine-tuned. Without this information, the purported advantage of using a foundation model is not established. The authors should report experiments isolating the contribution of the pretrained initialization versus the attention architecture.
minor comments (2)
  1. [Abstract] The phrases 'co-attention and/or cross-attention' and 'directional and/or interocular influences' are not defined. Please specify the exact attention mechanisms and how directionality and interocular coupling are encoded.
  2. [Abstract] The term 'raw gaze or eye fixations' is ambiguous: raw gaze is typically a continuous stream, while fixations are segmented events. The paper should clarify which signal is used as input to the model.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity identified: the abstract makes an empirical prediction claim with no derivation chain that reduces to its inputs.

full rationale

The available manuscript text is only the abstract. It describes a deep learning architecture (STARE) that tokenizes eye-movement coordinates into predefined ROIs and feeds these to Chronos, with co-attention/cross-attention, then compares against state-of-the-art alternatives on multiple datasets. There are no equations, no fitted parameters renamed as predictions, no uniqueness theorem imported from prior work, and no self-citation chain invoked as load-bearing support. The tokenization step is described as mapping raw x-y coordinates onto predefined ROIs, but nothing in the provided text indicates that these ROIs are derived from the outcome variable or from the test set; that would be a data-leakage/validity concern, not circularity under the specified criteria. Likewise, the architecture's dependence on ROI granularity is an empirical assumption, not a circular reduction. Since the paper is not available beyond the abstract, there is no exhibited derivation for any claim, and therefore no basis for a circularity finding. Score 0 reflects the absence of circularity evidence, not a judgment on the method's correctness.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free numerical parameters are identifiable from the abstract, and no new physical or conceptual entities are introduced. The method rests on unstated assumptions about the informativeness of ROI-discretized gaze, the transferability of Chronos, and the efficacy of attention mechanisms for interocular dynamics. These are domain assumptions without independent evidence in the abstract.

assumptions (4)
  • domain assumption Eye movement patterns during viewing carry information predictive of subsequent choice.
    The entire prediction task assumes a stable mapping from fixation and gaze dynamics to choice outcomes. This is a background assumption from consumer psychology literature, not established within the abstract.
  • ad hoc to paper Mapping continuous gaze coordinates to predefined, contiguous ROIs preserves decision-relevant spatial information.
    The abstract states the tokenization maps x-y pixel coordinates onto predefined ROIs. This coarse graining is assumed lossless enough for choice prediction, but no information-loss analysis is presented.
  • domain assumption The Chronos time-series foundation model, pretrained on generic time-series data, transfers to eye-movement time series.
    The architecture depends on adapting Chronos to a novel modality. The abstract provides no evidence of domain alignment or fine-tuning results.
  • domain assumption Co-attention and cross-attention can capture directional and interocular influences in eye movements.
    The paper claims these attention mechanisms model directional and interocular effects, but the abstract does not validate this claim. It is an architectural assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of STARE: Predicting Decision Making Based on Spatio-Temporal Eye Movements." pith.science (2026). https://pith.science/paper/CF2YOBYX

@misc{pith2026250804148,
  author       = {Pith},
  title        = {Pith review of: STARE: Predicting Decision Making Based on Spatio-Temporal Eye Movements},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CF2YOBYX}},
  note         = {Machine review of arXiv:2508.04148}
}
read the original abstract

The present work proposes a Deep Learning architecture for the prediction of various consumer choice behaviors from time series of raw gaze or eye fixations on images of the decision environment, for which currently no foundational models are available. The architecture, called STARE (Spatio-Temporal Attention Representation for Eye Tracking), uses a new tokenization strategy, which involves mapping the x- and y- pixel coordinates of eye-movement time series on predefined, contiguous Regions of Interest. That tokenization makes the spatio-temporal eye-movement data available to the Chronos, a time-series foundation model based on the T5 architecture, to which co-attention and/or cross-attention is added to capture directional and/or interocular influences of eye movements. We compare STARE with several state-of-the art alternatives on multiple datasets with the purpose of predicting consumer choice behaviors from eye movements. We thus make a first step towards developing and testing DL architectures that represent visual attention dynamics rooted in the neurophysiology of eye movements.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

50 extracted references · 40 canonical work pages

  1. [1]

    Martinovici, R

    A. Martinovici, R. Pieters, and T. Erdem, ``Attention trajectories capture utility accumulation and predict brand choice,'' Journal of Marketing Research, vol. 60, no. 4, pp. 625--645, 2023

  2. [2]

    Unger, M

    M. Unger, M. Wedel, and A. Tuzhilin, ``Predicting consumer choice from raw eye-movement data using the retina deep learning architecture,'' Data Mining and Knowledge Discovery, vol. 38, no. 3, pp. 1069--1100, 2024

  3. [3]

    J. L. Orquin and S. M. Loose, ``Attention and choice: A review on eye movements in decision making,'' Acta psychologica, vol. 144, no. 1, pp. 190--206, 2013

  4. [4]

    Wedel, R

    M. Wedel, R. Pieters, and R. van der Lans, ``Modeling eye movements during decision making: A review,'' Psychometrika, vol. 88, no. 2, pp. 697--729, 2023

  5. [5]

    S. A. Byrne, A. P. F. Reynolds, C. Biliotti, F. J. Bargagli-Stoffi, L. Polonio, and M. Riccaboni, ``Predicting choice behaviour in economic games using gaze data encoded as scanpath images,'' Scientific Reports, vol. 13, no. 1, p. 4722, 2023

  6. [6]

    Polonio, S

    L. Polonio, S. Di Guida, and G. Coricelli, ``Strategic sophistication and attention in games: An eye-tracking study,'' Games and Economic Behavior, vol. 94, pp. 80--96, 2015

  7. [7]

    S. W. Shi, M. Wedel, and F. Pieters, ``Information acquisition during online decision making: A model-based exploration using eye-tracking data,'' Management Science, vol. 59, no. 5, pp. 1009--1026, 2013

  8. [8]

    A. F. Ansari, L. Stella, C. Turkmen, X. Zhang, P. Mercado, H. Shen, O. Shchur, S. S. Rangapuram, S. P. Arango, S. Kapoor et al., ``Chronos: Learning the language of time series,'' arXiv preprint arXiv:2403.07815, 2024

Show all 50 references
  1. [9]

    R. Ma, X. Qiu, Q. Zhang, X. Hu, Y.-G. Jiang, and X. Huang, ``Co-attention memory network for multimodal microblog's hashtag recommendation,'' IEEE Transactions on Knowledge and Data Engineering, vol. 33, no. 2, pp. 388--400, 2019

  2. [10]

    Wedel, R

    M. Wedel, R. Pieters et al., ``Eye tracking for visual marketing,'' Foundations and Trends in Marketing , vol. 1, no. 4, pp. 231--320, 2008

  3. [11]

    D. L. Sparks, ``The brainstem control of saccadic eye movements,'' Nature Reviews Neuroscience, vol. 3, no. 12, pp. 952--964, 2002

  4. [12]

    King, ``Binocular coordination of eye movements--hering’s law of equal innervation or uniocular control?'' European Journal of Neuroscience, vol

    W. King, ``Binocular coordination of eye movements--hering’s law of equal innervation or uniocular control?'' European Journal of Neuroscience, vol. 33, no. 11, pp. 2139--2146, 2011

  5. [13]

    J. Tagu, K. Dor \'e -Mazars, C. Lemoine-Lardennois, and D. Vergilino-Perez, ``How eye dominance strength modulates the influence of a distractor on saccade accuracy,'' Investigative Ophthalmology & Visual Science, vol. 57, no. 2, pp. 534--543, 2016

  6. [14]

    Vaswani, ``Attention is all you need,'' Advances in Neural Information Processing Systems, 2017

    A. Vaswani, ``Attention is all you need,'' Advances in Neural Information Processing Systems, 2017

  7. [15]

    Liang, H

    Y. Liang, H. Wen, Y. Nie, Y. Jiang, M. Jin, D. Song, S. Pan, and Q. Wen, ``Foundation models for time series analysis: A tutorial and survey,'' in Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining, 2024, pp. 6555--6565

  8. [16]

    Q. Ma, Z. Liu, Z. Zheng, Z. Huang, S. Zhu, Z. Yu, and J. T. Kwok, ``A survey on time-series pre-trained models,'' IEEE Transactions on Knowledge and Data Engineering, 2024

  9. [17]

    Z. Shao, F. Wang, Y. Xu, W. Wei, C. Yu, Z. Zhang, D. Yao, T. Sun, G. Jin, X. Cao et al., ``Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis,'' IEEE Transactions on Knowledge and Data Engineering, 2024

  10. [18]

    S. Wang, J. Cao, and S. Y. Philip, ``Deep learning for spatio-temporal data mining: A survey,'' IEEE transactions on knowledge and data engineering, vol. 34, no. 8, pp. 3681--3700, 2020

  11. [19]

    S. S. Rangapuram, M. W. Seeger, J. Gasthaus, L. Stella, Y. Wang, and T. Januschowski, ``Deep state space models for time series forecasting,'' Advances in neural information processing systems, vol. 31, 2018

  12. [20]

    Y. Wang, A. Smola, D. Maddix, J. Gasthaus, D. Foster, and T. Januschowski, ``Deep factors for forecasting,'' in International conference on machine learning. 1em plus 0.5em minus 0.4em PMLR, 2019, pp. 6607--6617

  13. [21]

    Salinas, V

    D. Salinas, V. Flunkert, J. Gasthaus, and T. Januschowski, ``Deepar: Probabilistic forecasting with autoregressive recurrent networks,'' International journal of forecasting, vol. 36, no. 3, pp. 1181--1191, 2020

  14. [22]

    J. Dong, H. Wu, H. Zhang, L. Zhang, J. Wang, and M. Long, ``Simmtm: A simple pre-training framework for masked time-series modeling,'' Advances in Neural Information Processing Systems, vol. 36, 2024

  15. [23]

    H. Wu, T. Hu, Y. Liu, H. Zhou, J. Wang, and M. Long, ``Timesnet: Temporal 2d-variation modeling for general time series analysis,'' arXiv preprint arXiv:2210.02186, 2022

  16. [24]

    Zhang, Z

    X. Zhang, Z. Zhao, T. Tsiligkaridis, and M. Zitnik, ``Self-supervised contrastive pre-training for time series via time-frequency consistency,'' Advances in Neural Information Processing Systems, vol. 35, pp. 3988--4003, 2022

  17. [25]

    Z. Wang, Q. Wen, C. Zhang, L. Sun, and Y. Wang, ``Diffload: uncertainty quantification in load forecasting with diffusion model,'' arXiv preprint arXiv:2306.01001, 2023

  18. [26]

    H. Wen, Y. Lin, Y. Xia, H. Wan, Q. Wen, R. Zimmermann, and Y. Liang, ``Diffstg: Probabilistic spatio-temporal graph forecasting with denoising diffusion models,'' in Proceedings of the 31st ACM International Conference on Advances in Geographic Information Systems, 2023, pp. 1--12

  19. [27]

    B. Lim, S. \"O . Ar k, N. Loeff, and T. Pfister, ``Temporal fusion transformers for interpretable multi-horizon time series forecasting,'' International Journal of Forecasting, vol. 37, no. 4, pp. 1748--1764, 2021

  20. [28]

    Y. Nie, N. H. Nguyen, P. Sinthong, and J. Kalagnanam, ``A time series is worth 64 words: Long-term forecasting with transformers,'' arXiv preprint arXiv:2211.14730, 2022

  21. [29]

    B. Wang, B. Liang, L. Zhou, and R. Xu, ``Gaze-infused bert: Do human gaze signals help pre-trained language models?'' Neural Computing and Applications, vol. 36, no. 20, pp. 12\,461--12\,482, 2024

  22. [30]

    Rasul, A

    K. Rasul, A. Ashok, A. R. Williams, A. Khorasani, G. Adamopoulos, R. Bhagwatkar, M. Bilo s , H. Ghonia, N. V. Hassen, A. Schneider et al., ``Lag-llama: Towards foundation models for time series forecasting,'' arXiv preprint arXiv:2310.08278, 2023

  23. [31]

    A. Das, W. Kong, R. Sen, and Y. Zhou, ``A decoder-only foundation model for time-series forecasting,'' arXiv preprint arXiv:2310.10688, 2023

  24. [32]

    G. Woo, C. Liu, A. Kumar, C. Xiong, S. Savarese, and D. Sahoo, ``Unified training of universal time series forecasting transformers,'' arXiv preprint arXiv:2402.02592, 2024

  25. [33]

    T. Zhou, P. Niu, L. Sun, R. Jin et al., ``One fits all: Power general time series analysis by pretrained lm,'' Advances in neural information processing systems, vol. 36, pp. 43\,322--43\,355, 2023

  26. [34]

    M. Jin, S. Wang, L. Ma, Z. Chu, J. Y. Zhang, X. Shi, P.-Y. Chen, Y. Liang, Y.-F. Li, S. Pan et al., ``Time-llm: Time series forecasting by reprogramming large language models,'' arXiv preprint arXiv:2310.01728, 2023

  27. [35]

    Raffel, N

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, ``Exploring the limits of transfer learning with a unified text-to-text transformer,'' Journal of machine learning research, vol. 21, no. 140, pp. 1--67, 2020

  28. [36]

    Bhogade and B

    V. Bhogade and B. Nithya, ``Time series forecasting using transformer neural network,'' International Journal of Computers and Applications, vol. 46, no. 10, pp. 880--888, 2024

  29. [37]

    C. Yang, Y. Wang, B. Yang, and J. Chen, ``Graformer: A gated residual attention transformer for multivariate time series forecasting,'' Neurocomputing, vol. 581, p. 127466, 2024

  30. [38]

    Cheng, X

    X. Cheng, X. Chen, S. Li, D. Luo, X. Wang, D. Zhao, and R. Yan, ``Leveraging 2d information for long-term time series forecasting with vanilla transformers,'' arXiv preprint arXiv:2405.13810, 2024

  31. [39]

    Krol and M

    M. Krol and M. Krol, ``A novel approach to studying strategic decisions with eye-tracking and machine learning,'' Judgment and Decision Making, vol. 12, no. 6, p. 596, 2017

  32. [40]

    Palacios-Ib \'a \ n ez, J

    A. Palacios-Ib \'a \ n ez, J. Mar \' n-Morales, M. Contero, and M. Alca \ n iz, ``Predicting decision-making in virtual environments: an eye movement analysis with household products,'' Applied Sciences, vol. 13, no. 12, p. 7124, 2023

  33. [41]

    Simonyan and A

    K. Simonyan and A. Zisserman, ``Very deep convolutional networks for large-scale image recognition,'' arXiv preprint arXiv:1409.1556, 2014

  34. [42]

    Devlin, ``Bert: Pre-training of deep bidirectional transformers for language understanding,'' arXiv preprint arXiv:1810.04805, 2018

    J. Devlin, ``Bert: Pre-training of deep bidirectional transformers for language understanding,'' arXiv preprint arXiv:1810.04805, 2018

  35. [43]

    S. D. Sims and C. Conati, ``A neural architecture for detecting user confusion in eye-tracking data,'' in Proceedings of the 2020 international conference on multimodal interaction, 2020, pp. 15--23

  36. [44]

    Sriram, C

    H. Sriram, C. Conati, and T. Field, ``Classification of alzheimer's disease with deep learning on eye-tracking data,'' in Proceedings of the 25th International Conference on Multimodal Interaction, 2023, pp. 104--113

  37. [45]

    J. L. Orquin, N. J. Ashby, and A. D. Clarke, ``Areas of interest as a signal detection problem in behavioral eye-tracking research,'' Journal of Behavioral Decision Making, vol. 29, no. 2-3, pp. 103--115, 2016

  38. [46]

    S. M. Munn and J. B. Pelz, ``Fixtag: An algorithm for identifying and tagging fixations to simplify the analysis of data collected by portable eye trackers,'' ACM Transactions on Applied Perception (TAP), vol. 6, no. 3, pp. 1--25, 2009

  39. [47]

    Gundimeda, R

    V. Gundimeda, R. S. Murali, R. Joseph, and N. Naresh Babu, ``An automated computer vision system for extraction of retail food product metadata,'' in First International Conference on Artificial Intelligence and Cognitive Computing: AICC 2018. 1em plus 0.5em minus 0.4em Spring...

  40. [48]

    Hochreiter, ``Long short-term memory,'' Neural Computation MIT-Press, 1997

    S. Hochreiter, ``Long short-term memory,'' Neural Computation MIT-Press, 1997

  41. [49]

    A. K. Horn and H. Straka, ``Functional organization of extraocular motoneurons and eye muscles,'' Annual review of vision science, vol. 7, no. 1, pp. 793--825, 2021

  42. [50]

    Shimojo, C

    S. Shimojo, C. Simion, E. Shimojo, and C. Scheier, ``Gaze bias both reflects and influences preference,'' Nature neuroscience, vol. 6, no. 12, pp. 1317--1322, 2003

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.