REVIEW 3 major objections 2 minor 50 references
STARE: Predicting Decision Making Based on Spatio-Temporal Eye Movements
T0 review · 3 major / 2 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read STARE predicts consumer choices from raw eye-movement time series by mapping gaze onto image regions and processing the resulting token sequences with a time-series foundation model augmented by attention to gaze direction and binocular coo
desk verdict A plausible architecture for eye-tracking choice prediction, but the abstract alone can't show whether the ROI tokenization destroys the signal. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
ROI-based tokenization: raw eye-tracking coordinates at each time step are mapped to the predefined contiguous region of interest containing them, producing a discrete token per time step. This is what makes eye-movement time series compatible with a transformer-based time-series foundation model. Co-attention layers capture directional gaze dynamics—how the sequence of regions unfolds over time—and cross-attention layers capture interocular influences, i.e., the relationship between the two eyes' movements.
What would settle it
Compare STARE's ROI-tokenized input against a version that keeps continuous gaze coordinates on a task where choices depend on exact fixation position within a product image; if the continuous version predicts better, the ROI tokenization is discarding decision-relevant signal. Also, test a dataset with identical ROI sequences but different raw trajectories—if choices differ, ROI tokens alone cannot carry the deciding information.
Extended reading notes
Core claim
The paper claims that spatio-temporal eye movements contain enough signal to predict consumer choices and that this signal can be extracted by treating gaze as a sequence of region tokens. The proposed STARE architecture maps raw x-y pixel coordinates of gaze or fixations onto predefined contiguous Regions of Interest; the resulting token sequence is passed through a time-series foundation model built on the transformer architecture, with additional co-attention and cross-attention layers that model directional transitions of gaze and interocular influences. On multiple datasets, STARE is reported to outperform state-of-the-art alternatives at predicting consumer choice behaviors. The claim
Load-bearing premise
The method depends on the assumption that collapsing gaze positions into predefined contiguous regions of interest preserves the information needed to predict the choice; if exactly where inside a region a person looks matters, that signal is lost.
Editorial extensions
If this is right
- Eye-movement time series can be treated as token sequences suitable for transformer-based foundation models, opening a route to pretrained models for gaze data.
- Co-attention and cross-attention allow the model to capture directional and interocular gaze influences that standard single-stream models ignore.
- Choice prediction accuracy improves across multiple datasets relative to state-of-the-art alternatives, suggesting generalizability beyond any single stimulus set.
- The approach works directly on raw gaze or fixation time series with images of the decision environment, without hand-crafted features.
Reading between the lines
- Beyond the paper, a natural extension is to test a hybrid tokenization that concatenates ROI tokens with coarse continuous coordinates; this would reveal whether within-region gaze position carries residual predictive signal that the current discretization discards.
- Implicit extension: the same ROI-tokenization pipeline could transfer to other gaze-based prediction problems, such as visual search, reading comprehension, or driver attention, because the architecture makes no consumer-choice-specific assumptions beyond the image stimulus.
- Implicit extension: pretraining the time-series foundation model on large-scale eye-tracking data rather than generic time series would directly test how much of the gap to task-specific models is due to domain shift.
- The cross-attention between eyes may yield a testable proxy for binocular coordination that can be compared with known neurophysiological signatures of vergence and saccade programming.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes STARE (Spatio-Temporal Attention Representation for Eye Tracking), a deep-learning architecture for predicting consumer choice behaviors from eye-movement time series. The method tokenizes raw gaze coordinates by mapping them onto predefined, contiguous Regions of Interest (ROIs), then feeds these tokens to Chronos, a T5-based time-series foundation model, augmented with co-attention and/or cross-attention to capture directional and interocular influences. The abstract claims that STARE is compared with several state-of-the-art alternatives on multiple datasets, presumably outperforming them. Only the abstract is available; no methods details, experimental results, dataset descriptions, baseline numbers, error bars, or statistical analyses are presented.
Significance. If the claimed results are correct, STARE would represent a first foundational-model approach for eye-tracking-based choice prediction, with potential applications in consumer neuroscience and decision-making research. The architecture is well motivated: leveraging a pretrained time-series foundation model and a spatially meaningful tokenization is a reasonable design. The central claim is also falsifiable, which is a strength. However, because the submission contains no experimental evidence or technical detail, the significance cannot currently be assessed beyond the plausibility of the idea.
major comments (3)
- [Abstract] The central claim that STARE 'predicts various consumer choice behaviors' and 'compare[s] STARE with several state-of-the art alternatives on multiple datasets' is unsupported by any reported results. No dataset names, preprocessing steps, baseline descriptions, performance metrics, error bars, significance tests, or ablations are provided. This is the load-bearing empirical assertion of the paper and must be fully documented before the claim can be evaluated.
- [Abstract (tokenization)] The tokenization strategy maps continuous x-y pixel coordinates onto predefined, contiguous ROIs. The abstract does not state how these ROIs are defined. If ROIs are derived using outcome information or test-set gaze data, this would introduce target leakage. Even if ROIs are defined a priori, the mapping discards all within-ROI spatial information; the paper must justify that sub-ROI gaze position carries no decision-relevant signal, or provide an ablation (e.g., adding coordinate residuals) demonstrating that ROI granularity is sufficient.
- [Abstract (Chronos adaptation)] Chronos is a foundation model pretrained on univariate scalar time series, whereas STARE feeds it categorical ROI tokens. The abstract does not explain how this domain shift is handled, e.g., whether the token embeddings are initialized from Chronos's vocabulary or learned from scratch, and whether pretrained weights are fine-tuned. Without this information, the purported advantage of using a foundation model is not established. The authors should report experiments isolating the contribution of the pretrained initialization versus the attention architecture.
minor comments (2)
- [Abstract] The phrases 'co-attention and/or cross-attention' and 'directional and/or interocular influences' are not defined. Please specify the exact attention mechanisms and how directionality and interocular coupling are encoded.
- [Abstract] The term 'raw gaze or eye fixations' is ambiguous: raw gaze is typically a continuous stream, while fixations are segmented events. The paper should clarify which signal is used as input to the model.
Circularity Check
No circularity identified: the abstract makes an empirical prediction claim with no derivation chain that reduces to its inputs.
full rationale
The available manuscript text is only the abstract. It describes a deep learning architecture (STARE) that tokenizes eye-movement coordinates into predefined ROIs and feeds these to Chronos, with co-attention/cross-attention, then compares against state-of-the-art alternatives on multiple datasets. There are no equations, no fitted parameters renamed as predictions, no uniqueness theorem imported from prior work, and no self-citation chain invoked as load-bearing support. The tokenization step is described as mapping raw x-y coordinates onto predefined ROIs, but nothing in the provided text indicates that these ROIs are derived from the outcome variable or from the test set; that would be a data-leakage/validity concern, not circularity under the specified criteria. Likewise, the architecture's dependence on ROI granularity is an empirical assumption, not a circular reduction. Since the paper is not available beyond the abstract, there is no exhibited derivation for any claim, and therefore no basis for a circularity finding. Score 0 reflects the absence of circularity evidence, not a judgment on the method's correctness.
Assumptions & free parameters
assumptions (4)
- domain assumption Eye movement patterns during viewing carry information predictive of subsequent choice.
- ad hoc to paper Mapping continuous gaze coordinates to predefined, contiguous ROIs preserves decision-relevant spatial information.
- domain assumption The Chronos time-series foundation model, pretrained on generic time-series data, transfers to eye-movement time series.
- domain assumption Co-attention and cross-attention can capture directional and interocular influences in eye movements.
Cite this review
Pith. "Pith review of STARE: Predicting Decision Making Based on Spatio-Temporal Eye Movements." pith.science (2026). https://pith.science/paper/CF2YOBYX
@misc{pith2026250804148,
author = {Pith},
title = {Pith review of: STARE: Predicting Decision Making Based on Spatio-Temporal Eye Movements},
year = {2026},
howpublished = {\url{https://pith.science/paper/CF2YOBYX}},
note = {Machine review of arXiv:2508.04148}
}
read the original abstract
The present work proposes a Deep Learning architecture for the prediction of various consumer choice behaviors from time series of raw gaze or eye fixations on images of the decision environment, for which currently no foundational models are available. The architecture, called STARE (Spatio-Temporal Attention Representation for Eye Tracking), uses a new tokenization strategy, which involves mapping the x- and y- pixel coordinates of eye-movement time series on predefined, contiguous Regions of Interest. That tokenization makes the spatio-temporal eye-movement data available to the Chronos, a time-series foundation model based on the T5 architecture, to which co-attention and/or cross-attention is added to capture directional and/or interocular influences of eye movements. We compare STARE with several state-of-the art alternatives on multiple datasets with the purpose of predicting consumer choice behaviors from eye movements. We thus make a first step towards developing and testing DL architectures that represent visual attention dynamics rooted in the neurophysiology of eye movements.
Reference graph
Works this paper leans on
-
[1]
A. Martinovici, R. Pieters, and T. Erdem, ``Attention trajectories capture utility accumulation and predict brand choice,'' Journal of Marketing Research, vol. 60, no. 4, pp. 625--645, 2023
work page 2023
- [2]
-
[3]
J. L. Orquin and S. M. Loose, ``Attention and choice: A review on eye movements in decision making,'' Acta psychologica, vol. 144, no. 1, pp. 190--206, 2013
work page 2013
- [4]
-
[5]
S. A. Byrne, A. P. F. Reynolds, C. Biliotti, F. J. Bargagli-Stoffi, L. Polonio, and M. Riccaboni, ``Predicting choice behaviour in economic games using gaze data encoded as scanpath images,'' Scientific Reports, vol. 13, no. 1, p. 4722, 2023
work page 2023
-
[6]
L. Polonio, S. Di Guida, and G. Coricelli, ``Strategic sophistication and attention in games: An eye-tracking study,'' Games and Economic Behavior, vol. 94, pp. 80--96, 2015
work page 2015
-
[7]
S. W. Shi, M. Wedel, and F. Pieters, ``Information acquisition during online decision making: A model-based exploration using eye-tracking data,'' Management Science, vol. 59, no. 5, pp. 1009--1026, 2013
work page 2013
-
[8]
A. F. Ansari, L. Stella, C. Turkmen, X. Zhang, P. Mercado, H. Shen, O. Shchur, S. S. Rangapuram, S. P. Arango, S. Kapoor et al., ``Chronos: Learning the language of time series,'' arXiv preprint arXiv:2403.07815, 2024
arXiv 2024
Show all 50 references
-
[9]
R. Ma, X. Qiu, Q. Zhang, X. Hu, Y.-G. Jiang, and X. Huang, ``Co-attention memory network for multimodal microblog's hashtag recommendation,'' IEEE Transactions on Knowledge and Data Engineering, vol. 33, no. 2, pp. 388--400, 2019
2019
-
[10]
Wedel, R
M. Wedel, R. Pieters et al., ``Eye tracking for visual marketing,'' Foundations and Trends in Marketing , vol. 1, no. 4, pp. 231--320, 2008
2008
-
[11]
D. L. Sparks, ``The brainstem control of saccadic eye movements,'' Nature Reviews Neuroscience, vol. 3, no. 12, pp. 952--964, 2002
2002
-
[12]
King, ``Binocular coordination of eye movements--hering’s law of equal innervation or uniocular control?'' European Journal of Neuroscience, vol
W. King, ``Binocular coordination of eye movements--hering’s law of equal innervation or uniocular control?'' European Journal of Neuroscience, vol. 33, no. 11, pp. 2139--2146, 2011
2011
-
[13]
J. Tagu, K. Dor \'e -Mazars, C. Lemoine-Lardennois, and D. Vergilino-Perez, ``How eye dominance strength modulates the influence of a distractor on saccade accuracy,'' Investigative Ophthalmology & Visual Science, vol. 57, no. 2, pp. 534--543, 2016
2016
-
[14]
Vaswani, ``Attention is all you need,'' Advances in Neural Information Processing Systems, 2017
A. Vaswani, ``Attention is all you need,'' Advances in Neural Information Processing Systems, 2017
2017
-
[15]
Liang, H
Y. Liang, H. Wen, Y. Nie, Y. Jiang, M. Jin, D. Song, S. Pan, and Q. Wen, ``Foundation models for time series analysis: A tutorial and survey,'' in Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining, 2024, pp. 6555--6565
2024
-
[16]
Q. Ma, Z. Liu, Z. Zheng, Z. Huang, S. Zhu, Z. Yu, and J. T. Kwok, ``A survey on time-series pre-trained models,'' IEEE Transactions on Knowledge and Data Engineering, 2024
2024
-
[17]
Z. Shao, F. Wang, Y. Xu, W. Wei, C. Yu, Z. Zhang, D. Yao, T. Sun, G. Jin, X. Cao et al., ``Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis,'' IEEE Transactions on Knowledge and Data Engineering, 2024
2024
-
[18]
S. Wang, J. Cao, and S. Y. Philip, ``Deep learning for spatio-temporal data mining: A survey,'' IEEE transactions on knowledge and data engineering, vol. 34, no. 8, pp. 3681--3700, 2020
2020
-
[19]
S. S. Rangapuram, M. W. Seeger, J. Gasthaus, L. Stella, Y. Wang, and T. Januschowski, ``Deep state space models for time series forecasting,'' Advances in neural information processing systems, vol. 31, 2018
2018
-
[20]
Y. Wang, A. Smola, D. Maddix, J. Gasthaus, D. Foster, and T. Januschowski, ``Deep factors for forecasting,'' in International conference on machine learning. 1em plus 0.5em minus 0.4em PMLR, 2019, pp. 6607--6617
2019
-
[21]
Salinas, V
D. Salinas, V. Flunkert, J. Gasthaus, and T. Januschowski, ``Deepar: Probabilistic forecasting with autoregressive recurrent networks,'' International journal of forecasting, vol. 36, no. 3, pp. 1181--1191, 2020
2020
-
[22]
J. Dong, H. Wu, H. Zhang, L. Zhang, J. Wang, and M. Long, ``Simmtm: A simple pre-training framework for masked time-series modeling,'' Advances in Neural Information Processing Systems, vol. 36, 2024
2024
-
[23]
H. Wu, T. Hu, Y. Liu, H. Zhou, J. Wang, and M. Long, ``Timesnet: Temporal 2d-variation modeling for general time series analysis,'' arXiv preprint arXiv:2210.02186, 2022
2022 arXiv
-
[24]
Zhang, Z
X. Zhang, Z. Zhao, T. Tsiligkaridis, and M. Zitnik, ``Self-supervised contrastive pre-training for time series via time-frequency consistency,'' Advances in Neural Information Processing Systems, vol. 35, pp. 3988--4003, 2022
2022
-
[25]
Z. Wang, Q. Wen, C. Zhang, L. Sun, and Y. Wang, ``Diffload: uncertainty quantification in load forecasting with diffusion model,'' arXiv preprint arXiv:2306.01001, 2023
2023 arXiv
-
[26]
H. Wen, Y. Lin, Y. Xia, H. Wan, Q. Wen, R. Zimmermann, and Y. Liang, ``Diffstg: Probabilistic spatio-temporal graph forecasting with denoising diffusion models,'' in Proceedings of the 31st ACM International Conference on Advances in Geographic Information Systems, 2023, pp. 1--12
2023
-
[27]
B. Lim, S. \"O . Ar k, N. Loeff, and T. Pfister, ``Temporal fusion transformers for interpretable multi-horizon time series forecasting,'' International Journal of Forecasting, vol. 37, no. 4, pp. 1748--1764, 2021
2021
-
[28]
Y. Nie, N. H. Nguyen, P. Sinthong, and J. Kalagnanam, ``A time series is worth 64 words: Long-term forecasting with transformers,'' arXiv preprint arXiv:2211.14730, 2022
2022 arXiv
-
[29]
B. Wang, B. Liang, L. Zhou, and R. Xu, ``Gaze-infused bert: Do human gaze signals help pre-trained language models?'' Neural Computing and Applications, vol. 36, no. 20, pp. 12\,461--12\,482, 2024
2024
-
[30]
Rasul, A
K. Rasul, A. Ashok, A. R. Williams, A. Khorasani, G. Adamopoulos, R. Bhagwatkar, M. Bilo s , H. Ghonia, N. V. Hassen, A. Schneider et al., ``Lag-llama: Towards foundation models for time series forecasting,'' arXiv preprint arXiv:2310.08278, 2023
2023 arXiv
-
[31]
A. Das, W. Kong, R. Sen, and Y. Zhou, ``A decoder-only foundation model for time-series forecasting,'' arXiv preprint arXiv:2310.10688, 2023
2023 arXiv
-
[32]
G. Woo, C. Liu, A. Kumar, C. Xiong, S. Savarese, and D. Sahoo, ``Unified training of universal time series forecasting transformers,'' arXiv preprint arXiv:2402.02592, 2024
2024 arXiv
-
[33]
T. Zhou, P. Niu, L. Sun, R. Jin et al., ``One fits all: Power general time series analysis by pretrained lm,'' Advances in neural information processing systems, vol. 36, pp. 43\,322--43\,355, 2023
2023
-
[34]
M. Jin, S. Wang, L. Ma, Z. Chu, J. Y. Zhang, X. Shi, P.-Y. Chen, Y. Liang, Y.-F. Li, S. Pan et al., ``Time-llm: Time series forecasting by reprogramming large language models,'' arXiv preprint arXiv:2310.01728, 2023
2023 arXiv
-
[35]
Raffel, N
C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, ``Exploring the limits of transfer learning with a unified text-to-text transformer,'' Journal of machine learning research, vol. 21, no. 140, pp. 1--67, 2020
2020
-
[36]
Bhogade and B
V. Bhogade and B. Nithya, ``Time series forecasting using transformer neural network,'' International Journal of Computers and Applications, vol. 46, no. 10, pp. 880--888, 2024
2024
-
[37]
C. Yang, Y. Wang, B. Yang, and J. Chen, ``Graformer: A gated residual attention transformer for multivariate time series forecasting,'' Neurocomputing, vol. 581, p. 127466, 2024
2024
-
[38]
Cheng, X
X. Cheng, X. Chen, S. Li, D. Luo, X. Wang, D. Zhao, and R. Yan, ``Leveraging 2d information for long-term time series forecasting with vanilla transformers,'' arXiv preprint arXiv:2405.13810, 2024
2024 arXiv
-
[39]
Krol and M
M. Krol and M. Krol, ``A novel approach to studying strategic decisions with eye-tracking and machine learning,'' Judgment and Decision Making, vol. 12, no. 6, p. 596, 2017
2017
-
[40]
Palacios-Ib \'a \ n ez, J
A. Palacios-Ib \'a \ n ez, J. Mar \' n-Morales, M. Contero, and M. Alca \ n iz, ``Predicting decision-making in virtual environments: an eye movement analysis with household products,'' Applied Sciences, vol. 13, no. 12, p. 7124, 2023
2023
-
[41]
Simonyan and A
K. Simonyan and A. Zisserman, ``Very deep convolutional networks for large-scale image recognition,'' arXiv preprint arXiv:1409.1556, 2014
2014 arXiv
-
[42]
Devlin, ``Bert: Pre-training of deep bidirectional transformers for language understanding,'' arXiv preprint arXiv:1810.04805, 2018
J. Devlin, ``Bert: Pre-training of deep bidirectional transformers for language understanding,'' arXiv preprint arXiv:1810.04805, 2018
2018 arXiv
-
[43]
S. D. Sims and C. Conati, ``A neural architecture for detecting user confusion in eye-tracking data,'' in Proceedings of the 2020 international conference on multimodal interaction, 2020, pp. 15--23
2020
-
[44]
Sriram, C
H. Sriram, C. Conati, and T. Field, ``Classification of alzheimer's disease with deep learning on eye-tracking data,'' in Proceedings of the 25th International Conference on Multimodal Interaction, 2023, pp. 104--113
2023
-
[45]
J. L. Orquin, N. J. Ashby, and A. D. Clarke, ``Areas of interest as a signal detection problem in behavioral eye-tracking research,'' Journal of Behavioral Decision Making, vol. 29, no. 2-3, pp. 103--115, 2016
2016
-
[46]
S. M. Munn and J. B. Pelz, ``Fixtag: An algorithm for identifying and tagging fixations to simplify the analysis of data collected by portable eye trackers,'' ACM Transactions on Applied Perception (TAP), vol. 6, no. 3, pp. 1--25, 2009
2009
-
[47]
Gundimeda, R
V. Gundimeda, R. S. Murali, R. Joseph, and N. Naresh Babu, ``An automated computer vision system for extraction of retail food product metadata,'' in First International Conference on Artificial Intelligence and Cognitive Computing: AICC 2018. 1em plus 0.5em minus 0.4em Spring...
2018
-
[48]
Hochreiter, ``Long short-term memory,'' Neural Computation MIT-Press, 1997
S. Hochreiter, ``Long short-term memory,'' Neural Computation MIT-Press, 1997
1997
-
[49]
A. K. Horn and H. Straka, ``Functional organization of extraocular motoneurons and eye muscles,'' Annual review of vision science, vol. 7, no. 1, pp. 793--825, 2021
2021
-
[50]
Shimojo, C
S. Shimojo, C. Simion, E. Shimojo, and C. Scheier, ``Gaze bias both reflects and influences preference,'' Nature neuroscience, vol. 6, no. 12, pp. 1317--1322, 2003
2003
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.