Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

Where, What, Why: Towards Explainable Driver Attention Prediction

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that driver attention should be predicted as a triple — where, what, and why — and that a single model jointly trained on all three outperforms models trained on each separately.

desk verdict A useful new benchmark and a plausible joint model, but the headline SOTA claims rest on an unstated baseline training protocol and LLM-generated 'why' labels. read the letter →

arxiv 2506.23088 v1 pith:LQ2UBM3S submitted 2025-06-29 cs.CV

Yuchen Zhou , Jiayu Tang , Xiaoyan Xiao , Yueyao Lin , Linkai Liu , Zipeng Guo , Hao Fei , Xiaobo Xia
show 1 more author
Chao Gou
This is my paper
classification cs.CV
keywords driverattentionpredictionexplainableAIgazemultimodallargelanguagemodelssemanticparsingcognitivereasoningautonomousdrivingbenchmarkdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a new task, Explainable Driver Attention Prediction, in which a model must output not only a heatmap of where a driver looks but also a label of what is attended and a natural-language explanation of why. To support it, the authors build W3DA, a 69,980-sample dataset that merges four existing driver-gaze benchmarks and adds semantic and causal annotations through a semi-automatic pipeline in which a multimodal language model drafts labels and human experts correct them. They then introduce LLada, a large-language-model-driven architecture that produces the heatmap, the semantics, and the causal reasoning in one end-to-end pass. The central claim is that jointly predicting all three tasks improves each one, and that a model trained once on W3DA generalizes across normal, safety-critical, and accident scenarios.

What carries the argument

The load-bearing mechanism is the joint training objective together with the [ATTN] attention token. [ATTN] is a special token added to the language model's vocabulary that carries high-level cognitive cues; its output embedding is projected into a cognitive-aware attention decoder, which cross-attends to visual features to produce the pixel-wise attention map, while the same language model generates the what and why text. The W3DA annotation pipeline supplies the third necessary component: a key-sample selector that keeps only frames with significant scene or attention changes, a chain-of-thought prompt that makes a multimodal LLM first count attended regions, then describe them, then explain them, and a human verification step that removes hallucinations and enforces traffic-rule consistency. These three pieces are what let the model treat where, what, and why as one problem.

What would settle it

Take a set of driving videos where drivers' gaze and retrospective verbal reports are recorded under deliberately different task instructions (e.g., 'prepare to turn' vs. 'maintain lane'); if LLada, trained on W3DA, cannot predict the task-driven shift in the attention map and cannot produce why explanations that match the drivers' stated reasons, then the claim that the model captures cognitive motivation rather than scene salience is refuted.

Watch

Extended reading notes

Core claim

The claim is that explanation is not a post-hoc add-on to driver attention prediction but a learnable output that improves the prediction itself. The paper defines the task as estimating an attention map A, a set of semantic descriptions S, and a set of causal explanations E from a driving image and context. W3DA provides those labels across normal driving, safety-critical situations, and traffic accidents. LLada attaches a special [ATTN] token to a language model, decodes a spatial map from that token through a small convolutional decoder, and generates the semantic and causal text autoregressively; training jointly on the map loss and the text loss leads to better maps and better explanations than training either alone. The result is a unified, explainable model of driver attention that transfers across domains without per-dataset retraining.

Load-bearing premise

The 'why' labels in W3DA are produced by prompting a large language model and then corrected by human experts; the paper treats these explanations as ground truth for real drivers' cognitive reasoning, and if they do not match actual driver cognition, the explainability claim measures the annotation model's style rather than human attention.

Editorial extensions

If this is right

  • A model trained on W3DA can be evaluated on and transferred to DR(eye)VE, BDDA, and DADA-2000 test sets, outperforming most specialized models trained independently on those datasets.
  • Jointly optimizing the attention map and textual explanations improves both: removing the map lowers text quality, and removing the what/why text lowers map quality in the ablation results.
  • The [ATTN]-token mechanism gives a general recipe for decoding spatial outputs from a language model's token embeddings, enabling pixel grounding without a task-specific head outside the LLM.
  • The human-corrected multimodal-LLM annotation pipeline provides a scalable template for adding semantic and causal labels to other gaze or attention datasets.
  • Explainable Driver Attention Prediction supplies a first benchmark, W3DA, with metrics for all three outputs, so future work can be compared on where, what, and why simultaneously.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not report inter-annotator agreement for the human correction step, so the degree to which W3DA's why labels reflect consensus driver cognition rather than individual expert style is unmeasured; a published agreement study would sharpen the benchmark's validity.
  • If the joint where/what/why training genuinely causes mutual improvement, then feeding predicted semantic and causal labels at inference time could improve gaze prediction in novel scenarios, a testable extension beyond the paper's experiments.
  • The W3DA why annotations could be repurposed as explanations for an autonomous driving policy, letting a system justify its attentional allocation to passengers; that application is not tested here.
  • The key-sample selection thresholds (vision-encoder similarity score ≤ 0.9, spatial KL divergence ≥ 5) are tuned on driving data; applying the same pipeline to other video annotation tasks would likely need recalibration, and the paper does not investigate that transfer.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces Explainable Driver Attention Prediction, a paradigm that extends conventional heatmap-based driver attention modeling by jointly predicting where a driver looks, what semantic region is attended, and why attention is allocated there. To support this, the authors assemble W3DA, a dataset of 69,980 keyframes drawn from DR(eye)VE, LBW, BDDA, and DADA-2000, with semantic and causal annotations generated by Qwen-VL-Max and subsequently human-refined. They also propose LLada, an LLM-driven architecture with a special [ATTN] token and a cognitive-aware attention decoder, trained end-to-end to output attention maps, semantic descriptions, and causal explanations. Experiments compare LLada against ten attention-prediction methods on W3DA, against two-stage baselines for textual explanation generation, and against domain-specific models on DR(eye)VE, BDDA, and DADA; ablations are used to claim that the where, what, and why tasks mutually reinforce each other.

Significance. If the headline results hold, the paper makes a useful contribution: W3DA is a large, multi-scenario benchmark that goes beyond spatial heatmaps, and LLada is a unified end-to-end architecture for joint spatial, semantic, and causal reasoning about driver attention. The dataset construction effort, the cross-domain training setup, and the public code release are strengths. However, the main SOTA comparisons are weakened by an unspecified baseline training protocol, and the cognitive-reasoning annotations are LLM-generated and human-polished without validation against actual drivers' stated reasons, so the 'why' evaluation may partly measure imitation of the annotation model rather than human cognition. These issues are load-bearing for the paper's central claims and require substantial clarification or additional experiments, but they are fixable within the manuscript's scope.

major comments (4)
  1. [Section 5.2, Table 1] The training protocol for the learning-based attention baselines is not stated. For MLNet, CDNN, FBNet, ConvNeXt, and ERFNet, the text says only that they are 'specialized attention prediction models optimized solely for this task'; it does not say whether they were retrained on the W3DA train split or evaluated from their original released weights. Since LLada is trained on 39,642 W3DA keyframes (Section 5.3), the reported KLdiv gains (e.g., 38.40% over ERFNet in normal driving) could reflect training-set familiarity rather than the benefit of the where/what/why joint formulation. Please state the protocol explicitly and, if the baselines were not retrained on W3DA, add comparisons with baselines fine-tuned on the W3DA train split under the same protocol, with standard deviations across at least three random seeds.
  2. [Section 5.2, Table 2] The same ambiguity affects the two-stage baselines. The text states only that the second-stage LLaVA is fine-tuned on W3DA; the training status of the first-stage predictors (DeepGaze I, DeepGaze IIE, MLNet, CDNN, ConvNeXt, ERFNet) is unstated. If those predictors are used with source-dataset weights, the baselines are evaluated on W3DA keyframes they have never seen, while LLada is trained on that distribution. Please specify the first-stage training protocol and, ideally, retrain the first-stage predictors on W3DA so that the comparison isolates the contribution of the end-to-end joint formulation rather than a train/test distribution mismatch.
  3. [Sections 3.1 and 7.3.3] The 'why' ground truth is produced by prompting Qwen-VL-Max and then correcting the outputs by human experts, but no evidence is provided that these explanations correspond to the actual cognitive reasoning of the drivers whose gaze was recorded. These labels are used both as training targets and as evaluation references for LLada, which is itself an LLM; the strong textual metrics may therefore reflect imitation of the annotation LLM's style rather than human attention mechanisms. To support the cognitive-reasoning claim, please provide at least one of the following: (a) inter-annotator agreement statistics on a sample of the why labels, (b) a human study comparing LLada's explanations with explanations elicited from drivers viewing the same scenes, or (c) an analysis demonstrating that the why labels contain information beyond generic, template-like completions. Without such validation, the statements in Sections 5.4 and 6 about 'alignment with human driver attention' and 'cognitive reasoning' are not yet supported.
  4. [Section 5.5, Figures 5 and 6] The ablation results are reported as single values without variance or significance testing. Some of the reported differences are small (e.g., KLdiv of 1.500 versus 1.498 in Figure 6), and for a 7B-parameter model these differences could easily fall within run-to-run noise. Please report means and standard deviations over multiple seeds and, where possible, significance tests, so that the central claim that the three tasks mutually reinforce one another rests on a statistically solid foundation.
minor comments (6)
  1. [Figure 4] The caption reads 'Quantitative examples' but the section is a qualitative analysis; the caption should read 'Qualitative examples'.
  2. [Figure 5] The first panel of Figure 5 labels the metric as 'BLUE'; this should be 'BLEU'.
  3. [Table 4] The row for ETOD contains an unresolved citation '[?]'; please provide the full reference or remove the row.
  4. [Section 4.2] The sentence 'we set the scaling factors Lmap, Ltxt, Lbce, Lkl, Lwhat, and Lwhy to 2, 1, 1, 0.1, 1, and 1' mixes loss names and scaling factors; Equations (4)-(6) use lambda notation, so the notation should be aligned for clarity.
  5. [Sections 3.2 and 5.3] Section 3.2 reports 69,980 total key samples, while Section 5.3 says W3DA provides 39,642 keyframes for training; please clarify the train/validation/test split sizes so that the discrepancy is resolved.
  6. [Figure 5] Figure 5 appears to contain duplicate METEOR panels; please verify that the panels correspond to the intended metrics.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: LLada is trained and evaluated against fixed W3DA annotations, and its 'why' targets are externally produced (Qwen-VL-Max plus human correction), not derived from LLada itself.

full rationale

The derivation chain is supervised learning rather than circular reasoning. W3DA's 'where' labels come from eye-tracking in DR(eye)VE/BDDA/DADA-2000, and its 'what'/'why' labels are produced by a semi-automatic pipeline (Qwen-VL-Max prompted with attention-masked frames, then human experts correct outputs; Sec. 3.1 and Sec. 7.3.3). LLada (Vicuna-7B + CLIP-ViT-L + attention decoder) is trained with Eq. (4)-(6), where the ground-truth S and E are these fixed dataset labels. Nothing in the paper constructs S or E from LLada's own outputs, and LLada does not use Qwen-VL-Max at inference, so there is no equation-level reduction of a prediction to its input. Evaluation on W3DA held-out splits (Sec. 5.2) and on the original DR(eye)VE/BDDA/DADA test sets (Sec. 5.3) is standard supervised evaluation. The concern that MLLM-generated 'why' labels may not reflect actual driver cognition is a construct-validity threat, not circularity: it questions whether the labels measure the intended construct, not whether the model's predictions are derived from those labels by construction. The paper's self-citations (e.g., refs. 23, 24, 30, 31, 85-89) appear only as background on prior driver-attention work and are not load-bearing for the architecture or the empirical claims. The unstated baseline training protocol in Table 1 is a fairness/reproducibility gap, but it does not make LLada's numbers equivalent to its inputs. I therefore find no significant circularity.

Assumptions & free parameters 3 free parameters · 3 assumptions · 1 invented entities

The dataset construction introduces tuned thresholds and an LLM-derived label source, while the model adds a learned attention token. These are the main elements the reader must accept on the paper's own terms.

free parameters (3)
  • Key sample selection thresholds = Sscene <= 0.9, DKL >= 5, Sattn <= 0.9
    Tuned to balance redundancy reduction and keyframe retention in Sec. 7.3.1; these thresholds directly determine which frames enter W3DA and therefore all downstream results.
  • Loss weights = lambda_map=2, lambda_txt=1, lambda_bce=1, lambda_kl=0.1, lambda_what=1, lambda_why=1
    Scaling factors in Eqs. 4-6 are hand-set with no sensitivity analysis.
  • Attention region count = 1 to 3 per frame
    The prompt instructs the MLLM to list up to 3 regions; the count is model output, not a fixed protocol.
assumptions (3)
  • domain assumption Existing driver attention datasets provide valid where ground truth
    The eye-tracking data from DR(eye)VE, LBW, BDDA, and DADA-2000 is reused without reassessment; any systematic errors in those annotations propagate into W3DA.
  • ad hoc to paper LLM-generated and human-refined why annotations are a valid proxy for real driver cognitive reasoning
    Sec. 3.1 and 7.3.3: Qwen-VL-Max generates reasons and human experts refine them. There is no validation against drivers' actual self-reports or cognitive models, so the 'why' labels are synthetic.
  • domain assumption CLIP embeddings capture semantic similarity relevant to attention shifts
    Keyframe selection relies on CLIP cosine similarity between scenes and attended regions; CLIP's notion of semantic change need not align with driver-relevant events.
invented entities (1)
  • [ATTN] token
    purpose: A special LLM vocabulary token whose output embedding is decoded by a cross-attention module to generate the spatial attention map
    It is a learned latent with no independent validation outside the paper; its usefulness is only shown through ablations on W3DA.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Where, What, Why: Towards Explainable Driver Attention Prediction." pith.science (2026). https://pith.science/paper/LQ2UBM3S

@misc{pith2026250623088,
  author       = {Pith},
  title        = {Pith review of: Where, What, Why: Towards Explainable Driver Attention Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LQ2UBM3S}},
  note         = {Machine review of arXiv:2506.23088}
}
read the original abstract

Modeling task-driven attention in driving is a fundamental challenge for both autonomous vehicles and cognitive science. Existing methods primarily predict where drivers look by generating spatial heatmaps, but fail to capture the cognitive motivations behind attention allocation in specific contexts, which limits deeper understanding of attention mechanisms. To bridge this gap, we introduce Explainable Driver Attention Prediction, a novel task paradigm that jointly predicts spatial attention regions (where), parses attended semantics (what), and provides cognitive reasoning for attention allocation (why). To support this, we present W3DA, the first large-scale explainable driver attention dataset. It enriches existing benchmarks with detailed semantic and causal annotations across diverse driving scenarios, including normal conditions, safety-critical situations, and traffic accidents. We further propose LLada, a Large Language model-driven framework for driver attention prediction, which unifies pixel modeling, semantic parsing, and cognitive reasoning within an end-to-end architecture. Extensive experiments demonstrate the effectiveness of LLada, exhibiting robust generalization across datasets and driving conditions. This work serves as a key step toward a deeper understanding of driver attention mechanisms, with significant implications for autonomous driving, intelligent driver training, and human-computer interaction.

Figures

Figures reproduced from arXiv: 2506.23088 by the authors.

Figure 1
Figure 1. Prior studies focus only on predicting where drivers allo [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The W³DA annotation pipeline. (a) W³DA integrates multiple driver attention benchmarks, covering normal driving, safety [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Overall architecture of LLada. Given a driving scene represented as an image with contextual information, the image is processed [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Quantitative examples. More in supplementary materials. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Impact of where on textual explanation generation. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Impact of what and why on attention map prediction. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Data Sources & Categories of W³DA. 7.2.3. Mean Fixation Maps (Where) To further analyze the overall distribution of driver attention in different driving scenarios, we compute the mean fixation maps across all samples in the dataset [PITH_FULL_IMAGE:figures/full_fig_p…
Figure 8
Figure 8. Figure 8: Distribution of weather conditions, time periods, and geographic locations in the W³DA dataset. [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Mean fixation heatmaps for different driving scenarios. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Word cloud of high-frequency semantic labels representing key elements in the driving scene, such as “Road Ahead,” “Left [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Word cloud of high-frequency cognitive causes, illustrating key factors such as “road conditions,” “obstacle,” “safe navigation,” [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: Prompt templates for MLLM annotation in normal and safety-critical driving scenarios. [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Prompt templates for MLLM annotation in traffic accident scenarios. [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]
Figure 14
Figure 14. Figure 14: Comparison of MLLM-generated driver attention annotations (left) and human-corrected ground truth (right). The red text [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]
Figure 15
Figure 15. Figure 15: Prompt templates for LLada [PITH_FULL_IMAGE:figures/full_fig_p018_15.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LaP-Forensics: Latent-Pixel Consistency Guided Multimodal Reasoning for Deepfake Detection

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A dual-stream deepfake forensic model that adds DDIM reconstruction residuals to RGB features improves artifact localization and cross-generator detection in evaluations, with honest caveats about text faithfulness.

  2. RAMA: Retrieval-Augmented Multi-Agent Framework for Misinformation Detection in Multimodal Fact-Checking

    cs.CL 2025-07 reject novelty 4.0 of 10

    RAMA, a retrieval-augmented multi-agent detector, reports 0.910 accuracy and F1 on the ICMR 2024 public test set, placing it behind the top published method on the same benchmark.

Reference graph

Works this paper leans on

102 extracted references · 71 canonical work pages · cited by 2 Pith papers

  1. [1]

    Hammerdrive: A task-aware driving visual attention model

    Pierluigi Vito Amadori, Tobias Fischer, and Yiannis Demiris. Hammerdrive: A task-aware driving visual attention model. IEEE T-ITS, 23(6):5573–5585, 2021. 3

    2021

  2. [2]

    Aragan: A driver attention estimation model based on conditional gen- erative adversarial network

    Javier Araluce, Luis M Bergasa, Manuel Oca ˜na, Rafael Barea, Elena L´opez-Guill´en, and Pedro Revenga. Aragan: A driver attention estimation model based on conditional gen- erative adversarial network. In IV, pages 1066–1072. IEEE,

  3. [3]

    Medirl: Predicting the visual attention of drivers via maximum entropy deep inverse re- inforcement learning

    Sonia Baee, Erfan Pakdamanian, Inki Kim, Lu Feng, Vicente Ordonez, and Laura Barnes. Medirl: Predicting the visual attention of drivers via maximum entropy deep inverse re- inforcement learning. In ICCV, pages 13178–13188, 2021. 3

    2021

  4. [4]

    Drive: Deep reinforced accident anticipation with visual explanation

    Wentao Bao, Qi Yu, and Yu Kong. Drive: Deep reinforced accident anticipation with visual explanation. In ICCV, pages 7619–7628, 2021. 8

    2021

  5. [5]

    Re- current mixture density network for spatiotemporal visual at- tention

    Loris Bazzani, Hugo Larochelle, and Lorenzo Torresani. Re- current mixture density network for spatiotemporal visual at- tention. In ICLR, 2017. 8

    2017

  6. [6]

    The devil is in the fine- grained details: Evaluating open-vocabulary object detec- tors for fine-grained understanding

    Lorenzo Bianchi, Fabio Carrara, Nicola Messina, Claudio Gennaro, and Fabrizio Falchi. The devil is in the fine- grained details: Evaluating open-vocabulary object detec- tors for fine-grained understanding. In CVPR, pages 22520– 22529, 2024. 3

    2024

  7. [7]

    Trends, applications, and challenges in human attention modelling

    Giuseppe Cartella, Marcella Cornia, Vittorio Cuculo, Alessandro D’Amelio, Dario Zanca, Giuseppe Boccignone, and Rita Cucchiara. Trends, applications, and challenges in human attention modelling. In IJCAI, pages 7971–7979,

  8. [8]

    Sharegpt4video: Improving video understand- ing and generation with better captions.NeurIPS, 37:19472– 19495, 2025

    Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Zhenyu Tang, Li Yuan, et al. Sharegpt4video: Improving video understand- ing and generation with better captions.NeurIPS, 37:19472– 19495, 2025. 3

    2025

Show all 102 references
  1. [9]

    Air: At- tention with reasoning capability

    Shi Chen, Ming Jiang, Jinhui Yang, and Qi Zhao. Air: At- tention with reasoning capability. In ECCV, pages 91–107. Springer, 2020. 1

    2020

  2. [10]

    Gazexplain: Learn- ing to predict natural language explanations of visual scan- paths

    Xianyu Chen, Ming Jiang, and Qi Zhao. Gazexplain: Learn- ing to predict natural language explanations of visual scan- paths. In ECCV, pages 314–333. Springer, 2024. 3, 6, 7

    2024

  3. [11]

    Fblnet: Feed- back loop network for driver attention prediction

    Yilong Chen, Zhixiong Nan, and Tao Xiang. Fblnet: Feed- back loop network for driver attention prediction. In ICCV, pages 13371–13380, 2023. 1, 3, 6, 8

    2023

  4. [12]

    What do you see in vehicle? comprehensive vi- sion solution for in-vehicle gaze estimation

    Yihua Cheng, Yaning Zhu, Zongji Wang, Hongquan Hao, Yongwei Liu, Shiqing Cheng, Xi Wang, and Hyung Jin Chang. What do you see in vehicle? comprehensive vi- sion solution for in-vehicle gaze estimation. In CVPR, pages 1556–1565, 2024. 2

    2024

  5. [13]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhang- hao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yong- hao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, 2023. 5, 7

    2023

  6. [14]

    A survey on multimodal large language models for autonomous driving

    Can Cui, Yunsheng Ma, Xu Cao, Wenqian Ye, Yang Zhou, Kaizhao Liang, Jintai Chen, Juanwu Lu, Zichong Yang, Kuei-Da Liao, et al. A survey on multimodal large language models for autonomous driving. In WACV, pages 958–979,

  7. [15]

    Tao Deng, Hongmei Yan, Long Qin, Thuyen Ngo, and B. S. Manjunath. How do drivers allocate their potential atten- tion? driving fixation prediction via convolutional neural net- works. IEEE T-ITS, 21(5):2146–2154, 2020. 3, 6, 7

    2020

  8. [16]

    Driving vi- sual saliency prediction of dynamic night scenes via a spatio- temporal dual-encoder network

    Tao Deng, Lianfang Jiang, Yi Shi, Jiang Wu, Zhangbi Wu, Shun Yan, Xianshi Zhang, and Hongmei Yan. Driving vi- sual saliency prediction of dynamic night scenes via a spatio- temporal dual-encoder network. IEEE T-ITS, 25(3):2413– 2423, 2023. 8

    2023

  9. [17]

    Fbnet: Feedback- recursive cnn for saliency detection

    Guanqun Ding, Nevrez ˙Imamo˘glu, Ali Caglayan, Masahiro Murakawa, and Ryosuke Nakamura. Fbnet: Feedback- recursive cnn for saliency detection. In MVA, pages 1–5. IEEE, 2021. 6, 7

    2021

  10. [18]

    Visual saliency predic- tion using a mixture of deep neural networks

    Samuel F Dodge and Lina J Karam. Visual saliency predic- tion using a mixture of deep neural networks. IEEE T-IP, 27 (8):4080–4090, 2018. 6, 7, 8

    2018

  11. [19]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...

    2021

  12. [20]

    Dgaze: Driver gaze mapping on road

    Isha Dua, Thrupthi Ann John, Riya Gupta, and CV Jawahar. Dgaze: Driver gaze mapping on road. In IROS, pages 5946–

  13. [21]

    Dada-2000: Can driving accident be predicted by driver attentionƒ analyzed by a benchmark

    Jianwu Fang, Dingxin Yan, Jiahuan Qiao, Jianru Xue, He Wang, and Sen Li. Dada-2000: Can driving accident be predicted by driver attentionƒ analyzed by a benchmark. In ITSC, pages 4303–4309. IEEE, 2019. 2, 3, 7, 12, 13

    2000

  14. [22]

    Dada: Driver attention prediction in driving accident scenarios

    Jianwu Fang, Dingxin Yan, Jiahuan Qiao, Jianru Xue, and Hongkai Yu. Dada: Driver attention prediction in driving accident scenarios. IEEE T-ITS, 23(6):4959–4971, 2022. 1, 3, 6, 7, 8, 12, 13

    2022

  15. [23]

    Driver attention pre- diction based on convolution and transformers

    Chao Gou, Yuchen Zhou, and Dan Li. Driver attention pre- diction based on convolution and transformers. The Journal of Supercomputing, 78(6):8268–8284, 2022. 3

    2022

  16. [24]

    Cascade learning for driver facial monitoring

    Chao Gou, Yuchen Zhou, Yao Xiao, Xiao Wang, and Hui Yu. Cascade learning for driver facial monitoring. IEEE T-IV, 8 (1):404–412, 2023. 1

    2023

  17. [25]

    Graph- based visual saliency

    Jonathan Harel, Christof Koch, and Pietro Perona. Graph- based visual saliency. NeurIPS, 19, 2006. 2, 6, 7

    2006

  18. [26]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022. 6, 7

    2022

  19. [27]

    Data-driven estimation of driver attention us- ing calibration-free eye gaze and scene features

    Zhongxu Hu, Chen Lv, Peng Hang, Chao Huang, and Yang Xing. Data-driven estimation of driver attention us- ing calibration-free eye gaze and scene features. IEEE T-IE, 69(2):1800–1808, 2021. 3

    2021

  20. [28]

    A novel heterogeneous network for modeling driver attention with multi-level visual content

    Zhongxu Hu, Yiran Zhang, Qinghua Li, and Chen Lv. A novel heterogeneous network for modeling driver attention with multi-level visual content. IEEE T-ITS, 23(12):24343– 24354, 2022. 3

    2022

  21. [29]

    Predicting gaze in egocentric video by learning task- dependent attention transition

    Yifei Huang, Minjie Cai, Zhenqiang Li, and Yoichi Sato. Predicting gaze in egocentric video by learning task- dependent attention transition. In ECCV, pages 754–769,

  22. [30]

    Task-oriented scanpath prediction with spatial-temporal information in driving scenarios

    Zhixin Huang, Yuchen Zhou, and Chao Gou. Task-oriented scanpath prediction with spatial-temporal information in driving scenarios. In PRCV, pages 177–191. Springer, 2024. 1

    2024

  23. [31]

    Driver scanpath prediction based on inverse reinforcement learning

    Zhixin Huang, Yuchen Zhou, Jie Zhu, and Chao Gou. Driver scanpath prediction based on inverse reinforcement learning. In ICASSP, pages 8306–8310, 2024. 1

    2024

  24. [32]

    A model of saliency-based visual attention for rapid scene analysis

    Laurent Itti, Christof Koch, and Ernst Niebur. A model of saliency-based visual attention for rapid scene analysis. IEEE T-PAMI, 20(11):1254–1259, 1998. 2, 6, 7

    1998

  25. [33]

    Mtsf: Multi-scale temporal–spatial fusion network for driver attention predic- tion

    Lisheng Jin, Bingdong Ji, Baicang Guo, Huanhuan Wang, Zhuotong Han, and Xingchen Liu. Mtsf: Multi-scale temporal–spatial fusion network for driver attention predic- tion. IEEE T-ITS, 2024. 8

    2024

  26. [34]

    Look both ways: Self-supervising driver gaze estimation and road scene saliency

    Isaac Kasahara, Simon Stent, and Hyun Soo Park. Look both ways: Self-supervising driver gaze estimation and road scene saliency. In ECCV. Springer Nature Switzerland,

  27. [35]

    Attention for vision- based assistive and automated driving: A review of al- gorithms and datasets

    Iuliia Kotseruba and John K Tsotsos. Attention for vision- based assistive and automated driving: A review of al- gorithms and datasets. IEEE T-ITS, 23(11):19907–19928,

  28. [36]

    Iuliia Kotseruba and John K. Tsotsos. Understanding and modeling the effects of task and context on drivers’ gaze al- location. In IV, 2024. 12

    2024

  29. [37]

    Iuliia Kotseruba and John K. Tsotsos. Data limitations for modeling top-down effects on drivers’ attention. InIV, 2024. 1, 12

    2024

  30. [38]

    Deep gaze I: boosting saliency prediction with feature maps trained on imagenet

    Matthias K ¨ummerer, Lucas Theis, and Matthias Bethge. Deep gaze I: boosting saliency prediction with feature maps trained on imagenet. In ICLRW, 2015. 6, 7

    2015

  31. [39]

    Lisa: Reasoning segmenta- tion via large language model

    Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Reasoning segmenta- tion via large language model. In CVPR, pages 9579–9589,

  32. [40]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In Interna- tional conference on machine learning, pages 12888–12900. PMLR, 2022. 3

    2022

  33. [41]

    Adaptive short-temporal induced aware fusion network for predicting attention regions like a driver

    Qiang Li, Chunsheng Liu, Faliang Chang, Shuang Li, Hui Liu, and Zehao Liu. Adaptive short-temporal induced aware fusion network for predicting attention regions like a driver. IEEE T-ITS, 23(10):18695–18706, 2022. 3

    2022

  34. [42]

    Lmeye: An interactive perception network for large language models.IEEE T-MM, 26:10952–10964, 2024

    Yunxin Li, Baotian Hu, Xinyu Chen, Lin Ma, Yong Xu, and Min Zhang. Lmeye: An interactive perception network for large language models.IEEE T-MM, 26:10952–10964, 2024. 3

    2024

  35. [43]

    Perception, reason, think, and plan: A survey on large multimodal reasoning models

    Yunxin Li, Zhenyu Liu, Zitao Li, Xuanyu Zhang, Zhenran Xu, Xinyu Chen, Haoyuan Shi, Shenyuan Jiang, Xintong Wang, Jifang Wang, et al. Perception, reason, think, and plan: A survey on large multimodal reasoning models. arXiv preprint arXiv:2505.04921, 2025. 3

    2025 arXiv

  36. [44]

    Airvista: Empowering uavs with 3d spatial reasoning abilities through a multimodal large language model agent

    Fei Lin, Yonglin Tian, Yunzhe Wang, Tengchao Zhang, Xinyuan Zhang, and Fei-Yue Wang. Airvista: Empowering uavs with 3d spatial reasoning abilities through a multimodal large language model agent. In ITSC, pages 476–481. IEEE,

  37. [45]

    Deepgaze iie: Calibrated prediction in and out- of-domain for state-of-the-art saliency modeling

    Akis Linardos, Matthias K ¨ummerer, Ori Press, and Matthias Bethge. Deepgaze iie: Calibrated prediction in and out- of-domain for state-of-the-art saliency modeling. In ICCV, pages 12919–12928, 2021. 6, 7

    2021

  38. [46]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. NeurIPS, 36:34892–34916, 2023. 3, 5, 7

    2023

  39. [47]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In CVPR, pages 26296–26306, 2024. 3

    2024

  40. [48]

    Goal-oriented gaze estimation for zero-shot learning

    Yang Liu, Lei Zhou, Xiao Bai, Yifei Huang, Lin Gu, Jun Zhou, and Tatsuya Harada. Goal-oriented gaze estimation for zero-shot learning. In CVPR, pages 3794–3803, 2021. 1

    2021

  41. [49]

    A convnet for the 2020s

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feicht- enhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In CVPR, pages 11976–11986, 2022. 6, 7

    2022

  42. [50]

    Robollm: Robotic vision tasks grounded on multimodal large language models

    Zijun Long, George Killick, Richard McCreadie, and Ger- ardo Aragon-Camarasa. Robollm: Robotic vision tasks grounded on multimodal large language models. In ICRA, pages 12428–12435. IEEE, 2024. 3

    2024

  43. [51]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 6

    2017 arXiv

  44. [52]

    Gaze and eye- tracking solutions for psychological research

    Maria Laura Mele and Stefano Federici. Gaze and eye- tracking solutions for psychological research. Cognitive pro- cessing, 13:261–265, 2012. 1

    2012

  45. [53]

    ” looking at the right stuff”-guided semantic-gaze for au- tonomous driving

    Anwesan Pal, Sayan Mondal, and Henrik I Christensen. ” looking at the right stuff”-guided semantic-gaze for au- tonomous driving. In CVPR, pages 11883–11892, 2020. 1

    2020

  46. [54]

    Predicting the driver’s focus of attention: The dr(eye)ve project

    Andrea Palazzi, Davide Abati, simone Calderara, Francesco Solera, and Rita Cucchiara. Predicting the driver’s focus of attention: The dr(eye)ve project. IEEE T-PAMI, 41(7):1720– 1733, 2019. 1, 2, 3, 7, 8, 12, 13

    2019

  47. [55]

    Multi-scale interactive network for salient object detection

    Youwei Pang, Xiaoqi Zhao, Lihe Zhang, and Huchuan Lu. Multi-scale interactive network for salient object detection. In CVPR, pages 9413–9422, 2020. 8

    2020

  48. [56]

    Human generalization of in- ternal representations through prototype learning with goal- directed attention

    Warren Woodrich Pettine, Dhruva Venkita Raman, A David Redish, and John D Murray. Human generalization of in- ternal representations through prototype learning with goal- directed attention. Nature Human Behaviour, 7(3):442–463,

  49. [57]

    The attention sys- tem of the human brain

    Michael I Posner and Steven E Petersen. The attention sys- tem of the human brain. Annual Review of Neuroscience, 13 (1):25–42, 1990. 1

    1990

  50. [58]

    U2-net: Go- ing deeper with nested u-structure for salient object detec- tion

    Xuebin Qin, Zichen Zhang, Chenyang Huang, Masood De- hghan, Osmar R Zaiane, and Martin Jagersand. U2-net: Go- ing deeper with nested u-structure for salient object detec- tion. Pattern Recognition, 106:107404, 2020. 8

    2020

  51. [59]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In ICML, pages 8748–8763. PMLR, 2021. 4, 5, 7, 14

    2021

  52. [60]

    Deepspeed: System optimizations enable train- ing deep learning models with over 100 billion parameters

    Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. Deepspeed: System optimizations enable train- ing deep learning models with over 100 billion parameters. In ACM SIGKDD, pages 3505–3506, 2020. 6

    2020

  53. [61]

    Pixellm: Pixel reasoning with large multimodal model

    Zhongwei Ren, Zhicheng Huang, Yunchao Wei, Yao Zhao, Dongmei Fu, Jiashi Feng, and Xiaojie Jin. Pixellm: Pixel reasoning with large multimodal model. In CVPR, pages 26374–26383, 2024. 3

    2024

  54. [62]

    Weakly supervised fixated object de- tection in traffic videos based on driver’s selective attention mechanism

    Yi Shi, Long Qin, Shixuan Zhao, Kaifu Yang, Yuyong Cui, and Hongmei Yan. Weakly supervised fixated object de- tection in traffic videos based on driver’s selective attention mechanism. IEEE T-CSVT, 34(11):11478–11492, 2024. 3

    2024

  55. [63]

    Fixated object detection based on saliency prior in traffic scenes

    Yi Shi, Shixuan Zhao, Jiang Wu, Zhangbi Wu, and Hong- mei Yan. Fixated object detection based on saliency prior in traffic scenes. IEEE T-CSVT, 34(3):1413–1426, 2024. 3

    2024

  56. [64]

    Drivelm: Driving with graph visual question answering

    Chonghao Sima, Katrin Renz, Kashyap Chitta, Li Chen, Hanxue Zhang, Chengen Xie, Jens Beißwenger, Ping Luo, Andreas Geiger, and Hongyang Li. Drivelm: Driving with graph visual question answering. In ECCV, pages 256–274. Springer, 2024. 3

    2024

  57. [65]

    Vitgaze: gaze following with interaction features in vision transformers

    Yuehao Song, Xinggang Wang, Jingfeng Yao, Wenyu Liu, Jinglin Zhang, and Xiangmin Xu. Vitgaze: gaze following with interaction features in vision transformers. Visual Intel- ligence, 2(1):1–15, 2024. 1

    2024

  58. [66]

    From gaze perception to social cognition: The shared- attention system

    Lisa J Stephenson, S Gareth Edwards, and Andrew P Bayliss. From gaze perception to social cognition: The shared- attention system. Perspectives on Psychological Science, 16 (3):553–576, 2021. 1

    2021

  59. [67]

    Driving as well as on a sunny day? predicting driver’s fixation in rainy weather conditions via a dual-branch visual model

    Han Tian, Tao Deng, and Hongmei Yan. Driving as well as on a sunny day? predicting driver’s fixation in rainy weather conditions via a dual-branch visual model. IEEE/CAA Jour- nal of Automatica Sinica, 9(7):1335–1338, 2022. 8

    2022

  60. [68]

    Drivevlm: The convergence of autonomous driving and large vision-language models

    Xiaoyu Tian, Junru Gu, Bailin Li, Yicheng Liu, Yang Wang, Zhiyong Zhao, Kun Zhan, Peng Jia, Xianpeng Lang, and Hang Zhao. Drivevlm: The convergence of autonomous driving and large vision-language models. In CoRL, 2024. 3

    2024

  61. [69]

    Uavs meet llms: Overviews and perspectives toward agentic low-altitude mobility

    Yonglin Tian, Fei Lin, Yiduo Li, Tengchao Zhang, Qiyao Zhang, Xuan Fu, Jun Huang, Xingyuan Dai, Yutong Wang, Chunwei Tian, et al. Uavs meet llms: Overviews and perspectives toward agentic low-altitude mobility. arXiv preprint arXiv:2501.02341, 2025. 3

    2025 arXiv

  62. [70]

    Next-gpt: Any-to-any multimodal llm

    Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua. Next-gpt: Any-to-any multimodal llm. In ICML,

  63. [71]

    Predicting driver attention in critical situations

    Ye Xia, Danqing Zhang, Jinkyu Kim, Ken Nakayama, Karl Zipser, and David Whitney. Predicting driver attention in critical situations. In ACCV, pages 658–674. Springer, 2019. 1, 2, 3, 7, 8, 12, 13

    2019

  64. [72]

    Gsva: Generalized segmentation via multimodal large language models

    Zhuofan Xia, Dongchen Han, Yizeng Han, Xuran Pan, Shiji Song, and Gao Huang. Gsva: Generalized segmentation via multimodal large language models. In CVPR, pages 3858– 3869, 2024. 3

    2024

  65. [73]

    Pyramid grafting network for one- stage high resolution saliency detection

    Chenxi Xie, Changqun Xia, Mingcan Ma, Zhirui Zhao, Xi- aowu Chen, and Jia Li. Pyramid grafting network for one- stage high resolution saliency detection. In CVPR, pages 11717–11726, 2022. 8

    2022

  66. [74]

    Transconvnet: Perform perceptually relevant driver’s visual attention predictions

    Chuan Xu, Bo Jiang, and Yan Su. Transconvnet: Perform perceptually relevant driver’s visual attention predictions. Computers and Electrical Engineering , 115:109104, 2024. 8

    2024

  67. [75]

    V oila-a: Aligning vision-language models with user’s gaze attention

    Kun Yan, Zeyu Wang, Lei Ji, Yuntao Wang, Nan Duan, and Shuai Ma. V oila-a: Aligning vision-language models with user’s gaze attention. NeurIPS, 37:1890–1918, 2025. 3

    1918

  68. [76]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024. 3

    2024 arXiv

  69. [77]

    Progres- sive adapting and pruning: Domain-incremental learning for saliency prediction

    Kaihui Yang, Junwei Han, Guangyu Guo, Chaowei Fang, Yingzi Fan, Lechao Cheng, and Dingwen Zhang. Progres- sive adapting and pruning: Domain-incremental learning for saliency prediction. ACM Transactions on Multimedia Com- puting, Communications and Applications, 2024. 6, 7

    2024

  70. [78]

    Continual learning with bayesian model based on a fixed pre-trained feature extrac- tor

    Yang Yang, Zhiying Cui, Junjie Xu, Changhong Zhong, Wei- Shi Zheng, and Ruixuan Wang. Continual learning with bayesian model based on a fixed pre-trained feature extrac- tor. Visual Intelligence, 1(1):5, 2023. 3

    2023

  71. [79]

    From seeing to doing: Bridging reasoning and decision for robotic manipulation

    Yifu Yuan, Haiqin Cui, Yibin Chen, Zibin Dong, Fei Ni, Longxin Kou, Jinyi Liu, Pengyi Li, Yan Zheng, and Jianye Hao. From seeing to doing: Bridging reasoning and decision for robotic manipulation. arXiv preprint arXiv:2505.08548,

    arXiv

  72. [80]

    Contextual object detection with mul- timodal large language models

    Yuhang Zang, Wei Li, Jun Han, Kaiyang Zhou, and Chen Change Loy. Contextual object detection with mul- timodal large language models. IJCV, pages 1–19, 2024. 3

    2024

  73. [81]

    Da-bev: Depth aware bev transformer for 3d object detection

    Hao Zhang, Hongyang Li, Xingyu Liao, Feng Li, Shilong Liu, Lionel M Ni, and Lei Zhang. Da-bev: Depth aware bev transformer for 3d object detection. CoRR, 2023. 3

    2023

  74. [82]

    Llava-grounding: Grounded visual chat with large multimodal models

    Hao Zhang, Hongyang Li, Feng Li, Tianhe Ren, Xueyan Zou, Shilong Liu, Shijia Huang, Jianfeng Gao, Leizhang, Chunyuan Li, et al. Llava-grounding: Grounded visual chat with large multimodal models. In ECCV, pages 19–35. Springer, 2024. 3

    2024

  75. [83]

    Human gaze assisted artificial intelligence: A review

    Ruohan Zhang, Akanksha Saran, Bo Liu, Yifeng Zhu, Si- hang Guo, Scott Niekum, Dana Ballard, and Mary Hayhoe. Human gaze assisted artificial intelligence: A review. In IJ- CAI, page 4951. NIH Public Access, 2020. 1

    2020

  76. [84]

    Salm 2: An extremely lightweight saliency mamba model for real-time cognitive awareness of driver at- tention

    Chunyu Zhao, Wentao Mu, Xian Zhou, Wenbo Liu, Fei Yan, and Tao Deng. Salm 2: An extremely lightweight saliency mamba model for real-time cognitive awareness of driver at- tention. In AAAI, pages 1647–1655, 2025. 1

    2025

  77. [85]

    Learning from easy to hard pairs: Multi-step reasoning net- work for human-object interaction detection

    Yuchen Zhou, Guang Tan, Mengtang Li, and Chao Gou. Learning from easy to hard pairs: Multi-step reasoning net- work for human-object interaction detection. In ACM MM, pages 4368–4377, 2023. 1

    2023

  78. [86]

    Pit: Progressive interaction transformer for pedestrian crossing intention prediction

    Yuchen Zhou, Guang Tan, Rui Zhong, Yaokun Li, and Chao Gou. Pit: Progressive interaction transformer for pedestrian crossing intention prediction. IEEE T-ITS, 24(12):14213– 14225, 2023. 1

    2023

  79. [87]

    Learning from observer gaze: Zero-shot attention prediction oriented by human-object interaction recognition

    Yuchen Zhou, Linkai Liu, and Chao Gou. Learning from observer gaze: Zero-shot attention prediction oriented by human-object interaction recognition. In CVPR, pages 28390–28400, 2024. 1

    2024

  80. [88]

    Hktsg: A hierarchical knowledge-guided traffic scene graph representation learning framework for intelligent vehi- cles

    Yuchen Zhou, Xinxin Liu, Zipeng Guo, Ming Cai, and Chao Gou. Hktsg: A hierarchical knowledge-guided traffic scene graph representation learning framework for intelligent vehi- cles. IEEE T-IV, 2024. 1

    2024

  81. [89]

    Behavior-aware knowledge-embedded model for driver attention prediction

    Yuchen Zhou, Chao Gou, Zipeng Guo, Yihua Cheng, and Hyung Jin Chang. Behavior-aware knowledge-embedded model for driver attention prediction. IEEE T-CSVT, 2025. 1

    2025

  82. [90]

    Unsupervised self-driving attention prediction via un- certainty mining and knowledge embedding

    Pengfei Zhu, Mengshi Qi, Xia Li, Weijian Li, and Huadong Ma. Unsupervised self-driving attention prediction via un- certainty mining and knowledge embedding. In ICCV, pages 8524–8534, 2023. 1, 3

    2023

  83. [91]

    Dataset Comparisons To highlight the advantages of W³DA, we compare it with existing driving attention datasets, as shown in Table 4

    More Details about W³DA Dataset 7.1. Dataset Comparisons To highlight the advantages of W³DA, we compare it with existing driving attention datasets, as shown in Table 4. Un- like previous datasets that focus on single-domain scenarios with limited annotations, W³DA is the fir...

  84. [92]

    road ahead

    primarily focus on normal driving, while BDD-A [71] and DADA-2000 [21, 22], despite covering safety-critical and accident scenarios, are collected in controlled environ- ments. Moreover, previous datasets typically rely on uni- form frame sampling (3 frames per second) to trai...

    2000

  85. [93]

    The image shows your visual attention distribution

    More Details about LLada model In this section, we provide details on the prompt templates used in LLada, as illustrated in Fig. 15. While LLada’s input prompts share similarities with those used for dataset annotation, they differ in several key aspects. First, LLada takes ra...

  86. [96]

    Why is the driver focusing on these regions? - For each reason, consider the impact on upcoming driving decisions, start with 'To ... ', and format your response as follows: - 'Reason 1: [Explanation for the first region]' - - (If applicable) 'Reason 2: [Explanation for the se...

  87. [99]

    Why is the driver focusing on these regions? - For each reason, consider the impact on upcoming driving decisions, start with 'To ... ', and format your response as follows: - 'Reason 1: [Explanation for the first region]' - - (If applicable) 'Reason 2: [Explanation for the se...

  88. [100]

    'The driving environment is normal, and no abnormal events have occurred.'

  89. [101]

    {context_description}

    'An abnormal event is occurring, which may lead to an accident of {accident_name}. {context_description}. {accident_cause}.'

  90. [102]

    'An accident of {accident_name} has just occurred.'

  91. [103]

    Prompt templates for MLLM annotation in traffic accident scenarios

    'The driving environment has returned to normal.' Figure 13. Prompt templates for MLLM annotation in traffic accident scenarios. Qwen-VL-MAX API Human Correction-Ground Truth (What?) Attention Semantics [Road ahead] [Pedestrian crossing] [ To monitor the road conditions and an...

  92. [104]

    How many regions of attention are present in the image? - Format your response as: 'Number of regions: [number]'

  93. [105]

    What are the specific regions where the driver’s attention is focused? - For each region, format your response as follows: - 'Region 1: [Name of the first region]' - - (If applicable) 'Region 2: [Name of the second region]' - - (If applicable) 'Region 3: [Name of the third region]'

  94. [106]

    Why is the driver focusing on these regions? - For each reason, consider the impact on upcoming driving decisions, start with 'To ... ', and format your response as follows: - 'Reason 1: [Explanation for the first region]' - - (If applicable) 'Reason 2: [Explanation for the se...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.