REVIEW 4 major objections 5 minor 82 references
E3AD shows that one continuous valence-arousal-dominance emotion vector improves grounding (80.12 vs 74.62 IoU), planning (ADE 3.88 vs 4.54 m), and emotion estimation (ρ≈0.94–0.95) in a single end-to-end driving model.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 18:30 UTC pith:LBUN4L3Z
load-bearing objection A well-engineered VLA with a clever integration, but the emotion results are measured against the paper's own labels — validate against human ratings before believing the affective claims. the 4 major comments →
E3AD: An Emotion-Aware Vision-Language-Action Model for Human-Centric End-to-End Autonomous Driving
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim, on the paper's own terms, is that emotion is a control variable for end-to-end driving, not a side-channel. Conditioning both referent grounding and trajectory generation on a continuous VAD vector — predicted from the command before the target is located and the path is drawn — yields consistent gains: seven trajectory metrics improve by roughly 16–20% against the strongest baseline, visual grounding beats all prior methods on every dataset and split (with the largest margins on occluded, multi-agent, ambiguous, and long-text commands), and emotion estimation reaches rank correlations of 0.94–0.95. The VAD vector behaves as a continuous dial: for a fixed driving intent, h
What carries the argument
The central object is the continuous valence-arousal-dominance (VAD) vector e∈R³ — three numbers (positivity, activation, control) that compress the tone of the passenger's command and condition everything that follows. Two mechanisms carry that vector through the model. First, dual-pathway spatial reasoning, which fuses an egocentric pathway (first-person cues: relative 3D direction, distance, image-space grounding) with an allocentric pathway (bird's-eye-view map cues: target location and a coarse trajectory), mirroring how humans combine local perception with a cognitive map. Second, a consistency-oriented training scheme built on emotion-aware command augmentation: each command is paraph
Load-bearing premise
The load-bearing premise is that the VAD labels used for both training and evaluation (built per §3.3 and Appendix A.1, and scored in Table 3) reflect how real passengers experience the commands: they come from an automated pipeline — a GoEmotions classifier mapped through a label-to-VAD dictionary and fused at α=0.5 with word-level NRC-VAD lexical norms — so if those labels diverge from human perception, the emotion-to-planning results do not follow.
What would settle it
Have human raters score valence, arousal, and dominance on a sample of the original and emotion-augmented commands, then check whether E3AD's predicted VAD tracks human ratings rather than only the automated labels, and whether the emotion-to-planning effect (higher arousal → straighter trajectories, lower arousal → cautious curves) survives when trajectories are conditioned on human-annotated rather than pipeline-generated VAD. A second check: fix the command text and vary only the VAD conditioning (raising arousal while the words stay identical) and measure whether planned trajectories chang
If this is right
- The same command produces different, yet physically valid, plans depending on emotion: high-arousal commands yield straighter, more decisive trajectories with earlier hazard response, and low-arousal commands yield slower approaches with larger safety margins.
- Visual grounding improves most where commands are hardest — +6.95% to +8.26% IoU on occluded, multi-agent, and ambiguous scenes and +11.63% on long-text commands — so emotion and spatial priors help resolve referential ambiguity.
- Emotion estimation reaches ρ≈0.94–0.95 on valence, arousal, and dominance, far above language-model baselines, so the model tracks continuous shifts in tone rather than coarse categories.
- Verbal feedback is conditioned on the predicted emotion, so a calm request gets a brief confirmation while an urgent one gets direct, time-critical guidance — a direct attempt to reduce the 'black-box anxiety' that lowers trust in self-driving vehicles.
- Gains come from task-aligned structure, not scale: with the backbone frozen and only low-rank adapters trained, E3AD outperforms much larger general-purpose vision-language models.
Where Pith is reading between the lines
- The design pattern — a continuous affect vector conditioning both grounding and action generation — plausibly transfers to other language-guided robotic tasks, such as teleoperation, assistive navigation, or warehouse robots, wherever a user's tone signals how a stated goal should be executed.
- A sharper test of the thesis would hold the command text fixed while varying only the VAD conditioning, or substitute genuinely emotional speech prosody, to confirm trajectory changes are caused by emotion rather than by lexical correlates of urgency such as 'now!' or 'careful!'.
- The paper's own future-work list — real human preference feedback, multimodal emotion signals beyond language, and closed-loop simulator evaluation — concedes that the current emotion signal is text-only and never verified in a live vehicle; if the VAD-to-geometry correlation is a regularity of the synthetic training labels rather than a genuine passenger response, the comfort benefits would need
- If the arousal–geometry regularity survives human re-annotation, the same model could be calibrated to an individual passenger's preferred arousal-response curve, turning a one-size-fits-all emotion mapping into a personal comfort setting.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes E3AD, an emotion-aware vision-language-action (VLA) model for open-domain end-to-end autonomous driving. E3AD augments a Qwen2.5-VL backbone with continuous Valence-Arousal-Dominance (VAD) emotion modeling and a dual-pathway spatial reasoning module that fuses egocentric and allocentric views. A three-stage training scheme—modality pretraining, joint fine-tuning, and DPO-based emotion-action alignment—is used to enforce coherence between emotional intent, grounding, and trajectory generation. Experiments on Talk2Car, DrivePilot, MoCAD, and Talk2Car-Trajectory report improvements in visual grounding, waypoint planning, and emotion estimation, along with a user study indicating preference for E3AD's outputs.
Significance. If the central claims hold, E3AD would be a valuable step toward human-centric autonomous driving, demonstrating that continuous emotion vectors can condition both referent grounding and trajectory planning. The paper is also useful for its detailed experimental setup, including data construction details in Appendix A, prompt templates in Appendix E, and an ablation study isolating core components. The user study provides some independent evidence of perceived quality. However, the validity of the automatically generated VAD labels is load-bearing for the emotion modeling claims, and several design choices (automated label fusion, self-sampled DPO negatives, filtered spatial supervision) need to be examined before the main novelty is established.
major comments (4)
- [§3.3, Appendix A.1, Table 3] The VAD labels used both as supervision (Eq. 2) and as the evaluation target in Table 3 are generated by the paper's own pipeline: a RoBERTa GoEmotions classifier mapped through NRC-VAD, fused with TF-IDF-weighted word-level NRC-VAD scores via α=0.5 and an exclamation boost. No human validation of these labels is reported. Table 3 therefore measures correlation with an algorithmic labeler, not with human-perceived emotion. The high ρ≈0.94–0.95 values may largely reflect the model's ability to imitate the label-generation pipeline. This circularity also propagates to the emotion-to-planning claims in Fig. 5 and Table A1, which use the same labels. Please provide a human-annotation study on a held-out subset of commands: compare automated labels and E3AD predictions to human VAD ratings, and report both the labeler's agreement with humans and the model's correlation with human ratings. Wit
- [§3.6, Eqs. (4)–(5)] The pseudo-preference pairs for DPO are constructed by sampling the dispreferred trajectory eτ(i)_{k-} from p_θ, i.e., from the model being trained, conditioned on an emotion-augmented negative command. The manuscript does not state whether these negative samples are generated once before DPO training with a frozen model or resampled during training. If resampled, the preference signal depends on the current policy and the optimization can become degenerate or self-confirming. Moreover, the selection of the "most deviating" negative command is based on the same VAD labels whose validity is in question. Please clarify the sampling protocol and, if possible, use a fixed set of negative trajectories generated by a frozen reference model. Report the sensitivity of the DPO results to the choice of negative sampling strategy.
- [Appendix A.2, Table 4] The egocentric spatial relation and depth estimation supervision are built from FCOS3D detections filtered by a high confidence threshold and further restricted to scenes with no duplicate object classes. This exclusion of ambiguous multi-object cases may make the spatial reasoning task easier than the general Talk2Car setting used for the VLM baselines in Table 4. The table does not state whether evaluation is performed on the same filtered subset for all models. If E3AD is evaluated only on unambiguous samples while the Qwen baselines are evaluated on the full set, the reported MAE/PA gains are not comparable. Please report the exact number of samples used for training and evaluation, and either evaluate all models on the same filtered/unfiltered split or provide separate results on the full set.
- [§3.3, Appendix A.1] The emotion-augmented paraphrases are generated by Qwen2.5-VL with the instruction to preserve driving intent while varying tone, but no human verification or automatic check is reported for whether the paraphrases actually preserve semantics. If a paraphrase changes the referent or the action, the model is trained to associate a changed VAD label with a changed intent, conflating tone and semantics. This undermines the claim that the model learns to separate emotion from intent. Please provide a small-scale human evaluation of paraphrase fidelity (e.g., percentage of paraphrases where the driving goal is preserved) or an automatic filter based on semantic similarity, and report the effect of such filtering on downstream results.
minor comments (5)
- [Eq. (3)] The expectation in L_joint is written as -E_{(I,C,T)} without specifying the distribution (presumably the empirical data distribution). Please clarify.
- [Eq. (4)] The notation eτ(i)_{k-} is confusing; it appears to denote a sampled trajectory, not a variable named eτ. Please use a clearer symbol such as τ̃.
- [Tables 3 and 4] The model name is inconsistent: Qwen2.5-VL-7B-Instruct in the text vs. Qwen2.5-VL-7B in tables. Please standardize.
- [§4.4 and Table A1] The main text refers to Fig. 5 for emotion-trajectory consistency, but the quantitative metric definitions and full table are only in Appendix D.2. Consider moving a compact version to the main text or clearly referencing the appendix. Also, in Table 5 the reported gains for the Emotion Modeling branch on ambiguous/long-text are stated as +4.5%/+4.8% without specifying whether these are absolute or relative; the table suggests relative gains. Please clarify.
- [§4.7 and Appendix C] The user study reports mean Likert scores and Rank-1 counts but no statistical significance tests (e.g., pairwise comparisons with confidence intervals). Given 217 participants, adding significance tests would strengthen the preference claims.
Circularity Check
Emotion estimation is evaluated against the paper's own automatically generated VAD labels, making the SOTA emotion correlation partially self-referential; grounding/planning results remain independently benchmarked.
specific steps
-
self definitional
[§4.3 Table 3 caption; Appendix A.1 (VAD label construction); §3.3 Eq. (2)]
"Reported metrics are Spearman’s ρ and Kendall’s τ correlations with ground-truth VAD. ... The final label is obtained via linear fusion: efinal = α×e_goe + (1−α)×e_words, with α=0.5. ... E3AD achieves the highest correlation with human annotations, reaching 0.95/0.84 (ρ, τ) for valence."
The same efinal defined in Appendix A.1 is used as the supervision target in Lemo (Eq. 2) and as the ground truth for the emotion evaluation in Table 3. Because efinal is produced by the paper's own GoEmotions-to-NRC-VAD plus TF-IDF/NRC-VAD fusion pipeline, with no direct human VAD ratings of the driving commands reported, the ρ≈0.94–0.95 result measures how well E3AD imitates this algorithmic labeler rather than independently validating human emotion understanding. The claim of 'correlation with human annotations' and the derived emotion-to-planning conclusions therefore inherit the validity of the auto-generated labels. Grounding and planning metrics (Tables 1–2, 5–6) use human-annotated targets and are not affected by this circularity.
full rationale
The core grounding and planning claims are evaluated on externally annotated benchmarks (Talk2Car, MoCAD, Talk2Car-Trajectory) and are not circular. The emotion sub-task, however, is partially self-referential: the VAD labels are constructed by the paper's own fusion formula, used to train the emotion head, and then used as the Table 3 evaluation target. This makes the reported emotion correlations an internal consistency check of the label-generation pipeline, not an independent human-alignment result. The DPO pseudo-preferences use the model's own sampled trajectories as rejects (Eq. 4), but the emotion-trajectory consistency evaluation in Table A1 is anchored to real ground-truth trajectories, and the user study provides independent human preference evidence, though it does not isolate the VAD mechanism. Self-citations are present but not load-bearing for the main derivations. Overall score 5 reflects one partially circular sub-result while the central grounding/planning findings retain independent content.
Axiom & Free-Parameter Ledger
free parameters (5)
- VAD label fusion weight α =
0.5
- Exclamation Boost increment =
not specified
- Number of emotion-augmented paraphrases K =
not stated
- FCOS3D confidence threshold =
>0.3
- DPO temperature β =
not reported
axioms (5)
- domain assumption GoEmotions classifier and NRC-VAD lexicon provide valid continuous emotion labels for driving commands.
- ad hoc to paper Emotion-augmented paraphrases generated by Qwen2.5-VL preserve the original driving goal while changing only tone.
- ad hoc to paper A trajectory sampled from the current policy conditioned on the negative emotion command is a valid dispreferred example for DPO.
- domain assumption FCOS3D detections, after confidence filtering and duplicate-class exclusion, provide accurate 3D ground truth for egocentric spatial reasoning.
- domain assumption VAD axes (valence, arousal, dominance) are behaviorally meaningful for driving command urgency and should condition trajectory geometry.
read the original abstract
End-to-end autonomous driving (AD) systems increasingly adopt vision-language-action (VLA) models, yet they typically ignore the passenger's emotional state, which is central to comfort and AD acceptance. We introduce Open-Domain End-to-End (OD-E2E) autonomous driving, where an autonomous vehicle (AV) must interpret free-form natural-language commands, infer the emotion, and plan a physically feasible trajectory. We propose E3AD, an emotion-aware VLA framework that augments semantic understanding with two cognitively inspired components: a continuous Valenc-Arousal-Dominance (VAD) emotion model that captures tone and urgency from language, and a dual-pathway spatial reasoning module that fuses egocentric and allocentric views for human-like spatial cognition. A consistency-oriented training scheme, combining modality pretraining with preference-based alignment, further enforces coherence between emotional intent and driving actions. Across real-world datasets, E3AD improves visual grounding and waypoint planning and achieves state-of-the-art (SOTA) VAD correlation for emotion estimation. These evaluation results show that injecting emotion into VLA-style driving yields more human-aligned grounding, planning, and feedback.
Figures
Reference graph
Works this paper leans on
-
[1]
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025. 3, 6, 7
Pith/arXiv arXiv 2025
-
[2]
Spatial memory: how egocentric and allocentric combine.Trends in cognitive sciences, 10(12):551–557, 2006
Neil Burgess. Spatial memory: how egocentric and allocentric combine.Trends in cognitive sciences, 10(12):551–557, 2006. 4
2006
-
[3]
nuscenes: A multi- modal dataset for autonomous driving
Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Gi- ancarlo Baldan, and Oscar Beijbom. nuscenes: A multi- modal dataset for autonomous driving. InProceedings of the IEEE/CVF CVPR, pages 11621–11631, 2020. 14
2020
-
[4]
Ground- ing commands for autonomous vehicles via layer fusion with region-specific dynamic layer attention
Hou Pong Chan, Mingxi Guo, and Cheng-Zhong Xu. Ground- ing commands for autonomous vehicles via layer fusion with region-specific dynamic layer attention. In2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 12464–12470. IEEE, 2022. 6
2022
-
[5]
Eeg-based emotion recognition for road accidents in a simulated driving environment.Biomedical signal processing and control, 87:105411, 2024
Jielin Chen, Xuefen Lin, Weifeng Ma, Yuchen Wang, and Wei Tang. Eeg-based emotion recognition for road accidents in a simulated driving environment.Biomedical signal processing and control, 87:105411, 2024. 2
2024
-
[6]
End-to-end autonomous driving: Challenges and frontiers.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024
Li Chen, Penghao Wu, Kashyap Chitta, Bernhard Jaeger, Andreas Geiger, and Hongyang Li. End-to-end autonomous driving: Challenges and frontiers.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 1
2024
-
[7]
Emotion-aware design in automobiles: Embracing technology advancements to enhance human-vehicle interaction
Xingtong Chen, Xia Wang, Cong Fang, Le Fang, Wei Gong, Chengzhong Liu, and Stephen Jia Wang. Emotion-aware design in automobiles: Embracing technology advancements to enhance human-vehicle interaction. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems, pages 1–18, 2025. 1
2025
-
[8]
Emotion recognition in human-computer interaction
Roddy Cowie, Ellen Douglas-Cowie, Nicolas Tsapatsoulis, George V otsis, Stefanos Kollias, Winfried Fellenz, and John G Taylor. Emotion recognition in human-computer interaction. IEEE Signal processing magazine, 18(1):32–80, 2001. 2
2001
-
[9]
A survey on multimodal large language models for autonomous driving
Can Cui, Yunsheng Ma, Xu Cao, Wenqian Ye, Yang Zhou, Kaizhao Liang, Jintai Chen, Juanwu Lu, Zichong Yang, Kuei- Da Liao, et al. A survey on multimodal large language models for autonomous driving. InProceedings of the IEEE/CVF winter conference on applications of computer vision, pages 958–979, 2024. 2
2024
-
[10]
Com- mands for autonomous vehicles by progressively stacking visual-linguistic representations
Hang Dai, Shujie Luo, Yong Ding, and Ling Shao. Com- mands for autonomous vehicles by progressively stacking visual-linguistic representations. InComputer Vision– ECCV Workshops, pages 27–32, 2020. 6
2020
-
[11]
Goemo- tions: A dataset of fine-grained emotions.arXiv preprint arXiv:2005.00547, 2020
Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. Goemo- tions: A dataset of fine-grained emotions.arXiv preprint arXiv:2005.00547, 2020. 4
Pith/arXiv arXiv 2005
-
[12]
Goal- gan: Multimodal trajectory prediction based on goal position estimation
Patrick Dendorfer, Aljosa Osep, and Laura Leal-Taix´e. Goal- gan: Multimodal trajectory prediction based on goal position estimation. InProceedings of the Asian Conference on Com- puter Vision, 2020. 6
2020
-
[13]
Transvg: End-to-end visual ground- ing with transformers
Jiajun Deng, Zhengyuan Yang, Tianlang Chen, Wengang Zhou, and Houqiang Li. Transvg: End-to-end visual ground- ing with transformers. InProceedings of the IEEE/CVF ICCV, pages 1769–1779, 2021. 6
2021
-
[14]
Talk2car: Taking con- trol of your self-driving car.arXiv preprint arXiv:1909.10838,
Thierry Deruyttere, Simon Vandenhende, Dusan Grujicic, Luc Van Gool, and Marie-Francine Moens. Talk2car: Taking con- trol of your self-driving car.arXiv preprint arXiv:1909.10838,
Pith/arXiv arXiv 1909
-
[15]
Talk2car: Predicting physical trajectories for natural language commands.Ieee Access, 10: 123809–123834, 2022
Thierry Deruyttere, Dusan Grujicic, Matthew B Blaschko, and Marie-Francine Moens. Talk2car: Predicting physical trajectories for natural language commands.Ieee Access, 10: 123809–123834, 2022. 5, 14, 15
2022
-
[16]
Bert: Pre-training of deep bidirectional transform- ers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transform- ers for language understanding. InProceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171–4186, 2019. 7
2019
-
[17]
Multi-class emotion recognition within the valence-arousal-dominance space using eeg
Marvin Gaertner, Daniel Sauter, Hermann Baumgartl, Thilo Rieg, and Ricardo Buettner. Multi-class emotion recognition within the valence-arousal-dominance space using eeg. In AMCIS, 2021. 2
2021
-
[18]
Predicting physical world destina- tions for commands given to self-driving cars
Dusan Grujicic, Thierry Deruyttere, Marie-Francine Moens, and Matthew B Blaschko. Predicting physical world destina- tions for commands given to self-driving cars. InProceedings of the AAAI Conference on Artificial Intelligence, pages 715– 725, 2022. 5, 6
2022
-
[19]
Think before you drive: World model-inspired multimodal grounding for autonomous driving.arXiv preprint, 2025
Liao Haicheng, Shen Huanming, Wang Bonan, Li Yongkang, Tang Yihong, Wang Chengyue, Zhuang Dingyi, Chen Kehua, Yang Hai, Xu Chengzhong, and Li Zhenning. Think before you drive: World model-inspired multimodal grounding for autonomous driving.arXiv preprint, 2025. 5, 14
2025
-
[20]
A formal basis for the heuristic determination of minimum cost paths
Peter E Hart, Nils J Nilsson, and Bertram Raphael. A formal basis for the heuristic determination of minimum cost paths. IEEE transactions on Systems Science and Cybernetics, 4(2): 100–107, 1968. 6
1968
-
[21]
Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In ICLR, 2022. 5
2022
-
[22]
Planning-oriented autonomous driving
Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, et al. Planning-oriented autonomous driving. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17853–17862, 2023. 1
2023
-
[23]
Think twice before driving: Towards scalable decoders for end-to-end autonomous driving
Xiaosong Jia, Penghao Wu, Li Chen, Jiangwei Xie, Conghui He, Junchi Yan, and Hongyang Li. Think twice before driving: Towards scalable decoders for end-to-end autonomous driving. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21983–21994, 2023. 1
2023
-
[24]
Bo Jiang, Shaoyu Chen, Bencheng Liao, Xingyu Zhang, Wei Yin, Qian Zhang, Chang Huang, Wenyu Liu, and Xinggang Wang. Senna: Bridging large vision-language models and end- to-end autonomous driving.arXiv preprint arXiv:2410.22313,
-
[25]
Sicong Jiang, Zilin Huang, Kangan Qian, Ziang Luo, Tianze Zhu, Yang Zhong, Yihong Tang, Menglin Kong, Yunlong Wang, Siwen Jiao, et al. A survey on vision-language- action models for autonomous driving.arXiv preprint arXiv:2506.24044, 2025. 1, 2
Pith/arXiv arXiv 2025
-
[26]
Mdetr-modulated detection for end-to-end multi-modal understanding
Aishwarya Kamath, Mannat Singh, Yann LeCun, Gabriel Syn- naeve, Ishan Misra, and Nicolas Carion. Mdetr-modulated detection for end-to-end multi-modal understanding. InPro- ceedings of the IEEE/CVF ICCV, pages 1780–1790, 2021. 6
2021
-
[27]
Detection of drivers’ anxiety invoked by driving situations using multimodal biosignals.Processes, 8 (2):155, 2020
Seungji Lee, Taejun Lee, Taeyang Yang, Changrak Yoon, and Sung-Phil Kim. Detection of drivers’ anxiety invoked by driving situations using multimodal biosignals.Processes, 8 (2):155, 2020. 3
2020
-
[28]
Review and perspectives on human emotion for connected automated vehicles.Automotive Innovation, 7(1):4–44, 2024
Wenbo Li, Guofa Li, Ruichen Tan, Cong Wang, Zemin Sun, Ying Li, Gang Guo, Dongpu Cao, and Keqiang Li. Review and perspectives on human emotion for connected automated vehicles.Automotive Innovation, 7(1):4–44, 2024. 2
2024
-
[29]
Yingyan Li, Shuyao Shang, Weisong Liu, Bing Zhan, Haochen Wang, Yuqi Wang, Yuntao Chen, Xiaoman Wang, Yasong An, Chufeng Tang, et al. Drivevla-w0: World mod- els amplify data scaling law in autonomous driving.arXiv preprint arXiv:2510.12796, 2025. 1
Pith/arXiv arXiv 2025
-
[30]
Fine-grained evaluation of large vision-language mod- els in autonomous driving
Yue Li, Meng Tian, Zhenyu Lin, Jiangtong Zhu, Dechang Zhu, Haiqiang Liu, Yueyi Zhang, Zhiwei Xiong, and Xinhai Zhao. Fine-grained evaluation of large vision-language mod- els in autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 9431– 9442, 2025. 2
2025
-
[31]
Yongkang Li, Kaixin Xiong, Xiangyu Guo, Fang Li, Sixu Yan, Gangwei Xu, Lijun Zhou, Long Chen, Haiyang Sun, Bing Wang, et al. Recogdrive: A reinforced cognitive frame- work for end-to-end autonomous driving.arXiv preprint arXiv:2506.08052, 2025. 1
Pith/arXiv arXiv 2025
-
[32]
Steering the future: Redefining intelligent transportation systems with foundation models.Chain, 1(1):46–53, 2024
Zhenning Li, Zhiyong Cui, Haicheng Liao, John Ash, Guohui Zhang, Chengzhong Xu, and Yinhai Wang. Steering the future: Redefining intelligent transportation systems with foundation models.Chain, 1(1):46–53, 2024. 1, 2
2024
-
[33]
Mamba-va: A mamba-based approach for continuous emotion recognition in valence-arousal space
Yuheng Liang, Zheyu Wang, Feng Liu, Mingzhou Liu, and Yu Yao. Mamba-va: A mamba-based approach for continuous emotion recognition in valence-arousal space. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 5651–5656, 2025. 3
2025
-
[34]
Gpt- 4 enhanced multimodal grounding for autonomous driving: Leveraging cross-modal attention with large language models
Haicheng Liao, Huanming Shen, Zhenning Li, Chengyue Wang, Guofa Li, Yiming Bie, and Chengzhong Xu. Gpt- 4 enhanced multimodal grounding for autonomous driving: Leveraging cross-modal attention with large language models. Communications in Transportation Research, 4:100116, 2024. 3, 5, 6, 14
2024
-
[35]
Cot-drive: Efficient motion forecasting for autonomous driving with llms and chain-of-thought prompting.IEEE Transactions on Artificial Intelligence, 2025
Haicheng Liao, Hanlin Kong, Bonan Wang, Chengyue Wang, Wang Ye, Zhengbing He, Chengzhong Xu, and Zhenning Li. Cot-drive: Efficient motion forecasting for autonomous driving with llms and chain-of-thought prompting.IEEE Transactions on Artificial Intelligence, 2025. 2
2025
-
[36]
Toward human-like trajectory predic- tion for autonomous driving: A behavior-centric approach
Haicheng Liao, Zhenning Li, Guohui Zhang, Keqiang Li, and Chengzhong Xu. Toward human-like trajectory predic- tion for autonomous driving: A behavior-centric approach. Transportation Science, 2025. 1
2025
-
[37]
Pei Liu, Qingtian Ning, Xinyan Lu, Haipeng Liu, Weil- iang Ma, Dangen She, Peng Jia, Xianpeng Lang, and Jun Ma. Omnireason: A temporal-guided vision-language- action framework for autonomous driving.arXiv preprint arXiv:2509.00789, 2025. 2
Pith/arXiv arXiv 2025
-
[38]
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection.arXiv preprint arXiv:2303.05499, 2023. 6
Pith/arXiv arXiv 2023
-
[39]
Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692,
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettle- moyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692,
Pith/arXiv arXiv 1907
-
[40]
C4av: learning cross-modal representations from transformers
Shujie Luo, Hang Dai, Ling Shao, and Yong Ding. C4av: learning cross-modal representations from transformers. In Computer Vision–ECCV 2020, pages 33–38, 2020. 6
2020
-
[41]
From goals, waypoints & paths to long term human trajectory forecasting
Karttikeya Mangalam, Yang An, Harshayu Girase, and Jiten- dra Malik. From goals, waypoints & paths to long term human trajectory forecasting. InProceedings of the IEEE/CVF inter- national conference on computer vision, pages 15233–15242,
-
[42]
Person- ality correlates of driver stress.Personality and Individual Differences, 12(6):535–549, 1991
Gerald Matthews, Lisa Dorn, and A Ian Glendon. Person- ality correlates of driver stress.Personality and Individual Differences, 12(6):535–549, 1991. 4
1991
-
[43]
Attngrounder: Talking to cars with attention
Vivek Mittal. Attngrounder: Talking to cars with attention. In Computer Vision– ECCV Workshops, pages 62–73, 2020. 6
2020
-
[44]
Mohammad
Saif M. Mohammad. Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 english words. InProceedings of The Annual Conference of the Association for Computational Linguistics (ACL), Melbourne, Australia,
-
[45]
Driver emotion recognition with a hybrid attentional multimodal fusion framework.IEEE Transactions on Affective Computing, 14(4):2970–2981, 2023
Luntian Mou, Yiyuan Zhao, Chao Zhou, Bahareh Nakisa, Mohammad Naim Rastgoo, Lei Ma, Tiejun Huang, Baocai Yin, Ramesh Jain, and Wen Gao. Driver emotion recognition with a hybrid attentional multimodal fusion framework.IEEE Transactions on Affective Computing, 14(4):2970–2981, 2023. 2
2023
-
[46]
Steerable adversarial scenario generation through test-time preference alignment
Tong Nie, Yuewen Mei, Yihong Tang, Junlin He, Jie Sun, Haotian Shi, Wei Ma, and Jian Sun. Steerable adversarial scenario generation through test-time preference alignment. arXiv preprint arXiv:2509.20102, 2025. 1
Pith/arXiv arXiv 2025
-
[47]
Vlp: Vision language planning for autonomous driving
Chenbin Pan, Burhaneddin Yaman, Tommaso Nesti, Abhirup Mallik, Alessandro G Allievi, Senem Velipasalar, and Liu Ren. Vlp: Vision language planning for autonomous driving. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14760–14769, 2024. 2
2024
-
[48]
Multi- modal fusion transformer for end-to-end autonomous driving
Aditya Prakash, Kashyap Chitta, and Andreas Geiger. Multi- modal fusion transformer for end-to-end autonomous driving. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7077–7087, 2021. 1
2021
-
[49]
Direct prefer- ence optimization: Your language model is secretly a reward model.Advances in neural information processing systems, 36:53728–53741, 2023
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct prefer- ence optimization: Your language model is secretly a reward model.Advances in neural information processing systems, 36:53728–53741, 2023. 5
2023
-
[50]
Simlingo: Vision-only closed-loop autonomous driving with language-action alignment
Katrin Renz, Long Chen, Elahe Arani, and Oleg Sinavski. Simlingo: Vision-only closed-loop autonomous driving with language-action alignment. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 11993– 12003, 2025. 2
2025
-
[51]
Cosine meets softmax: A tough-to-beat baseline for visual grounding
Nivedita Rufus, Unni Krishnan R Nair, K Madhava Krishna, and Vineet Gandhi. Cosine meets softmax: A tough-to-beat baseline for visual grounding. InComputer Vision– ECCV Workshops, pages 39–50, 2020. 6
2020
-
[52]
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.arXiv preprint arXiv:1910.01108, 2019. 7
Pith/arXiv arXiv 1910
-
[53]
Ranjan Sapkota, Yang Cao, Konstantinos I Roumeliotis, and Manoj Karkee. Vision-language-action models: Con- cepts, progress, applications and challenges.arXiv preprint arXiv:2505.04769, 2025. 2
arXiv 2025
-
[54]
Lmdrive: Closed-loop end-to-end driving with large language models
Hao Shao, Yuxuan Hu, Letian Wang, Guanglu Song, Steven L Waslander, Yu Liu, and Hongsheng Li. Lmdrive: Closed-loop end-to-end driving with large language models. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15120–15130, 2024. 2
2024
-
[55]
Passengers’ emotions recognition to improve social acceptance of autonomous driving vehicles
Jacopo Sini, Antonio Costantino Marceddu, Massimo Vi- olante, and Riccardo Dess`ı. Passengers’ emotions recognition to improve social acceptance of autonomous driving vehicles. InProgresses in Artificial Intelligence and Neural Systems, pages 25–32. Springer, 2020. 2
2020
-
[56]
Liang Tan, Keping Yu, Long Lin, Xiaofan Cheng, Gau- tam Srivastava, Jerry Chun-Wei Lin, and Wei Wei. Speech emotion recognition enhanced traffic efficiency solution for autonomous vehicles in a 5g-enabled space–air–ground in- tegrated intelligent transportation system.IEEE Transac- tions on Intelligent Transportation Systems, 23(3):2830–2842,
-
[57]
Yihong Tang and Wei Ma. Intent: Trajectory prediction framework with intention-guided contrastive clustering.arXiv preprint arXiv:2503.04952, 2025. 2
Pith/arXiv arXiv 2025
-
[58]
Itinera: Integrating spatial optimization with large language models for open-domain urban itinerary planning
Yihong Tang, Zhaokai Wang, Ao Qu, Yihao Yan, Zhaofeng Wu, Dingyi Zhuang, Jushi Kai, Kebing Hou, Xiaotong Guo, Jinhua Zhao, et al. Itinera: Integrating spatial optimization with large language models for open-domain urban itinerary planning. InProceedings of the 2024 Conference on Empiri- cal Methods in Natural Language Processing: Industry Track, pages 14...
2024
-
[59]
Sparkle: Mastering basic spatial ca- pabilities in vision language models elicits generalization to spatial reasoning
Yihong Tang, Ao Qu, Zhaokai Wang, Dingyi Zhuang, Zhaofeng Wu, Wei Ma, Shenhao Wang, Yunhan Zheng, Zhan Zhao, and Jinhua Zhao. Sparkle: Mastering basic spatial ca- pabilities in vision language models elicits generalization to spatial reasoning. InFindings of the Association for Compu- tational Linguistics: EMNLP 2025, pages 4083–4103, 2025. 2
2025
-
[60]
Qwen2.5: A party of foundation models, 2024
Qwen Team. Qwen2.5: A party of foundation models, 2024. 7
2024
-
[61]
Xiaoyu Tian, Junru Gu, Bailin Li, Yicheng Liu, Yang Wang, Zhiyong Zhao, Kun Zhan, Peng Jia, Xianpeng Lang, and Hang Zhao. Drivevlm: The convergence of autonomous driving and large vision-language models.arXiv preprint arXiv:2402.12289, 2024. 1
Pith/arXiv arXiv 2024
-
[62]
Fcos3d: Fully convolutional one-stage monocular 3d object detection
Tai Wang, Xinge Zhu, Jiangmiao Pang, and Dahua Lin. Fcos3d: Fully convolutional one-stage monocular 3d object detection. InProceedings of the IEEE/CVF international conference on computer vision, pages 913–922, 2021. 14
2021
-
[63]
Norms of valence, arousal, and dominance for 13,915 english lemmas.Behavior research methods, 45(4):1191–1207, 2013
Amy Beth Warriner, Victor Kuperman, and Marc Brysbaert. Norms of valence, arousal, and dominance for 13,915 english lemmas.Behavior research methods, 45(4):1191–1207, 2013. 4
2013
-
[64]
Driver multi-task emotion recognition network based on multi-modal facial video analysis.Pattern Recognition, 161:111241, 2025
Guoliang Xiang, Song Yao, Xianhui Wu, Hanwen Deng, Guo- jie Wang, Yu Liu, Fan Li, and Yong Peng. Driver multi-task emotion recognition network based on multi-modal facial video analysis.Pattern Recognition, 161:111241, 2025. 3
2025
-
[65]
On-road driver emotion recognition using facial expression
Huafei Xiao, Wenbo Li, Guanzhong Zeng, Yingzhang Wu, Jiyong Xue, Juncheng Zhang, Chengmou Li, and Gang Guo. On-road driver emotion recognition using facial expression. Applied Sciences, 12(2):807, 2022. 2
2022
-
[66]
Openemma: Open-source multimodal model for end-to-end autonomous driving
Shuo Xing, Chengyuan Qian, Yuping Wang, Hongyuan Hua, Kexin Tian, Yang Zhou, and Zhengzhong Tu. Openemma: Open-source multimodal model for end-to-end autonomous driving. InProceedings of the Winter Conference on Applica- tions of Computer Vision, pages 1001–1009, 2025. 2
2025
-
[67]
Drivegpt4: Interpretable end-to-end autonomous driving via large language model.IEEE Robotics and Automation Letters,
Zhenhua Xu, Yujia Zhang, Enze Xie, Zhen Zhao, Yong Guo, Kwan-Yee K Wong, Zhenguo Li, and Hengshuang Zhao. Drivegpt4: Interpretable end-to-end autonomous driving via large language model.IEEE Robotics and Automation Letters,
-
[68]
Universal instance percep- tion as object discovery and retrieval
Bin Yan, Yi Jiang, Jiannan Wu, Dong Wang, Ping Luo, Ze- huan Yuan, and Huchuan Lu. Universal instance percep- tion as object discovery and retrieval. InProceedings of the IEEE/CVF CVPR, pages 15325–15336, 2023. 6
2023
-
[69]
Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025. 6, 7
Pith/arXiv arXiv 2025
-
[70]
Improving visual grounding with visual- linguistic verification and iterative reasoning
Li Yang, Yan Xu, Chunfeng Yuan, Wei Liu, Bing Li, and Weiming Hu. Improving visual grounding with visual- linguistic verification and iterative reasoning. InProceedings of the IEEE/CVF CVPR, pages 9499–9508, 2022. 6
2022
-
[71]
Human-centric autonomous systems with llms for user command reasoning
Yi Yang, Qingwen Zhang, Ci Li, Daniel Sim˜oes Marta, Nazre Batool, and John Folkesson. Human-centric autonomous systems with llms for user command reasoning. InProceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 988–994, 2024. 3
2024
-
[72]
Zhenjie Yang, Yilin Chai, Xiaosong Jia, Qifeng Li, Yuqian Shao, Xuekai Zhu, Haisheng Su, and Junchi Yan. Drivemoe: Mixture-of-experts for vision-language-action model in end- to-end autonomous driving.arXiv preprint arXiv:2505.16278,
-
[73]
Shuang Zeng, Xinyuan Chang, Mengwei Xie, Xinran Liu, Yi- fan Bai, Zheng Pan, Mu Xu, and Xing Wei. Futuresightdrive: Thinking visually with spatio-temporal cot for autonomous driving.arXiv preprint arXiv:2505.17685, 2025. 2, 6, 8
Pith/arXiv arXiv 2025
-
[74]
Driver emotion recog- nition for intelligent vehicles: A survey.ACM Computing Surveys (CSUR), 53(3):1–30, 2020
Sebastian Zepf, Javier Hernandez, Alexander Schmitt, Wolf- gang Minker, and Rosalind W Picard. Driver emotion recog- nition for intelligent vehicles: A survey.ACM Computing Surveys (CSUR), 53(3):1–30, 2020. 1
2020
-
[75]
A comprehensive review: Multisen- sory and cross-cultural approaches to driver emotion modula- tion in vehicle systems.Applied Sciences, 14(15):6819, 2024
Jieshu Zhang, Raja Ariffin Bin Raja Ghazilla, Hwa Jen Yap, and Woun Yoong Gan. A comprehensive review: Multisen- sory and cross-cultural approaches to driver emotion modula- tion in vehicle systems.Applied Sciences, 14(15):6819, 2024. 1
2024
-
[76]
Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, et al. Qwen3 embedding: Advancing text embedding and reranking through foundation models.arXiv preprint arXiv:2506.05176, 2025. 7
Pith/arXiv arXiv 2025
-
[77]
Where are you heading? dynamic trajectory prediction with expert goal examples
He Zhao and Richard P Wildes. Where are you heading? dynamic trajectory prediction with expert goal examples. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7629–7638, 2021. 6
2021
-
[78]
A survey of autonomous driving from a deep learning perspective.ACM Computing Surveys, 57(10): 1–60, 2025
Jingyuan Zhao, Yuyan Wu, Rui Deng, Susu Xu, Jinpeng Gao, and Andrew Burke. A survey of autonomous driving from a deep learning perspective.ACM Computing Surveys, 57(10): 1–60, 2025. 1
2025
-
[79]
Swift: A scalable lightweight infrastructure for fine-tuning.arXiv preprint arXiv:2408.05517, 2024
Yuze Zhao, Jintao Huang, Jinghan Hu, Daoze Zhang, Zeyinzi Jiang, Zhikai Wu, Baole Ai, Ang Wang, Wenmeng Zhou, and Yingda Chen. Swift: A scalable lightweight infrastructure for fine-tuning.arXiv preprint arXiv:2408.05517, 2024. 5
Pith/arXiv arXiv 2024
-
[80]
Opendrivevla: Towards end-to-end au- tonomous driving with large vision language action model
Xingcheng Zhou, Xuyuan Han, Feng Yang, Yunpu Ma, and Alois C Knoll. Opendrivevla: Towards end-to-end au- tonomous driving with large vision language action model. arXiv preprint arXiv:2503.23463, 2025. 2
arXiv 2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.