Pith. sign in

REVIEW 3 major objections 5 minor 32 references

SenseRAG: Constructing Environmental Knowledge Bases with Proactive Querying for LLM-Based Autonomous Driving

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SenseRAG, a retrieval-augmented driving framework, cuts trajectory prediction error by 76.5% ADE and 72.2% FDE on the DLR Urban Traffic dataset.

desk verdict The 70% error reduction is a confound, not a validation: the RAG condition gets ground-truth kinematics while the baseline doesn't; the framework itself is a reasonable but unvalidated system sketch. read the letter →

arxiv 2501.03535 v2 pith:CYBKYGAI submitted 2025-01-07 cs.AI cs.RO

Xuewen Luo , Fan Ding , Fengze Yang , Yang Zhou , Junnyong Loo , Hwa Hui Tew , Chenxi Liu This is my paper · ORCID
classification cs.AIcs.RO
keywords retrieval-augmentedgenerationautonomousdrivinglargelanguagemodelssituationalawarenesschain-of-thoughtpromptingSQLretrievalvehicle-to-everythingtrajectoryprediction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes that an LLM-based autonomous driving system can achieve dramatically better trajectory prediction when it is allowed to actively query a structured environmental knowledge base instead of relying only on its own onboard perception. The key idea is a proactive retrieval loop: the LLM notices what it cannot see, expresses the missing information as a natural-language query, translates it to SQL, retrieves the relevant records from a database that fuses camera, LiDAR, signal-timing, and weather data, and folds the retrieved facts back into its reasoning. On the DLR Urban Traffic dataset, this SenseRAG approach reduces average displacement error (ADE) by 76.5% and final displacement error (FDE) by 72.2% relative to a GPT-4 baseline that sees only self-perception data. The work matters because it treats information access, not just model scale, as the lever for situation awareness in autonomous vehicles.

What carries the argument

The load-bearing mechanism is the proactive retrieval loop, formalized as $\hat{P} = \text{LLM}(\text{Combine}(S, E))$ with $E = \text{Search}(S, Q(S))$, where $S$ is the ego vehicle's self-perception data and $Q(S)$ is a query produced by chain-of-thought reasoning. The loop runs in four stages: the LLM inspects $S$ to decide which environmental facts would resolve its uncertainty; it writes a natural-language query; the query is compiled into SQL and executed against a knowledge database; and the retrieved records are verbalized back into the prompt as ordinary text. The second load-bearing piece is the knowledge database itself, which harmonizes unstructured camera and LiDAR inputs with structured signal-timing and weather records into a standardized, language-compatible schema with spatial and temporal indexes, making the targeted SQL retrieval possible.

What would settle it

Give the GPT-4 baseline the same out-of-perception ground-truth kinematics and signal-timing data directly as plain text in its prompt, without any retrieval, SQL, or chain-of-thought machinery. If its ADE and FDE drop by roughly the same 76.5% and 72.2%, then the reported improvement is explained by information access rather than by the SenseRAG framework itself.

Watch

Extended reading notes

Core claim

The central claim is that coupling chain-of-thought query generation with SQL-based retrieval from a unified multimodal knowledge base materially improves LLM-based trajectory prediction. In the authors' controlled comparison, the SenseRAG-enhanced GPT-4 achieves an ADE of 0.1564 at the 3-timestamp horizon versus 0.7531 for the baseline, and the gap widens at longer horizons (2.1410 versus 8.5083 at timestamp 10); overall ADE and FDE fall by 76.5% and 72.2%. The authors attribute the improvement to the model's ability to identify gaps in its own perception, issue targeted queries about traffic signals, weather, and nearby vehicles beyond its 30-meter visible range, and integrate the returned information as verbalized context. This is presented as evidence that retrieval-augmented generation can deliver the flexible, context-rich understanding that fixed label-based perception systems lack.

Load-bearing premise

The evaluation assumes that the baseline and the SenseRAG model differ only by whether retrieval is used, but the retrieval database actually provides ground-truth positions, velocities, and accelerations of vehicles beyond the ego vehicle's 30-meter perception range, which the baseline is never allowed to see.

Editorial extensions

If this is right

  • The paper's results predict that giving an LLM-based planner access to a queryable environmental knowledge base will improve trajectory prediction at both short and long horizons, with the largest gains at longer horizons.
  • The chain-of-thought-to-SQL pipeline offers a reusable method for making LLM reasoning spatially and temporally grounded in structured databases.
  • The qualitative example indicates the model can use retrieved kinematic data about nearby vehicles to anticipate interactions and refine its predictions.
  • The authors conclude that this reduces dependence on rigid, pre-defined labels and supports more dynamic scene understanding.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the retrieval database contains ground-truth information beyond the ego vehicle's perception range, the 76.5% ADE reduction probably conflates the framework's contribution with the value of the additional information itself; a control that injects the same facts without the retrieval machinery would separate the two.
  • A natural testable extension is to apply SenseRAG on a dataset with realistic sensor noise and occlusions, comparing it against a baseline that receives the same out-of-perception truths through an oracle channel; the framework's genuine value would then show up as the difference between query-based retrieval and passive data injection.
  • Retrieval latency is acknowledged as a constraint, so the practical deployment of this design in real-time vehicles would require a faster query planner or precomputed retrieval indices beyond what the current GPT-4-based demonstration shows.
  • The same proactive RAG loop could be evaluated on plan validity and control metrics, not just displacement error, to test whether the perception improvement actually changes driving decisions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes SenseRAG, a retrieval-augmented generation framework for LLM-based autonomous driving. It constructs a knowledge database from multimodal sensor data and uses chain-of-thought prompting to generate natural-language queries that are translated into SQL to retrieve environmental information, which is then combined with self-perception data for trajectory prediction. The authors evaluate on the DLR Urban Traffic dataset, reporting that SenseRAG reduces ADE by 76.5% and FDE by 72.2% compared to a GPT-4 baseline without retrieval.

Significance. If the reported improvements were valid, the work would demonstrate a practical way to give LLMs real-time environmental context in autonomous driving. However, the evaluation design makes the central claim unsupported: the retrieval condition receives ground-truth kinematic states of surrounding vehicles that the baseline cannot observe, so the measured gains likely reflect information leakage rather than the proposed querying or knowledge-base components. The paper also reports no statistical variability and no ablations, making it impossible to attribute the outcomes to any specific design choice. The framework itself is sensible and the writing is generally clear, but the experimental evidence does not substantiate the claimed contributions.

major comments (3)
  1. [Section 4.1 and Section 4.2, Tables 1-2] The headline improvement is not a valid test of SenseRAG. Section 4.1 states that 'the retrieval database included information beyond the vehicle's perception range in the dataset,' and the example in Section 4.2 queries for the location, velocity, and acceleration of all other vehicles at the current timestamp from the same DLR UT dataset used for ADE/FDE evaluation. The baseline is restricted to a simulated 30 m perception range and does not have access to those states. Thus the comparison is between GPT-4 with and without the ground-truth current kinematics of all traffic participants, rather than with and without the proposed retrieval/querying mechanism. With perfect current states, even a constant-velocity extrapolation can explain much of the reported error reduction, so the 76.5% and 72.2% figures cannot be attributed to the chain-of-thought prompting, SQL translation, or knowledge-base construction.
  2. [Section 4.2, Tables 1 and 2] The results are reported for a single run per condition, with no error bars, no across-seed or across-prompt variation, and no statistical test. Since both conditions use the same GPT-4 model, the differences at timestamps 3, 5, and 10 could be affected by sampling variability, prompt phrasing, or decoding temperature. Without replication, the claimed consistent improvement is not established, and the paper overstates the quantitative findings.
  3. [Section 4.1 and Section 4.2] There is no ablated comparison that isolates the contributions of the framework's components. The enhanced condition differs from the baseline by at least three factors: (i) access to additional ground-truth state information, (ii) the RAG pipeline that converts queries to SQL and verbalizes results, and (iii) a different prompting structure with chain-of-thought instructions. Providing the same ground-truth states directly to the baseline, or degrading the retrieved states to reflect realistic V2X latency and noise, would be necessary to test whether the query mechanism adds value. As it stands, the experiment cannot support the causal claim that the proposed proactive RAG is responsible for the improvement.
minor comments (5)
  1. [Section 4.2] The word 'enchanced' should be 'enhanced'.
  2. [Section 4.2, Figure 3] The SQL query example is cut off and Figure 3 is not displayed; the figure reference is incomplete.
  3. [Abstract and Section 1] The paper describes the evaluation dataset as 'real-world V2X datasets,' but the DLR Urban Traffic dataset is an infrastructure-based intersection dataset, not a V2X communication dataset. This mischaracterization should be corrected.
  4. [Section 3.2.3, Eqs. (5)-(6)] Equations (5) and (6) are generic sequence-modeling objective functions and do not reflect any autonomous-driving-specific constraint; the connection to the proposed framework is not explained, and the notation for v and v' is inconsistent between Eqs. (3) and (4).
  5. [Section 3.3] No prompt templates or concrete examples of the chain-of-thought reasoning are given; the only example appears in Section 4.2, making the method difficult to reproduce.

Circularity Check

1 steps flagged · score 6.0 of 10

The 76.5%/72.2% error reduction is not attributable to SenseRAG: the retrieval condition receives ground-truth positions, velocities, and accelerations of all nearby vehicles from the same DLR UT evaluation dataset, while the baseline is deliberately denied them.

  1. other [Section 4.1 (Setup) and Section 4.2 (Results Evaluation), including the example query at timestamp 2023-09-24 00:01:17]
    "The experimental dataset is derived from the DLR Urban Traffic dataset (DLR UT)... It includes trajectory data of all participants at the intersection, along with ... positions of traffic participants, speed, acceleration... The retrieval database included information beyond the vehicle's perception range in the dataset... At timestamp 2023-09-24 00:01:17, provide the location, velocity, and acceleration of my car... In addition, provide the same information for other vehicles around my car... Compared to the baseline, our model reduced the ADE and FDE by 76.5% and 72.2%, respectively."

    The headline claim in Section 4.2 is that SenseRAG reduces ADE/FDE by 76.5%/72.2% relative to a GPT-4 baseline. Section 4.1 states that the two conditions differ by whether a retrieval database is used, but the database is built from the same DLR UT dataset used for evaluation and contains positions, speeds, and accelerations of all traffic participants. The example query retrieves exactly the current location, velocity, and acceleration of the ego vehicle and surrounding vehicles at the evaluation timestamp. The baseline is restricted to a simulated 30 m perception range and cannot observe these states. Therefore the enhanced model is given the ground-truth kinematic state of the scene as input, while the baseline is not.

full rationale

The paper's equations (Eq. 1 through Eq. 7) are descriptive and do not themselves smuggle in the result; the circularity is in the evaluation construction. The claimed contribution is validated only by the internal comparison in Section 4.2, and that comparison is confounded: the retrieval database is populated from the same DLR UT trajectories used for ADE/FDE computation, and it supplies the current ground-truth kinematic states of all vehicles, including those outside the baseline's simulated perception range. The baseline is deliberately denied this information, so the 76.5%/72.2% reduction is not a controlled test of the retrieval-augmented mechanism. There is no meaningful self-citation chain: references [19] and [26] are prior work by overlapping authors but are not load-bearing for the central empirical claim. The issue is therefore not definitional circularity in the formal derivation, but a form of evaluation circularity in which the 'prediction' improvement is largely determined by the experimental setup. This warrants a partial-circularity score of 6 rather than a higher score, because the framework could in principle be useful in a genuine V2X setting; the paper simply does not provide evidence that its specific contributions are responsible for the reported gains.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The framework rests on the assumption that LLM text reasoning over formatted traffic data can predict trajectories, that the logged data act as clean real-time V2X information, and that LLM-generated queries convert to SQL correctly; none of these are validated in the experiments. One hand-set parameter, the 30 m perception range, defines the baseline's blindness and is not tested for sensitivity.

free parameters (1)
  • Perception range threshold = 30 m
    Set by hand in Section 4.1 ('the perception range set to within 30 meters') to define what the baseline GPT-4 can see; no sensitivity analysis is provided, and it directly controls how much information the baseline lacks.
assumptions (3)
  • domain assumption LLM text reasoning over formatted traffic data is sufficient for trajectory prediction
    Section 4.1 feeds all environment information to GPT-4 as text and assumes it can interpret coordinates, velocities, and traffic states to predict trajectories.
  • domain assumption Retrieved database records represent accurate real-time ground truth at inference time
    Section 4.1 says the retrieval database 'included information beyond the vehicle's perception range in the dataset', i.e., ground-truth logged data are served to the model as if available through V2X; no latency, noise, or staleness is modeled.
  • domain assumption LLM-generated natural-language queries can be converted to SQL without errors
    Section 3.3.2 describes this transformation and gives one example, but the paper reports no accuracy evaluation of the generated SQL queries.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SenseRAG: Constructing Environmental Knowledge Bases with Proactive Querying for LLM-Based Autonomous Driving." pith.science (2026). https://pith.science/paper/CYBKYGAI

@misc{pith2026250103535,
  author       = {Pith},
  title        = {Pith review of: SenseRAG: Constructing Environmental Knowledge Bases with Proactive Querying for LLM-Based Autonomous Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CYBKYGAI}},
  note         = {Machine review of arXiv:2501.03535}
}
read the original abstract

This study addresses the critical need for enhanced situational awareness in autonomous driving (AD) by leveraging the contextual reasoning capabilities of large language models (LLMs). Unlike traditional perception systems that rely on rigid, label-based annotations, it integrates real-time, multimodal sensor data into a unified, LLMs-readable knowledge base, enabling LLMs to dynamically understand and respond to complex driving environments. To overcome the inherent latency and modality limitations of LLMs, a proactive Retrieval-Augmented Generation (RAG) is designed for AD, combined with a chain-of-thought prompting mechanism, ensuring rapid and context-rich understanding. Experimental results using real-world Vehicle-to-everything (V2X) datasets demonstrate significant improvements in perception and prediction performance, highlighting the potential of this framework to enhance safety, adaptability, and decision-making in next-generation AD systems.

Figures

Figures reproduced from arXiv: 2501.03535 by the authors.

Figure 1
Figure 1. Framework of Methodology reliable AD systems. 3. Methodology 3.1. Framework Overview To enhance the perception and situation awareness capa￾bilities of LLM-based AD systems, we propose a proactive SenseRAG framework, centered around a synthetic knowl￾edge database. It empowers AVs to interpret and adapt to dynamic scenarios by leveraging an accumulated repository of multimodal environmental data. Functioning as a cl… view at source ↗
Figure 2
Figure 2. Overview of the data integration pipeline [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Example of SQL Query Generation location, velocity, and acceleration of my car lo￾cated at (604739.287, 5792784.4887500005). In addition, provide the same information for other vehicles around my car. This query was seamlessly transformed into an SQL query for database retrieval 3: The retrieved data was formatted as follows: At timestamp 2023-09-24 00:01:17, a vehicle was located at (604750.30, 5792780.20) with a v… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 22 canonical work pages

  1. [1]

    An overview of sensors in autonomous vehicles.Procedia Com- puter Science, 198:736–741, 2022

    Henry Alexander Ignatious, Manzoor Khan, et al. An overview of sensors in autonomous vehicles.Procedia Com- puter Science, 198:736–741, 2022. 1

    work page 2022

  2. [2]

    Object detection in 20 years: A survey.Proceed- ings of the IEEE, 111(3):257–276, 2023

    Zhengxia Zou, Keyan Chen, Zhenwei Shi, Yuhong Guo, and Jieping Ye. Object detection in 20 years: A survey.Proceed- ings of the IEEE, 111(3):257–276, 2023. 1

    work page 2023

  3. [3]

    Review the state-of-the-art technologies of seman- tic segmentation based on deep learning

    Yujian Mo, Yan Wu, Xinneng Yang, Feilin Liu, and Yujun Liao. Review the state-of-the-art technologies of seman- tic segmentation based on deep learning. Neurocomputing, 493:626–646, 2022. 1

    work page 2022

  4. [4]

    Deep learning for computer vision: A brief review

    Athanasios V oulodimos, Nikolaos Doulamis, Anastasios Doulamis, and Eftychios Protopapadakis. Deep learning for computer vision: A brief review. Computational intelligence and neuroscience, 2018(1):7068349, 2018. 1

    work page 2018

  5. [5]

    A survey on multimodal large language models for autonomous driving

    Can Cui et al. A survey on multimodal large language models for autonomous driving. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 958–979, 2024. 1, 2

    work page 2024

  6. [6]

    Dilu: A knowledge-driven approach to au- tonomous driving with large language models

    Licheng Wen, Daocheng Fu, Xin Li, Xinyu Cai, Tao Ma, Pinlong Cai, Min Dou, Botian Shi, Liang He, and Yu Qiao. Dilu: A knowledge-driven approach to au- tonomous driving with large language models. arXiv preprint arXiv:2309.16292, 2023. 1

    arXiv 2023

  7. [7]

    Language models are few-shot learners

    Tom B Brown. Language models are few-shot learners. arXiv preprint arXiv:2005.14165, 2020. 1

    arXiv 2005

  8. [8]

    Deep learning in computer vision: A critical review of emerg- ing techniques and application scenarios

    Junyi Chai, Hao Zeng, Anming Li, and Eric WT Ngai. Deep learning in computer vision: A critical review of emerg- ing techniques and application scenarios. Machine Learning with Applications, 6:100134, 2021. 1

    work page 2021

Show all 32 references
  1. [9]

    Vehicle- to-everything (v2x) in the autonomous vehicles domain–a technical review of communication, sensor, and ai technolo- gies for road user safety

    Syed Adnan Yusuf, Arshad Khan, and Riad Souissi. Vehicle- to-everything (v2x) in the autonomous vehicles domain–a technical review of communication, sensor, and ai technolo- gies for road user safety. Transportation Research Interdis- ciplinary Perspectives, 23:100980, 2024. 1

    2024

  2. [10]

    Large language models: a comprehensive survey of its applications, challenges, limitations, and future prospects

    Muhammad Usman Hadi, Qasem Al Tashi, Abbas Shah, Rizwan Qureshi, Amgad Muneer, Muhammad Irfan, Anas Zafar, Muhammad Bilal Shaikh, Naveed Akhtar, Jia Wu, et al. Large language models: a comprehensive survey of its applications, challenges, limitations, and future prospects. Aut...

    2024

  3. [11]

    To- wards optimizing the costs of llm usage

    Shivanshu Shekhar, Tanishq Dubey, Koyel Mukherjee, Apoorv Saxena, Atharv Tyagi, and Nishanth Kotla. To- wards optimizing the costs of llm usage. arXiv preprint arXiv:2402.01742, 2024. 1

    2024 arXiv

  4. [12]

    Collabora- tive perception for autonomous driving: Current status and future trend

    Shunli Ren, Siheng Chen, and Wenjun Zhang. Collabora- tive perception for autonomous driving: Current status and future trend. In Proceedings of 2021 5th Chinese Confer- ence on Swarm Intelligence and Cooperative Control, pages 682–692. Springer, 2022. 2

    2021

  5. [13]

    Collaborative perception in autonomous driv- ing: Methods, datasets, and challenges

    Yushan Han, Hui Zhang, Huifang Li, Yi Jin, Congyan Lang, and Yidong Li. Collaborative perception in autonomous driv- ing: Methods, datasets, and challenges. IEEE Intelligent Transportation Systems Magazine, 2023. 2

    2023

  6. [14]

    How does c-v2x help autonomous driving to avoid acci- dents? Sensors, 22(2):686, 2022

    Lili Miao, Shang-Fu Chen, Yu-Ling Hsu, and Kai-Lung Hua. How does c-v2x help autonomous driving to avoid acci- dents? Sensors, 22(2):686, 2022. 2

    2022

  7. [15]

    Au- tonomous driving under v2x environment: state-of-the-art survey and challenges

    Xun Yang, Yunyang Shi, Jiping Xing, and Zhiyuan Liu. Au- tonomous driving under v2x environment: state-of-the-art survey and challenges. Intelligent Transportation Infrastruc- ture, 1:liac020, 2022. 2

    2022

  8. [16]

    Towards vehicle-to-everything autonomous driving: A survey on collaborative perception.arXiv preprint arXiv:2308.16714, 2023

    Si Liu, Chen Gao, Yuan Chen, Xingyu Peng, Xianghao Kong, Kun Wang, Runsheng Xu, Wentao Jiang, Hao Xiang, Jiaqi Ma, et al. Towards vehicle-to-everything autonomous driving: A survey on collaborative perception.arXiv preprint arXiv:2308.16714, 2023. 2

    2023 arXiv

  9. [17]

    Set- membership estimation in shared situational awareness for automated vehicles in occluded scenarios

    Vandana Narri, Amr Alanwar, Jonas M ˚artensson, Christof- fer Nor´en, Laura Dal Col, and Karl Henrik Johansson. Set- membership estimation in shared situational awareness for automated vehicles in occluded scenarios. In 2021 IEEE Intelligent Vehicles Symposium (IV), pages 385–...

    2021

  10. [18]

    Overcoming occlusions: Per- ception task-oriented information sharing in connected and autonomous vehicles

    Zhu Xiao, Jinmei Shu, Hongbo Jiang, Geyong Min, Hongyang Chen, and Zhu Han. Overcoming occlusions: Per- ception task-oriented information sharing in connected and autonomous vehicles. IEEE Network, 37(4):224–229, 2023. 2 7

    2023

  11. [19]

    Cross-domain transfer learning using attention latent features for multi-agent trajectory prediction, 2024

    Jia Quan Loh, Xuewen Luo, Fan Ding, Hwa Hui Tew, Junn Yong Loo, Ze Yang Ding, Susilawati Susilawati, and Chee Pin Tan. Cross-domain transfer learning using attention latent features for multi-agent trajectory prediction, 2024. 2

    2024

  12. [20]

    Toward ensuring safety for autonomous driving perception: standardization progress, research advances, and perspectives

    Chen Sun, Ruihe Zhang, Yukun Lu, Yaodong Cui, Zejian Deng, Dongpu Cao, and Amir Khajepour. Toward ensuring safety for autonomous driving perception: standardization progress, research advances, and perspectives. IEEE Trans- actions on Intelligent Transportation Systems, 2023. 2

    2023

  13. [21]

    Towards communication-efficient collaborative per- ception: Harnessing channel-spatial attention and knowledge distillation

    Fei Wang, Penglin Dai, Chuzhao Li, Zhangjie Meng, and Kai Liu. Towards communication-efficient collaborative per- ception: Harnessing channel-spatial attention and knowledge distillation. In Zhipeng Cai, Daniel Takabi, Shaoyong Guo, and Yifei Zou, editors, Wireless Artificial ...

  14. [22]

    Communication-efficient collaborative percep- tion via information filling with codebook

    Yue Hu, Juntong Peng, Sifei Liu, Junhao Ge, Si Liu, and Si- heng Chen. Communication-efficient collaborative percep- tion via information filling with codebook. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 15481–15490, 2024. 2

    2024

  15. [23]

    Will large language models be a panacea to au- tonomous driving? arXiv preprint arXiv:2409.14165, 2024

    Yuxuan Zhu, Shiyi Wang, Wenqing Zhong, Nianchen Shen, Yunqi Li, Siqi Wang, Zhiheng Li, Cathy Wu, Zhengbing He, and Li Li. Will large language models be a panacea to au- tonomous driving? arXiv preprint arXiv:2409.14165, 2024. 2

    2024 arXiv

  16. [24]

    Llm4drive: A survey of large language models for au- tonomous driving

    Zhenjie Yang, Xiaosong Jia, Hongyang Li, and Junchi Yan. Llm4drive: A survey of large language models for au- tonomous driving. In NeurIPS 2024 Workshop on Open- World Agents, 2023. 2

    2024

  17. [25]

    Drivemllm: A benchmark for spatial understanding with multimodal large language models in autonomous driving

    Xianda Guo, Ruijun Zhang, Yiqun Duan, Yuhang He, Chen- ming Zhang, Shuai Liu, and Long Chen. Drivemllm: A benchmark for spatial understanding with multimodal large language models in autonomous driving. arXiv preprint arXiv:2411.13112, 2024. 2

    2024 arXiv

  18. [26]

    Pkrd-cot: A unified chain-of-thought prompting for multi-modal large language models in au- tonomous driving

    Xuewen Luo, Fan Ding, Yinsheng Song, Xiaofeng Zhang, and Junnyong Loo. Pkrd-cot: A unified chain-of-thought prompting for multi-modal large language models in au- tonomous driving. arXiv preprint arXiv:2412.02025, 2024. 2

    2024 arXiv

  19. [27]

    Towards knowledge-driven autonomous driving

    Xin Li, Yeqi Bai, Pinlong Cai, Licheng Wen, Daocheng Fu, Bo Zhang, Xuemeng Yang, Xinyu Cai, Tao Ma, Jianfei Guo, et al. Towards knowledge-driven autonomous driving. arXiv preprint arXiv:2312.04316, 2023. 2

    2023 arXiv

  20. [28]

    A survey on rag meeting llms: Towards retrieval-augmented large language models

    Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. A survey on rag meeting llms: Towards retrieval-augmented large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages...

    2024

  21. [29]

    Rag-driver: Gen- eralisable driving explanations with retrieval-augmented in- context learning in multi-modal large language model.arXiv preprint arXiv:2402.10828, 2024

    Jianhao Yuan, Shuyang Sun, Daniel Omeiza, Bo Zhao, Paul Newman, Lars Kunze, and Matthew Gadd. Rag-driver: Gen- eralisable driving explanations with retrieval-augmented in- context learning in multi-modal large language model.arXiv preprint arXiv:2402.10828, 2024. 2

    2024

  22. [30]

    Rag-guided large language models for visual spatial description with adaptive hallucina- tion corrector

    Jun Yu, Yunxiang Zhang, Zerui Zhang, Zhao Yang, Gong- peng Zhao, Fengzhao Sun, Fanrui Zhang, Qingsong Liu, Jianqing Sun, Jiaen Liang, et al. Rag-guided large language models for visual spatial description with adaptive hallucina- tion corrector. InProceedings of the 32nd ACM I...

    2024

  23. [31]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024. 4 8

    2024

  24. [2025]

    Springer Nature Switzerland. 2

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.