DriveCode: Domain Specific Numerical Encoding for LLM-Based Autonomous Driving

Bo Zhang; Fang Zhang; Jianqiang Wang; Rui Zhou; Yanbo Jiang; Yaqin Zhang; Zhenhua Xu; Zhiye Wang

REVIEW 4 major objections 5 minor 47 references

DriveCode: Domain Specific Numerical Encoding for LLM-Based Autonomous Driving

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DriveCode claims that representing numbers as continuous embeddings rather than text tokens makes LLM-based autonomous driving predict speed, steering, and trajectories more precisely.

desk verdict DriveCode is a sensible xVal-style extension for LLM driving that delivers small, mixed gains; the architecture is coherent and the ablations are well designed, but the results need statistical grounding and the preprocessing pipeline needs to be specified before the claims can be fully trusted. read the letter →

arxiv 2603.00919 v3 pith:FYSOTYOS submitted 2026-03-01 cs.CV cs.RO

Zhiye Wang , Yanbo Jiang , Rui Zhou , Bo Zhang , Fang Zhang , Zhenhua Xu , Yaqin Zhang , Jianqiang Wang This is my paper

classification cs.CVcs.RO

keywords DriveCodenumericalencodingcontinuousnumberembeddingsLLMautonomousdrivingtrajectorypredictioncontrolsignalgenerationprojectorhead

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DriveCode argues that the reason LLM-based self-driving models make imprecise control predictions is that numbers are tokenized as text, which destroys their magnitude and forces digit-by-digit generation. The paper proposes a number projector that maps each numerical value into the model's embedding space, and a number head that regresses output numbers in a single step. On three autonomous-driving datasets, this continuous representation reduces trajectory, speed, and steering-angle errors compared with text-token and scaled-embedding baselines. If correct, it gives a way to make LLM-generated driving commands numerically reliable without changing the language model itself.

What carries the argument

The load-bearing mechanism is the paired number projector and number head. The projector maps a scalar x to a high-dimensional embedding through a two-layer MLP with GELU, inserted where a <number> placeholder sits in the token sequence; the number head regresses a scalar from the hidden state one position before each placeholder. Together they let the LLM attend over numeric values alongside image and text features, and generate numbers continuously rather than token by token. The authors also use an ordered number list extracted from each dialogue to keep every placeholder aligned with its real value.

What would settle it

Run the same training recipe on a dialogue where numbers are written in varied formats (e.g., '1.5m', 'one point five', '≈2') and compare alignment errors; or construct a dataset where the regex misses a number and show the model silently ignores it. A direct test: take a trained DriveCode model and feed a prompt whose placeholder order has been shuffled relative to the number list—if performance collapses, the method depends on exact alignment rather than on continuous number semantics.

Watch

Extended reading notes

Core claim

The central claim is that treating numbers as a dedicated continuous modality—rather than as discrete vocabulary tokens—improves both the perception and the generation of physical quantities in LLM-based driving. Input numbers are extracted from prompts, replaced by placeholders, and injected as learned embeddings via a two-layer projector; output numbers are predicted by regressing from the hidden state immediately before each placeholder, so the model emits a value in one decoding step instead of one token per digit. The paper reports that this scheme outperforms text-token and scaled-embedding baselines on speed, steering angle, point, and trajectory metrics across three driving datasets,

Load-bearing premise

DriveCode assumes that a simple regex can find every meaningful number in every prompt and that the extracted list stays in exactly the same order as the placeholders in the token sequence; if either fails, the number supervision is corrupted.

Editorial extensions

If this is right

If numbers are processed continuously, LLM driving models can produce speed, waypoint, and steering commands with lower mean error than text-based number generation, on the three tested datasets.
Because a number is produced in a single decoding step, inference latency drops slightly, which matters for real-time control.
The method is architecture-agnostic: it only replaces numeric tokens, so it can be grafted onto existing vision-language driving models without retraining the vision or language backbone from scratch.
The output-side regression head alone (without the input projector) already reduces numeric error, suggesting that direct numeric supervision is the main driver; the input side adds further gains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A likely consequence beyond driving: any LLM task that needs exact scalar outputs—robotics control, sensor calibration, financial quantities—could use the same projector/head pattern.
The reliance on regex extraction implies the method's benefit shrinks on free-form text where numbers aren't in the expected format; a learned number span detector would be a natural extension.
The paper's speed advantage is modest; if bigger models with more tokens per number shrink the gap, the main value may be precision rather than latency.
Testable extension: use the number projector in a math word-problem setting with unseen number scales to see if continuous encoding improves generalization beyond the driving domain.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

What you should know: this paper is a direct extension of xVal, but it adds two things xVal doesn't have — an input number projector that maps scalar numbers into the LLM embedding space, and a parallel output number head that regresses numerical tokens instead of decoding digits. That combination, applied to autonomous driving, is a genuine architectural contribution. The comparisons against text and xVal baselines, plus the Variant ablation that isolates input vs. output numeric conditioning, are the right way to test the idea.

The paper is mostly sound. The architecture is coherent, the training objective that combines LM loss with a scalar or trajectory regression loss is plausible, and the efficiency argument is correct in principle: one regression step replaces several autoregressive tokens. The OmniDrive trajectory result (3.08 to 2.83 L2) is the most convincing single number. The limitations section is honest, which is more than many papers offer. The authors clearly know the weak spots.

The soft spots are real. There are no error bars, confidence intervals, or multiple seeds anywhere. Several gains are tiny — Table II shows differences of 0.0003 and 0.0003 in theta and point error on DriveGPT4-V2, which could easily be noise. The per-metric results are mixed: DriveGPT4 beats DriveCode at the tightest A0.1 threshold on both speed and angle, and the Variant (output-only numeric regression) actually beats DriveCode on theta in Table V. The abstract's "superior performance" overstates what the tables show; the honest summary is "better on most, not on all, and sometimes by a hair."

The stress-test worry about the regex-based number extraction pipeline is legitimate but, I think, not load-bearing. Yes, the pipeline is under-specified — no regex patterns, no error rate, no manual verification — and the paper's own limitations section admits "mismatches, missing numbers, or inconsistent formats" are possible. That is a genuine reproducibility problem and a robustness concern. But the architecture's value does not collapse if extraction is perfect; the ablations comparing text vs. numeric input and output are independent of the extraction pipeline's accuracy, and those ablations show consistent, if small, improvements. I would not call the pipeline the "central claim." The central claim is that continuous encoding helps, and that claim is supported, weakly and needing more seeds, but supported.

Who this is for: someone working on numeric encodings for LLMs, or on LLM-based driving control, will want to read this. It is not a breakthrough paper, but it is a useful, concrete step. The lack of code is the biggest practical obstacle to citing it seriously.

Recommendation: send it to peer review. It deserves a real referee, but the reviewers should demand error bars, multiple seeds, a full specification (or release) of the preprocessing pipeline, and a toned-down abstract. As is, I would not cite it in the next year without code.

Referee Report

4 major / 5 minor

Summary. DriveCode proposes a continuous numerical encoding for LLM-based autonomous driving. In the preprocessing stage, numerical quantities in prompts and targets are replaced by a special <number_token> placeholder, with the actual values stored in an ordered list. A learned number projector maps each value into the LLM embedding space, and a number head regresses numerical outputs from hidden states, so that numbers are processed as continuous quantities rather than discrete text tokens. The model is evaluated on three datasets (OmniDrive, DriveGPT4, DriveGPT4-V2) against text-token and xVal baselines, reporting lower RMSE and point/speed errors in most settings, with ablations separating input- and output-side numeric conditioning. The central claim is that this dedicated numeric modality improves trajectory prediction and control-signal generation.

Significance. The idea of treating numbers as a dedicated modality in driving LLMs is timely and practically relevant for control-signal regression, where small absolute errors matter. The architecture is clearly specified, and the ablation design (Text / Variant / DriveCode) is a sensible way to isolate the contributions of numeric conditioning on the input and regression supervision on the output. The paper also provides an efficiency analysis and explicitly acknowledges limitations. If the empirical claims held up with adequate statistical support and a validated preprocessing pipeline, the method would be a credible component for real-time LLM-based driving systems. However, the current evidence is insufficient: results are single-run point estimates without confidence intervals, the preprocessing heuristic is unquantified, and the training loss directly matches the evaluation metric, which partially explains the reported improvements.

major comments (4)

[III-B, IV-G] The preprocessing pipeline is load-bearing but unvalidated. The regex-based selection of 'meaningful' numbers is not specified (no patterns or decision rule for excluding terms like 'number of camera views'), and no precision/recall statistics are reported. Since Eqs. (3)-(8) use the ordered number list for both input embeddings and regression targets, a mis-extraction or order misalignment injects corrupted supervision. The Limitations (IV-G) concede this dependency but do not quantify its effect. Please report extraction accuracy on a random sample, provide the exact patterns/rules, and analyze failure cases, especially on natural-language DriveGPT4 versus templated DriveGPT4-V2.
All results are single-run point estimates without error bars, confidence intervals, or seed variability. The claimed improvements are often small (e.g., speed RMSE 1.08 vs 1.13; DriveGPT4-V2 point error 0.01137 vs 0.01166; latency 3.1798 vs 3.1920 s), and Table I shows DriveCode losing to DriveGPT4 at the tight threshold A0.1 for both speed (27.50 vs 30.09) and angle (57.18 vs 59.23). Without paired significance tests over multiple seeds, the headline 'superior performance' is not supported.
[III-E, IV-C] The numerical loss L_num (Eqs. 7-8) is, up to sign, the same as the reported metrics (L1 error, L2 error, heading error). Thus DriveCode directly optimizes the evaluation metric, whereas text-token baselines are trained only with cross-entropy (Eq. 5). Part of the observed gain is therefore a consequence of the supervision signal, not of the continuous encoding per se. Please add a baseline that applies the same regression head (or an auxiliary L1/L2 loss) to hidden states derived from text numbers, and/or discuss this confound explicitly.
[IV.E, Table V] Table V shows that 'Variant' achieves a lower theta error (0.07078 degrees) than DriveCode (0.07377 degrees) on DriveGPT4-V2, while DriveCode wins on point and speed errors. The text says DriveCode 'yields the best point and speed accuracy' but omits the theta result. The abstract and conclusion claim 'superior performance in trajectory prediction and control signal generation'; this is not consistent with Table V. The claims should be limited to the metrics on which the method actually wins, or the discrepancy explained (e.g., variance).

minor comments (5)

[Fig. 4] Figure 4 appears to show 'number: [...]' appended to the answer; please clarify whether this list is part of the input/target text or only an illustration of the stored aligned values. If it is part of the target, the LM could copy numbers from the list, circumventing the number head.
[III-D] Specify whether training uses teacher forcing with ground-truth numbers for the next-step embedding or the predicted number from the number head; the current text describes inference only.
[Eq. (15)] Define the relation between the normalized L2 norm in Eq. (15) and the RMSE/MAE used in the tables; the notation is ambiguous.
[III-D] No normalization details for input numbers; the Limitations mention sensitivity to scale/outliers, but the training procedure does not state how x_k is normalized (e.g., min-max or z-score).
[Throughout] Minor typos and formatting issues: 'LLaV A' should be 'LLaVA', 'V arious' etc. Please proofread.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: DriveCode's claims are empirical comparisons on independent test splits; the number projector/head are new components trained with standard regression objectives, and no predicted quantity is an input renamed as a prediction.

full rationale

The paper's derivation chain is architectural and empirical rather than deductive: Eq. (3) defines a number projector, Eq. (4) inserts numeric embeddings into the token sequence, Eqs. (6)-(8) define regression heads and losses, and Eqs. (10)-(14) define evaluation metrics. None of these equations is equivalent to another by construction. The fact that L_scalar (Eq. 7) and L_traj (Eq. 8) coincide with the reported speed error (Eq. 14) and trajectory L2 error (Eq. 10) is ordinary supervised training on held-out splits, not a fitted parameter being renamed as a prediction; the baselines are evaluated on the same splits. The Section III-B regex extraction pipeline is heuristic, and Section IV-G concedes that 'mismatches, missing numbers, or inconsistent formats can introduce noise,' but this is a label-construction robustness concern, not a circular step: the extracted number list is an input for training, not the model's predicted output. The self-citations to DriveGPT4 [28] and DriveGPT4-V2 [5] supply datasets and one baseline, while ADAPT [47] and xVal [42] provide independent comparisons, so the self-citations are not load-bearing. There is no uniqueness theorem, no imported ansatz, and no renaming of a known result. Overall, the central empirical claim is self-contained against test data; the reliability of the preprocessing pipeline is a correctness risk but not a circularity.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claim is empirical rather than derivational, so the key burden is on data preprocessing quality, the learnability of the number head from a single hidden state, and the transfer of open-loop metrics to driving. The only hand-set hyperparameter identified is lambda, set to 1 without tuning. Model weights are not counted as free parameters because they are standard learned parameters.

free parameters (1)

lambda (numerical loss weight) = 1.0
Set to 1 in all experiments (Section III-E). Chosen by hand to balance L_text and L_num; no tuning or sensitivity analysis is reported, yet it controls the training objective.

assumptions (3)

domain assumption Regular expression extraction identifies every physically meaningful number and the ordered number list aligns perfectly with placeholder positions.
Assumed by the preprocessing pipeline in Section III-B; acknowledged as fragile in Section IV-G. Any extraction or alignment error corrupts the number projector's supervision.
domain assumption The hidden state at position i_m-1 before a <number_token> carries sufficient information for a lightweight MLP number head to regress the target number.
Core design of the number head in Section III-E (Eq. 6). There is no theoretical analysis; it relies on the LM's learned representation being rich enough at that exact position.
domain assumption Offline open-loop benchmark metrics are a valid proxy for the method's usefulness in autonomous driving.
All experiments are open-loop; the authors themselves list closed-loop evaluation as future work in Section V.

how reviews work

0 comments

Cite this review

Pith. "Pith review of DriveCode: Domain Specific Numerical Encoding for LLM-Based Autonomous Driving." pith.science (2026). https://pith.science/paper/FYSOTYOS

@misc{pith2026260300919,
  author       = {Pith},
  title        = {Pith review of: DriveCode: Domain Specific Numerical Encoding for LLM-Based Autonomous Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FYSOTYOS}},
  note         = {Machine review of arXiv:2603.00919}
}

read the original abstract

Large language models (LLMs) have shown great promise for autonomous driving. However, discretizing numbers into tokens limits precise numerical reasoning, fails to reflect the positional significance of digits in the training objective, and makes it difficult to achieve both decoding efficiency and numerical precision. These limitations affect both the processing of sensor measurements and the generation of precise control commands, creating a fundamental barrier for deploying LLM-based autonomous driving systems. In this paper, we introduce DriveCode, a novel numerical encoding method that represents numbers as dedicated embeddings rather than discrete text tokens. DriveCode employs a number projector to map numbers into the language model's hidden space, enabling seamless integration with visual and textual features in a unified multimodal sequence. Evaluated on OmniDrive, DriveGPT4, and DriveGPT4-V2 datasets, DriveCode demonstrates superior performance in trajectory prediction and control signal generation, confirming its effectiveness for LLM-based autonomous driving systems.

Figures

Figures reproduced from arXiv: 2603.00919 by the authors.

**Figure 2.** DriveCode overview. Our proposed approach consists of three parts: image projection, text tokenization and number [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Parallel autoregressive generation of text and num [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Examples of three datasets. All numbers in these datasets are replaced with [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

47 extracted references · 6 linked inside Pith

[1]

The role of the hercules autonomous vehicle during the covid-19 pandemic: An autonomous logistic vehicle for contactless goods transportation,

T. Liu, Q. hai Liao, L. Gan, F. Ma, J. Cheng, X. Xie, Z. Wang, Y . Chen, Y . Zhu, S. Zhanget al., “The role of the hercules autonomous vehicle during the covid-19 pandemic: An autonomous logistic vehicle for contactless goods transportation,”IEEE Robotics & Automation Magazine, 2021

2021
[2]

Hidden biases of end-to-end driving models,

B. Jaeger, K. Chitta, and A. Geiger, “Hidden biases of end-to-end driving models,” inICCV, 2023

2023
[3]

Vad: Vectorized scene representation for efficient autonomous driving,

B. Jiang, S. Chen, Q. Xu, B. Liao, J. Chen, H. Zhou, Q. Zhang, W. Liu, C. Huang, and X. Wang, “Vad: Vectorized scene representation for efficient autonomous driving,” inICCV, 2023

2023
[4]

Planning-oriented autonomous driving,

Y . Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wanget al., “Planning-oriented autonomous driving,” in CVPR, 2023

2023
[5]

Drivegpt4-v2: Harnessing large language model capabilities for enhanced closed-loop autonomous driving,

Z. Xu, Y . Bai, Y . Zhang, Z. Li, F. Xia, K.-Y . K. Wong, J. Wang, and H. Zhao, “Drivegpt4-v2: Harnessing large language model capabilities for enhanced closed-loop autonomous driving,” inCVPR, 2025

2025
[6]

Tinyllama: An open-source small language model,

P. Zhang, G. Zeng, T. Wang, and W. Lu, “Tinyllama: An open-source small language model,”arXiv:2401.02385, 2024

arXiv 2024
[7]

Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,

W.-L. Chiang, Z. Li, Z. Lin, Y . Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y . Zhuang, J. E. Gonzalezet al., “Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,” https://vicuna. lmsys.org, 2023

2023
[8]

Visual instruction tuning,

H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual instruction tuning,” in NeurIPS, 2023

2023

Show all 47 references

[9]

Qwen2. 5-vl technical report,

S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tanget al., “Qwen2. 5-vl technical report,”arXiv preprint arXiv:2502.13923, 2025

2025 arXiv
[10]

Video- llava: Learning united visual representation by alignment before pro- jection,

B. Lin, Y . Ye, B. Zhu, J. Cui, M. Ning, P. Jin, and L. Yuan, “Video- llava: Learning united visual representation by alignment before pro- jection,” inEMNLP, 2024

2024
[11]

Llava- next: Improved reasoning, ocr, and world knowledge,

H. Liu, C. Li, Y . Li, B. Li, Y . Zhang, S. Shen, and Y . J. Lee, “Llava- next: Improved reasoning, ocr, and world knowledge,” https://llava-vl. github.io/blog/2024-01-30-llava-next, 2024

2024
[12]

Language models do not embed numbers continuously,

A. O. Davies, R. Nzoyem, N. Ajmeriet al., “Language models do not embed numbers continuously,”arXiv preprint arXiv:2510.08009, 2025

2025
[13]

Number representations in llms: A computational parallel to human perception,

H. AlquBoj, H. AlQuabeh, V . Bojkovic, T. Hiraoka, A. O. El- Shangiti, M. Nwadike, and K. Inui, “Number representations in llms: A computational parallel to human perception,”CoRR, 2025

2025
[14]

Glm-4.5: Agentic, reasoning, and coding (arc) foundation models,

A. Zeng, X. Lv, Q. Zheng, Z. Hou, B. Chen, C. Xie, C. Wang, D. Yin, H. Zeng, J. Zhanget al., “Glm-4.5: Agentic, reasoning, and coding (arc) foundation models,”arXiv preprint arXiv:2508.06471, 2025. TABLE I: Quantitative results of control signals prediction on the whole DriveG...

2025 arXiv
[15]

Llama- 3.1-foundationai-securityllm-base-8b technical report,

P. Kassianik, B. Saglam, A. Chen, B. Nelson, A. Vellore, M. Aufiero, F. Burch, D. Kedia, A. Zohary, S. Weerawardhenaet al., “Llama- 3.1-foundationai-securityllm-base-8b technical report,”arXiv preprint arXiv:2504.21039, 2025

2025 arXiv
[16]

Benchmarking the performance of large language models in uveitis: a comparative analysis of chatgpt- 3.5, chatgpt-4.0, google gemini, and anthropic claude3,

F.-F. Zhao, H.-J. He, J.-J. Liang, J. Cen, Y . Wang, H. Lin, F. Chen, T.-P. Li, J.-F. Yang, L. Chenet al., “Benchmarking the performance of large language models in uveitis: a comparative analysis of chatgpt- 3.5, chatgpt-4.0, google gemini, and anthropic claude3,”Eye, 2025

2025
[17]

Llava-mini: Efficient image and video large multimodal models with one vision token,

S. Zhang, Q. Fang, Z. Yang, and Y . Feng, “Llava-mini: Efficient image and video large multimodal models with one vision token,” inICLR, 2025

2025
[18]

Llava-more: A comparative study of llms and visual backbones for enhanced visual instruction tuning,

F. Cocchi, N. Moratelli, D. Caffagni, S. Sarto, L. Baraldi, M. Cornia, and R. Cucchiara, “Llava-more: A comparative study of llms and visual backbones for enhanced visual instruction tuning,”CoRR, 2025

2025
[19]

Omni-r1: Do you really need audio to fine-tune your audio llm?

A. Rouditchenko, S. Bhati, E. Araujo, S. Thomas, H. Kuehne, R. Feris, and J. Glass, “Omni-r1: Do you really need audio to fine-tune your audio llm?”arXiv preprint arXiv:2505.09439, 2025

2025
[20]

Exploring finetuned audio-llm on heart murmur features,

A. Florea, X. Jiang, N. Mesgarani, and X. Jiang, “Exploring finetuned audio-llm on heart murmur features,”Smart Health, 2025

2025
[21]

Surveying the mllm landscape: A meta- review of current surveys,

M. Li, K. Chen, Z. Bi, M. Liu, B. Peng, Q. Niu, J. Liu, J. Wang, S. Zhang, X. Panet al., “Surveying the mllm landscape: A meta- review of current surveys,”arXiv preprint arXiv:2409.18991, 2024

2024
[22]

Vtimellm: Empower llm to grasp video moments,

B. Huang, X. Wang, H. Chen, Z. Song, and W. Zhu, “Vtimellm: Empower llm to grasp video moments,” inCVPR, 2024

2024
[23]

An image is worth 16x16 words: Transformers for image recognition at scale,

A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gellyet al., “An image is worth 16x16 words: Transformers for image recognition at scale,” inICLR, 2021

2021
[24]

Attention is all you need,

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in NeurIPS,, 2017

2017
[25]

Blip: Bootstrapping language- image pre-training for unified vision-language understanding and generation,

J. Li, D. Li, C. Xiong, and S. Hoi, “Blip: Bootstrapping language- image pre-training for unified vision-language understanding and generation,” inICML, 2022

2022
[26]

Vadv2: End-to-end vectorized autonomous driving via probabilistic planning,

S. Chen, B. Jiang, H. Gao, B. Liao, Q. Xu, Q. Zhang, C. Huang, W. Liu, and X. Wang, “Vadv2: End-to-end vectorized autonomous driving via probabilistic planning,” inICLR, 2026

2026
[27]

Lmdrive: Closed-loop end-to-end driving with large language models,

H. Shao, Y . Hu, L. Wang, G. Song, S. L. Waslander, Y . Liu, and H. Li, “Lmdrive: Closed-loop end-to-end driving with large language models,” inCVPR, 2024

2024
[28]

Drivegpt4: Interpretable end-to-end autonomous driving via large language model,

Z. Xu, Y . Zhang, E. Xie, Z. Zhao, Y . Guo, K.-Y . K. Wong, Z. Li, and H. Zhao, “Drivegpt4: Interpretable end-to-end autonomous driving via large language model,”RA-L, 2024. TABLE IV: Comparison of Variant and DriveCode on control signals prediction (DriveGPT4). Method Theta E...

2024
[29]

Insmapper: Exploring inner- instance information for vectorized hd mapping,

Z. Xu, K.-Y . K. Wong, and H. Zhao, “Insmapper: Exploring inner- instance information for vectorized hd mapping,” inECCV, 2024

2024
[30]

Invdriver: Intra-instance aware vectorized query-based autonomous driving trans- former,

B. Zhang, H. Huang, C. Liu, Y . Zhang, and Z. Xu, “Invdriver: Intra-instance aware vectorized query-based autonomous driving trans- former,”Journal of Intelligent and Connected Vehicles, 2025

2025
[31]

Recent advancements in end-to-end au- tonomous driving using deep learning: A survey,

P. S. Chib and P. Singh, “Recent advancements in end-to-end au- tonomous driving using deep learning: A survey,”IEEE Transactions on Intelligent Vehicles, 2023

2023
[32]

Lc-llm: Explainable lane-change intention and trajectory predictions with large language models,

M. Peng, X. Guo, X. Chen, K. Chen, M. Zhu, L. Chen, and F.-Y . Wang, “Lc-llm: Explainable lane-change intention and trajectory predictions with large language models,”Communications in Transportation Re- search, 2025

2025
[33]

sam-llm: interpretable lane change trajectoryprediction via parametric finetuning,

Z. Cao, Y . Shi, and M. Xu, “sam-llm: interpretable lane change trajectoryprediction via parametric finetuning,”arXiv preprint arXiv:2509.03462, 2025

2025 arXiv
[34]

Autovla: A vision-language-action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine- tuning,

Z. Zhou, T. Cai, S. Z. Zhao, Y . Zhang, Z. Huang, B. Zhou, and J. Ma, “Autovla: A vision-language-action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine- tuning,”NeurIPS, 2025

2025
[35]

Applications of large language models and multimodal large models in autonomous driving: A comprehensive review,

J. Li, J. Li, G. Yang, L. Yang, H. Chi, and L. Yang, “Applications of large language models and multimodal large models in autonomous driving: A comprehensive review,”Drones, 2025

2025
[36]

A survey on large language model-powered autonomous driving,

Y . Zhu, S. Wang, W. Zhong, N. Shen, Y . Li, S. Wang, Z. Li, C. Wu, Z. He, and L. Li, “A survey on large language model-powered autonomous driving,”Engineering, 2025

2025
[37]

Hallucination reduction and optimization for large language model-based autonomous driving,

J. Wang, “Hallucination reduction and optimization for large language model-based autonomous driving,”Symmetry, 2024

2024
[38]

Teaching arithmetic to small transformers,

N. Lee, K. Sreenivasan, J. D. Lee, K. Lee, and D. Papailiopoulos, “Teaching arithmetic to small transformers,” inICLR, 2024

2024
[39]

Positional description matters for transformers arithmetic,

R. Shen, S. Bubeck, R. Eldan, Y . T. Lee, Y . Li, and Y . Zhang, “Positional description matters for transformers arithmetic,”arXiv preprint arXiv:2311.14737, 2023

2023 arXiv
[40]

Numerologic: Number encoding for enhanced llms’ numerical reasoning,

E. Schwartz, L. Choshen, J. Shtok, S. Doveh, L. Karlinsky, and A. Arbelle, “Numerologic: Number encoding for enhanced llms’ numerical reasoning,” inEMNLP, 2024

2024
[41]

Safeauto: Knowledge-enhanced safe autonomous driving with multi- modal foundation models,

J. Zhang, X. Yang, T. Wang, Y . Yao, A. Petiushko, and B. Li, “Safeauto: Knowledge-enhanced safe autonomous driving with multi- modal foundation models,” inICML, 2025

2025
[42]

xval: A continuous number encoding for large language models,

S. Golkar, M. Pettee, M. Eickenberg, A. Bietti, M. Cranmer, G. Krawezik, F. Lanusse, M. McCabe, R. Ohana, L. Parkeret al., “xval: A continuous number encoding for large language models,” in NeurIPS Workshop, 2023

2023
[43]

Interleaving text and number embeddings to solve mathemathics problems,

M. Alberts, G. Gabrieli, and I. E. Morales, “Interleaving text and number embeddings to solve mathemathics problems,” inNeurIPS, 2024

2024
[44]

The geometry of numerical reasoning: Language models compare numeric properties in linear subspaces,

A. O. El-Shangiti, T. Hiraoka, H. AlQuabeh, B. Heinzerling, and K. Inui, “The geometry of numerical reasoning: Language models compare numeric properties in linear subspaces,” inNAACL, 2025

2025
[45]

Language models encode the value of numbers linearly,

F. Zhu, D. Dai, and Z. Sui, “Language models encode the value of numbers linearly,” inCOLING, 2025

2025
[46]

Pre- trained language models learn remarkably accurate representations of numbers,

M. Kadl ˇc´ık, M. ˇStef´anik, T. Mickus, J. Kucha ˇr, and M. Spiegel, “Pre- trained language models learn remarkably accurate representations of numbers,” inEMNLP, 2025

2025
[47]

Adapt: Action-aware driving caption transformer,

B. Jin, X. Liu, Y . Zheng, P. Li, H. Zhao, T. Zhang, Y . Zheng, G. Zhou, and J. Liu, “Adapt: Action-aware driving caption transformer,” in ICRA, 2023

2023

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.

[1] [1]

The role of the hercules autonomous vehicle during the covid-19 pandemic: An autonomous logistic vehicle for contactless goods transportation,

T. Liu, Q. hai Liao, L. Gan, F. Ma, J. Cheng, X. Xie, Z. Wang, Y . Chen, Y . Zhu, S. Zhanget al., “The role of the hercules autonomous vehicle during the covid-19 pandemic: An autonomous logistic vehicle for contactless goods transportation,”IEEE Robotics & Automation Magazine, 2021

2021

[2] [2]

Hidden biases of end-to-end driving models,

B. Jaeger, K. Chitta, and A. Geiger, “Hidden biases of end-to-end driving models,” inICCV, 2023

2023

[3] [3]

Vad: Vectorized scene representation for efficient autonomous driving,

B. Jiang, S. Chen, Q. Xu, B. Liao, J. Chen, H. Zhou, Q. Zhang, W. Liu, C. Huang, and X. Wang, “Vad: Vectorized scene representation for efficient autonomous driving,” inICCV, 2023

2023

[4] [4]

Planning-oriented autonomous driving,

Y . Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wanget al., “Planning-oriented autonomous driving,” in CVPR, 2023

2023

[5] [5]

Drivegpt4-v2: Harnessing large language model capabilities for enhanced closed-loop autonomous driving,

Z. Xu, Y . Bai, Y . Zhang, Z. Li, F. Xia, K.-Y . K. Wong, J. Wang, and H. Zhao, “Drivegpt4-v2: Harnessing large language model capabilities for enhanced closed-loop autonomous driving,” inCVPR, 2025

2025

[6] [6]

Tinyllama: An open-source small language model,

P. Zhang, G. Zeng, T. Wang, and W. Lu, “Tinyllama: An open-source small language model,”arXiv:2401.02385, 2024

arXiv 2024

[7] [7]

Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,

W.-L. Chiang, Z. Li, Z. Lin, Y . Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y . Zhuang, J. E. Gonzalezet al., “Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,” https://vicuna. lmsys.org, 2023

2023

[8] [8]

Visual instruction tuning,

H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual instruction tuning,” in NeurIPS, 2023

2023