Pith. sign in

REVIEW 3 major objections 4 minor 44 references

WorkDrive claims that grounding reasoning in structured perception and aligning it with a single trajectory-consistency reward improves work-zone trajectory prediction by roughly 12% on the largest public work-zone benchmark.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 01:12 UTC pith:UT54J6NH

load-bearing objection A useful recipe with a hindsight-labeling problem: the 9% ADE gain from CoC isn't causal evidence as presented, but the GRPO consistency result stands. the 3 major comments →

arxiv 2607.14727 v1 pith:UT54J6NH submitted 2026-07-16 cs.CV

WorkDrive: Roadwork Chain of Causation for Autonomous Driving

Tianyi Jiang , Wen Zhang , Sihan Yang , Ming Lu , Wentao Zhang This is my paper
classification cs.CV
keywords work zonestrajectory predictionvision-language modelcausal reasoningperception groundingconsistency rewardreinforcement learningautonomous driving
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

WorkDrive claims that vision-language models can plan through roadwork zones more accurately if their reasoning is anchored to what is actually visible—cones, barriers, arrow boards—rather than the familiar cues they memorize from general driving data. The paper builds an automated perception pipeline that extracts structured scene facts and feeds them into a chain-of-causation annotation pipeline, producing reasoning labels that fine-tune an 8-billion-parameter vision-language model. It then applies reinforcement learning with a single reward: consistency between the model's declared lateral intent and the direction implied by its predicted trajectory. On the largest public work-zone benchmark, this combination reduces average displacement error by about 12% cumulatively (roughly 9% from grounded reasoning and 3% from consistency alignment) and halves the collision rate. The claim matters because it suggests that coarse behavioral consistency, not dense trajectory supervision, may be enough to align reasoning and planning under distribution shift.

Core claim

WorkDrive claims that work-zone trajectory planning improves when reasoning is built from structured perception rather than pretraining priors, and when reasoning is aligned to the trajectory by a single consistency reward. A four-step annotation pipeline extracts depth, detections, masks, and tracks, then has an LLM pick a decision from a six-frame window, identify causal factors from past frames only, synthesize a [Decision] because [Key Factors] chain, and auto-score it. Supervised fine-tuning injects this reasoning plus a lateral meta-action; group-relative policy optimization then maximizes only the agreement between the meta-action and trajectory curvature. On the largest public work-z

What carries the argument

The load-bearing mechanism is the perception-grounded Chain-of-Causation (CoC) annotation: a four-step pipeline (decision classification, causal-factor identification, chain synthesis, auto-evaluation) in which every step receives structured perception facts—object categories, positions, depth tiers, track IDs, and on-path/peripheral labels—as mandatory context, forcing the annotator to reason from work-zone objects rather than pretraining priors. The second load-bearing mechanism is the meta-action consistency reward: a single reinforcement-learning signal that compares the model's declared lateral intent (one of five categories) with the direction implied by the mean curvature of the predi

Load-bearing premise

The load-bearing premise is that a decision label produced by an annotator who sees the future trajectory (the six-frame annotation window includes three future frames) can be learned from the past-only inputs the model receives at inference; if the annotator's choices actually depend on seeing where the vehicle went, the model is learning hindsight rather than causal reasoning.

What would settle it

Generate the same annotations but with Step 1 decisions made from a past-only window (keyframe plus two previous frames, no future frames). If ADE@15 improvement over trajectory-only SFT drops or disappears, the gains come from future trajectory information visible in the six-frame decision window, not from causal reasoning grounded in perception.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Perception-grounded chain-of-causation labels, added through supervised fine-tuning, reduce average displacement error at the full horizon by about 9% relative to trajectory-only fine-tuning.
  • Adding a single consistency reward between the declared lateral meta-action and trajectory curvature brings a further ~3% ADE reduction, a 6.5% final-displacement improvement, and a 38.8% collision-rate drop (14.66% to 7.03%), with no explicit trajectory regression reward.
  • The gains are progressive: reasoning supervision alone mostly lowers collision rate; adding the meta-action anchor produces the larger ADE drop; the reinforcement-learning alignment then lifts meta-consistency from about 80% to 100% while improving all trajectory metrics.
  • At inference the model needs only the keyframe, two previous frames, and the observed trajectory; structured perception facts and ground-truth future are used only during annotation, so the deployed system is no heavier than the baseline.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the single-reward recipe transfers, any driving scenario with a coarse behavioral descriptor—turn type, speed regime, gap acceptance—could be aligned to trajectories without dense supervision, extending the approach beyond work zones to other long-tail events.
  • The planned release of the reasoning dataset would allow a direct test of the hindsight-label concern: re-annotate decisions from past-only frames and compare. If the gap persists, causal reasoning is doing the work; if it shrinks, the supervision signal is future information.
  • Because the consistency metric needs only the predicted trajectory and the declared meta-action, it could serve as a cheap, annotation-free proxy for planning quality on other datasets, independent of the full chain-of-causation pipeline.
  • The attention-redirecting hypothesis implies a testable scaling law: the benefit of perception grounding should grow with the prevalence of work-zone-specific objects in a scene and vanish on scenes dominated by familiar road furniture.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. WorkDrive proposes a three-stage framework to improve VLM trajectory prediction in roadwork zones. Phase I uses off-the-shelf perception models (Depth Anything 3, Rex-Omni, SAM 3, cross-frame tracking) to build structured per-frame scene JSON. Phase II uses an LLM (Qwen3-VL-32B or GPT-5.2high) to generate Chain-of-Causation reasoning labels through decision classification, causal factor identification, causal chain synthesis, and auto-evaluation. Phase III fine-tunes Qwen3-VL-8B with LoRA on reasoning/meta-action/trajectory targets, then applies GRPO with a single reward measuring consistency between the declared lateral meta-action and the mean curvature of the predicted trajectory. On ROADWork, the method improves ADE@15 from 12.10 (trajectory-only SFT) to 10.68 (with GRPO), FDE from 42.34 to 35.87, and CR from 14.66% to 7.03%. The paper claims perception-grounded causal reasoning is responsible for the 9.0% SFT-stage ADE gain and that consistency-based RL provides a further 3.0%.

Significance. If the causal-grounding interpretation were established, this would be a noteworthy contribution: the first automated perception-grounded Chain-of-Causation annotation pipeline for work zones, a reasoning dataset to be released, and evidence that single-reward consistency alignment can improve open-loop planning metrics. The paper's strengths include careful use of external trajectory metrics (ADE/FDE/CR) and a clear staged ablation. However, the central causal claim is undermined by the use of future trajectory during annotation, and the consistency metric is identical to the training reward. The empirical trajectory gains may still be practically valuable, but the paper's interpretation is not yet supported.

major comments (3)
  1. [§3.3, Eq. (6), Eq. (7), Eq. (4)] Step 1 (Eq. 6) gives the annotator a six-frame window including three future frames with the future trajectory rendered; Eq. (4) defines the on-path/peripheral separation using the ego's future trajectory; Step 2 (Eq. 7) conditions factor extraction on the future-informed decision. These future-informed labels become SFT targets for a model that at inference sees only keyframe+history (Eq. 1). The §3.3 sentence 'precluding any trajectory leakage' addresses only inference inputs, not training targets. Thus the Table 2 SFT-stage gains (+R+Meta, ADE 12.10→11.01, CR 14.66→11.49) can be explained by multi-task learning with a future-collapsed action label; they do not establish the causal-grounding hypothesis of §3.4. Please add an annotation variant that removes future information (e.g., Step 1 restricted to past frames, or corridor defined from past motion) and re-run SFT, or explicitly ref
  2. [§3.4 Eq. (14), §4.1] Meta-Consistency is defined in §4.1 exactly as the GRPO reward of Eq. (14), and the threshold constants 0.03/0.08 are fitted to the training curvature distribution. Reaching 100% Meta-Consistency after GRPO (Table 2) is therefore optimizing the reported metric, not an independent confirmation of alignment. This is acknowledged in the text, but the metric is presented in Table 1 alongside external metrics without a clear disclaimer. Please either report an independent measure (e.g., agreement between predicted meta-action and the direction of the ground-truth future trajectory) or relegate Meta-Consistency to a training diagnostic and base the alignment claim solely on ADE/FDE/CR.
  3. [§3.4] The consistency reward (Eq. 14) only checks self-agreement between the model's declared meta-action and the curvature of its own predicted trajectory. A model can achieve high consistency by adjusting either output; high consistency does not imply the meta-action is correct relative to actual road geometry or the ground-truth trajectory. The paper never evaluates meta-action prediction accuracy against ground truth. Since the GRPO stage improves ADE/FDE/CR, the empirical effect is real, but the interpretation that 'reasoning accurately describes the lateral intent' requires evidence that the intent itself is accurate. Please report meta-action accuracy before/after GRPO and, ideally, a control reward that encourages consistency with randomly assigned or fixed actions.
minor comments (4)
  1. [Figure 4] 'Casual Valid Yes' should be 'Causal Valid Yes'.
  2. [Eq. (11)] The equation includes a KL-regularization term but §4.1 says 'no KL regularization'; specify λ_KL=0 or remove the term from the equation.
  3. [§4.1] All experiments appear to be single-run; no random seeds or confidence intervals are reported. Given that the headline gains are a few percent, error bars or multiple seeds would strengthen the claims.
  4. [§4.1] The comparison set is limited to zero-shot VLMs. The trajectory-only SFT baseline is the appropriate reference for the internal ablations, but an independent trained trajectory-prediction baseline (e.g., a non-reasoning CNN/transformer planner) would make the external benchmarking more convincing.

Circularity Check

2 steps flagged

Future-informed CoC labels and reward-as-metric make the headline reasoning gains partially circular

specific steps
  1. fitted input called prediction [§3.3 Step 1 (Eq. 6) and Eq. (4); §3.4 Eq. (10); Table 1]
    "Step 1: Closed-set decision classification. The annotator LLM receives a six-frame window {I_{k−2},...,I_{k+3}} ... with observed and future trajectories rendered in red and green ... (d_lon,d_lat)=LLM({I_{k−2},...,I_{k+3}},P_k). The driving corridor is defined based on the ego vehicle's future trajectory ... w(y)=60+y/H×190 ... at inference time the trained model receives only the keyframe, two preceding frames, and the observed trajectory, with no access to P_k or the GT future trajectory, thereby precluding any trajectory leakage from the annotation stage into model predictions."

    The lateral decisions—and therefore the meta-action and reasoning labels used as SFT targets—are selected while the annotator is shown the future trajectory (Eq. 6), and the on-path/peripheral split in P_k is computed from a corridor built from the future trajectory (Eq. 4). Eq. (10) trains the past-only model to reproduce these future-informed labels; the +Meta-action ADE drop in Table 1 (12.10→11.01, the claimed 9.0%) is then attributed to perception-grounded causal reasoning. The inference-time statement about no access to the GT future trajectory addresses inputs only, not training labels, so the reported improvement is not an independent validation of the causal-grounding hypothesis.

  2. self definitional [§4.1 Metrics; Eq. (14) in §3.4; Table 1/2]
    "For reasoning-behavior consistency, we report Meta-Consistency, the agreement rate between the model’s declared lateral meta-action and the curvature-derived direction of the predicted trajectory (Eq. 14); this metric also serves as the sole reward signal during GRPO alignment (§3.4)."

    R_consistency (Eq. 14) is multiplied into the GRPO reward in Eq. (12) and is simultaneously reported as the Meta-Consistency evaluation metric. The post-GRPO value of 100.0% is therefore the optimized objective itself, not an independent measure of reasoning–trajectory alignment. Table 1's Meta-Consistency column cannot be read as validation of the consistency claim; the only independent evidence would be the ADE/FDE/CR columns, which are not the metric being reported as 100%. The curvature thresholds are also fitted to the training curvature distribution, further tying the metric to the training data.

full rationale

WorkDrive's RL-stage result has a genuinely independent component: after SFT, GRPO with only the consistency reward improves ADE@15 from 11.01 to 10.68, FDE from 38.35 to 35.87, and CR from 11.49% to 7.03%, with no explicit trajectory reward. Those numbers are not derivable from the reward by construction. However, the flagship attribution—perception-grounded reasoning reduces ADE by 9.0%—rests on CoC labels generated with the annotator shown the future trajectory (Eq. 6) and with the perception prompt's on-path split defined by the future trajectory (Eq. 4); the SFT model is trained to reproduce these hindsight labels from past-only inputs. The claimed 9.0% is therefore plausibly a label-leakage artifact rather than evidence for causal grounding. In addition, the post-GRPO 100% Meta-Consistency is the optimization target (Eq. 14 as reward and metric), so it provides no independent validation; the curvature thresholds are also fitted to the training curvature distribution. No load-bearing self-citation chain appears: the CoC framing is taken from external AR1/Alpamayo-R1, not from the present authors' prior work. Overall, the RL-stage result remains partially independent, but the headline CoC contribution is partially circular/leaky, giving a score of 6.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 1 invented entities

The framework depends on many hand-chosen thresholds (corridor width, depth percentiles, tracking thresholds, curvature reward thresholds) and on strong assumptions about the reliability of each perception and LLM stage. These do not necessarily invalidate the empirical result, but they make the causal-grounding claim stronger than the evidence.

free parameters (5)
  • Corridor half-width constants (Eq. 4) = w(y) = 60 + (y/H) * 190
    Hand-chosen perspective-scaled corridor width; determines which objects are labeled ON-PATH versus peripheral and therefore which facts enter the reasoning prompts.
  • Depth-level percentile thresholds = 33rd and 66th percentiles of detected object mean depths
    Hand-selected divisions for closest/mid-range/farthest; affects every object's depth label in the perception prompt P_k.
  • Cross-frame tracking thresholds = IoU > 0.3 and embedding cosine similarity > 0.7
    Hand-thresholded instance matching with majority-vote class unification; no ablation or sensitivity analysis is given.
  • Curvature-to-meta-action thresholds = |κ| < 0.03; 0.03–0.08; ≥ 0.08
    Chosen as the 25th/75th and 5th/95th percentiles of the training trajectory-curvature distribution; directly sets both the GRPO reward mapping and the Meta-Consistency evaluation.
  • Reward grading values = 1.0 / 0.5 / 0.0
    Hand-chosen grading for exact match, same-direction-different-intensity, and mismatch; no sensitivity analysis is provided.
axioms (5)
  • domain assumption Per-frame percentile depth binning from an affine-invariant monocular depth model yields meaningful closest/mid-range/farthest categorizations across frames.
    Depth Anything 3 maps are not metrically comparable across frames; the 33/66 percentile scheme assumes the relative ordering is decision-relevant (§3.2).
  • domain assumption The two-pass category lists (ROADWork work-zone categories + nuScenes/Waymo general categories) cover every decision-relevant object, and retaining work-zone categories on IoU overlap is always correct.
    If a temporary object is missing from both lists or loses to a general category, the reasoning prompt misses a potential causal factor (§3.2).
  • domain assumption The bounding-box bottom-center is a reliable ground-contact proxy for on-path status.
    On-path/peripheral classification in Eq. (5) tests only the bbox bottom-center, not mask overlap; a cone partially entering the corridor could be misclassified and excluded from P_k (§3.3).
  • domain assumption LLM-generated decisions and causal factors are stable and causally valid enough to serve as training supervision.
    The entire CoC dataset is produced by Qwen3-VL-32B-Instruct or GPT-5.2high with only n=100 samples human-scored by a single annotator (§4.4, Table 3).
  • domain assumption GRPO with no KL regularization and a single consistency reward is a stable and sufficient alignment method.
    Eq. (11) is applied with λ_KL = 0, full-parameter training, and one epoch; the paper attributes the ADE/FDE/CR gains to this specific choice without reward-shaping or trajectory supervision (§3.4, §4.2).
invented entities (1)
  • Lateral meta-action (5-class steering intent) no independent evidence
    purpose: Intermediate output bridging reasoning text and trajectory: a <meta_action> token trained during SFT and used as the comparison target in the GRPO consistency reward.
    The meta-action is not validated against external behavioral annotations; its only quantitative support is the Meta-Consistency metric, which is the same formula as the reward (Eq. 14), so it does not provide independent evidence of meaningful intent semantics.

pith-pipeline@v1.3.0-alltime-deepseek · 15617 in / 16755 out tokens · 157752 ms · 2026-08-02T01:12:52.164677+00:00 · methodology

0 comments
read the original abstract

Autonomous driving vision-language models (VLMs) struggle in roadwork zones, where familiar visual cues such as lane markings and permanent signs are altered or absent, and temporary devices such as cones and barriers redefine the drivable corridor. VLMs can detect these objects, but without explicit guidance they anchor their reasoning on familiar elements from pre-training and fail to connect work-zone observations to correct planning decisions. We propose WorkDrive, a framework that constructs perception-grounded causal reasoning for work zones and aligns it with trajectory prediction. An automated multitask perception pipeline extracts structured scene facts and injects them into a Chain-of-Causation (CoC) annotation pipeline, redirecting the annotator's attention to domain-specific elements. The resulting reasoning labels are used for supervised fine-tuning, followed by reinforcement learning with a single reward: consistency between lateral meta-actions and the predicted trajectory. On ROADWork, the largest public work-zone dataset, the proposed roadwork CoC reduces trajectory average displacement error (ADE) by 9.0\%, and consistency-based GRPO yields a further 3.0\%, achieving progressive improvement over the trajectory-only baseline. Code and data will be publicly released.

Figures

Figures reproduced from arXiv: 2607.14727 by Ming Lu, Sihan Yang, Tianyi Jiang, Wentao Zhang, Wen Zhang.

Figure 1
Figure 1. Figure 1: Trajectory prediction in a work-zone scene with [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the WorkDrive framework. Phase I extracts structured scene representations via depth estimation, [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Driving corridor and on-path classification. The [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: A complete Chain-of-Causation annotation exam [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: SFT training loss curves across three configurations. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative comparison of CoC reasoning with and [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

44 extracted references · 15 linked inside Pith

  1. [1]

    Autoweek. 2017. Tesla Model S Autopilot Strikes Again: Dallas Crash. https://www.autoweek.com/news/technology/a1816086/tesla-model-s- autopilot-strikes-again-dallas-crash. News article; accessed 2026-03-29

    2017

  2. [2]

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, et al . 2025. Qwen3-VL Technical Report.arXiv preprint arXiv:2511.21631(2025). https://arxiv.org/abs/2511.21631

    Pith/arXiv arXiv 2025

  3. [3]

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. 2020. nuscenes: A multimodal dataset for autonomous driving. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 11621–11631

    2020

  4. [4]

    Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath, Ronghang Hu, Didac Suris, Chaitanya Ryali, et al. 2025. SAM 3: Segment Anything with Concepts.arXiv preprint arXiv:2511.16719(2025). https://arxiv.org/abs/2511. 16719

    Pith/arXiv arXiv 2025

  5. [5]

    2009.Manual on Uniform Traffic Control Devices for Streets and Highways

    Federal Highway Administration. 2009.Manual on Uniform Traffic Control Devices for Streets and Highways. U.S. Department of Transportation. https: //mutcd.fhwa.dot.gov/ 2009 Edition with Revisions 1 and 2

    2009

  6. [6]

    Haoyu Fu, Diankun Zhang, Zongchuang Zhao, Jianfeng Cui, Dingkang Liang, Chong Zhang, Dingyuan Zhang, Hongwei Xie, Bing Wang, and Xiang Bai. 2025. Orion: A holistic end-to-end autonomous driving framework by vision-language instructed action generation. InProceedings of the IEEE/CVF International Confer- ence on Computer Vision. 24823–24834

    2025

  7. [7]

    Anurag Ghosh, Shen Zheng, Robert Tamburo, Khiem Vuong, Juan Alvarez- Padilla, Hailiang Zhu, Michael Cardei, Nicholas Dunn, Christoph Mertz, and Srinivasa G Narasimhan. 2025. Roadwork: A dataset and benchmark for learning to recognize, observe, analyze and drive through work zones. InProceedings of the IEEE/CVF International Conference on Computer Vision. ...

    2025

  8. [8]

    Google. 2025. A New Era of Intelligence with Gemini 3. https://blog.google/ products-and-platforms/products/gemini/gemini-3/ Accessed: 2026-07-14

    2025

  9. [9]

    Thomas Gumpp, Dennis Nienhuser, Rebecca Liebig, and J Marius Zollner. 2009. Recognition and tracking of temporary lanes in motorway construction sites. In 2009 IEEE Intelligent Vehicles Symposium. IEEE, 305–310

    2009

  10. [10]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. InInternational Conference on Learning Representations. https://openreview.net/forum?id=nZeVKeeFYf9

    2022

  11. [11]

    Yihan Hu, Jiazhi Yang, Li Chen, Keyu Li, Chonghao Sima, Xizhou Zhu, Siqi Chai, Senyao Du, Tianwei Lin, Wenhai Wang, Lewei Lu, Xiaosong Jia, Qiang Liu, Jifeng Dai, Yu Qiao, and Hongyang Li. 2023. Planning-oriented autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 17853–17862

    2023

  12. [12]

    Jyh-Jing Hwang, Runsheng Xu, Hubert Lin, Wei-Chih Hung, Jingwei Ji, Kristy Choi, Di Huang, Tong He, Paul Covington, Benjamin Sapp, Yin Zhou, James Guo, Dragomir Anguelov, and Mingxing Tan. 2024. Emma: End-to-end multimodal model for autonomous driving.arXiv preprint arXiv:2410.23262(2024)

    Pith/arXiv arXiv 2024

  13. [13]

    Anqing Jiang, Yu Gao, Zhigang Sun, Yiru Wang, Jijun Wang, Jinghao Chai, Qian Cao, Yuweng Heng, Hao Jiang, Yunda Dong, Zongzheng Zhang, Xianda Guo, Hao Sun, and Hao Zhao. 2025. Diffvla: Vision-language guided diffusion planning for autonomous driving.arXiv preprint arXiv:2505.19381(2025)

    Pith/arXiv arXiv 2025

  14. [14]

    Bo Jiang, Shaoyu Chen, Bencheng Liao, Xingyu Zhang, Wei Yin, Qian Zhang, Chang Huang, Wenyu Liu, and Xinggang Wang. 2024. Senna: Bridging large vision-language models and end-to-end autonomous driving.arXiv preprint arXiv:2410.22313(2024)

    Pith/arXiv arXiv 2024

  15. [15]

    Bo Jiang, Shaoyu Chen, Qian Zhang, Wenyu Liu, and Xinggang Wang. 2025. Al- phadrive: Unleashing the power of vlms in autonomous driving via reinforcement learning and reasoning.arXiv preprint arXiv:2503.07608(2025)

    Pith/arXiv arXiv 2025

  16. [16]

    Qing Jiang, Junan Huo, Xingyu Chen, Yuda Xiong, Zhaoyang Zeng, Yihao Chen, Tianhe Ren, Junzhi Yu, and Lei Zhang. 2025. Detect Anything via Next Point Prediction.arXiv preprint arXiv:2510.12798(2025). https://arxiv.org/abs/2510. 12798

    arXiv 2025

  17. [17]

    Jinwoo Kim, Kyounghwan An, and Donghwan Lee. 2024. RoSA Dataset: Road Construct Zone Segmentation for Autonomous Driving. InEuropean Conference on Computer Vision. Springer, 322–338

    2024

  18. [18]

    Kaican Li, Kai Chen, Haoyu Wang, Lanqing Hong, Chaoqiang Ye, Jianhua Han, Yukuai Chen, Wei Zhang, Chunjing Xu, Dit-Yan Yeung, Xiaodan Liang, Zhenguo Li, and Hang Xu. 2022. Coda: A real-world road corner case dataset for object detection in autonomous driving. InEuropean Conference on Computer Vision. Springer, 406–423

    2022

  19. [19]

    Yue Li, Meng Tian, Dechang Zhu, Jiangtong Zhu, Zhenyu Lin, Zhiwei Xiong, and Xinhai Zhao. 2026. Drive-r1: Bridging reasoning and planning in vlms for autonomous driving with reinforcement learning. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 40. 6708–6716

    2026

  20. [20]

    Yongkang Li, Kaixin Xiong, Xiangyu Guo, Fang Li, Sixu Yan, Gangwei Xu, Lijun Zhou, Long Chen, Haiyang Sun, Bing Wang, Kun Ma, Guang Chen, Hangjun Ye, Wenyu Liu, and Xinggang Wang. 2025. Recogdrive: A reinforced cognitive framework for end-to-end autonomous driving.arXiv preprint arXiv:2506.08052 (2025)

    Pith/arXiv arXiv 2025

  21. [21]

    Yifan Liao, Zhen Sun, Xiaoyun Qiu, Zixiao Zhao, Wenbing Tang, Xinlei He, Xinhu Zheng, Tianwei Zhang, Xinyi Huang, and Xingshuo Han. 2025. Work zones challenge vlm trajectory planning: Toward mitigation and robust autonomous driving.arXiv preprint arXiv:2510.02803(2025)

    arXiv 2025

  22. [22]

    Chen, Zhenyu Li, Guang Shi, Jiashi Feng, and Bingyi Kang

    Haotong Lin, Sili Chen, Jun Hao Liew, Donny Y. Chen, Zhenyu Li, Guang Shi, Jiashi Feng, and Bingyi Kang. 2025. Depth Anything 3: Recovering the Visual Space from Any Views.arXiv preprint arXiv:2511.10647(2025). https://arxiv. org/abs/2511.10647

    Pith/arXiv arXiv 2025

  23. [23]

    Los Angeles Times. 2023. Robot Car Collides with Fire Truck in San Fran- cisco. https://www.latimes.com/california/story/2023-08-19/robot-car-collides- with-fire-truck. News article; accessed 2026-03-29

    2023

  24. [24]

    Yuechen Luo, Fang Li, Shaoqing Xu, Zhiyi Lai, Lei Yang, Qimao Chen, Ziang Luo, Zixun Xie, Shengyin Jiang, Jiaxin Liu, Long Chen, Bing Wang, and Zhi xin Yang. 2025. Adathinkdrive: Adaptive thinking via reinforcement learning for autonomous driving.arXiv preprint arXiv:2509.13769(2025)

    arXiv 2025

  25. [25]

    Bonolo Mathibela, Ingmar Posner, and Paul Newman. 2013. A roadwork scene signature based on the opponent colour model. In2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 4394–4400

    2013

  26. [26]

    OpenAI. 2025. Introducing GPT-5.2. https://openai.com/index/introducing-gpt- 5-2/ Accessed: 2026-07-14

    2025

  27. [27]

    Qihang Peng, Xuesong Chen, Chenye Yang, Shaoshuai Shi, and Hongsheng Li

  28. [28]

    2019.Safe Integration of Automated Vehicles into Work Zones

    Pennsylvania Department of Transportation. 2019.Safe Integration of Automated Vehicles into Work Zones. Technical Report. U.S. Department of Transportation. Accessed: 2026-03-29

    2019

  29. [29]

    Nishad Sahu, Gregory T Su, Shounak Sural, Sean Brennan, and Ragunathan Raj Rajkumar. 2025. Towards the Safe Operation of Autonomous Vehicles in Work Zones. In2025 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2380–2387

    2025

  30. [30]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and Guo Daya. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300(2024)

    Pith/arXiv arXiv 2024

  31. [31]

    Chonghao Sima, Katrin Renz, Kashyap Chitta, Li Chen, Hanxue Zhang, Chen- gen Xie, Jens Beißwenger, Ping Luo, Andreas Geiger, and Hongyang Li. 2024. Drivelm: Driving with graph visual question answering. InEuropean conference on computer vision. Springer, 256–274

    2024

  32. [32]

    Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Sheng Zhao, Shuyang Cheng, Yu Zhang, Jonathon Shlens, Zhifeng Chen, and Dragomir Anguelov...

    2020

  33. [33]

    Shounak Sural, Nishad Sahu, and Ragunathan Rajkumar. 2026. WorkZone3D: A Multimodal Dataset for 3D Work Zone Perception in Autonomous Driving. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 3972–3981

    2026

  34. [34]

    Xiaoyu Tian, Junru Gu, Bailin Li, Yicheng Liu, Yang Wang, Zhiyong Zhao, Kun Zhan, Peng Jia, Xianpeng Lang, and Hang Zhao. 2024. Drivevlm: The conver- gence of autonomous driving and large vision-language models.arXiv preprint arXiv:2402.12289(2024)

    Pith/arXiv arXiv 2024

  35. [35]

    Tianqi Wang, Enze Xie, Ruihang Chu, Zhenguo Li, and Ping Luo. 2024. Drivecot: Integrating chain-of-thought reasoning with end-to-end driving.arXiv preprint arXiv:2403.16996(2024). 9 Jiang et al

    Pith/arXiv arXiv 2024

  36. [36]

    Yan Wang, Wenjie Luo, Junjie Bai, Yulong Cao, Tong Che, Ke Chen, Yuxiao Chen, Jenna Diamond, Yifan Ding, Wenhao Ding, Liang Feng, Greg Heinrich, Jack Huang, Peter Karkus, Boyi Li, Pinyi Li, Tsung-Yi Lin, Dongran Liu, Ming-Yu Liu, Langechuan Liu, Zhijian Liu, Jason Lu, Yunxiang Mao, Pavlo Molchanov, Lindsey Pavao, Zhenghao Peng, Mike Ranzinger, Ed Schmerli...

    Pith/arXiv arXiv 2025

  37. [37]

    Zhaohui Wang, Tengbo Yu, and Hao Tang. 2025. CoT4AD: A Vision-Language- Action Model with Explicit Chain-of-Thought Reasoning for Autonomous Driv- ing.arXiv preprint arXiv:2511.22532(2025)

    arXiv 2025

  38. [38]

    Zhenhua Xu, Yujia Zhang, Enze Xie, Zhen Zhao, Yong Guo, Kwan-Yee K Wong, Zhenguo Li, and Hengshuang Zhao. 2024. Drivegpt4: Interpretable end-to-end autonomous driving via large language model.IEEE Robotics and Automation Letters9, 10 (2024), 8186–8193

    2024

  39. [39]

    Songyan Zhang, Wenhui Huang, Zhan Chen, Chua Jiahao Collister, Qihang Huang, and Chen Lv. 2025. OpenREAD: Reinforced Open-Ended Reason- ing for End-to-End Autonomous Driving with LLM-as-Critic.arXiv preprint arXiv:2512.01830(2025)

    arXiv 2025

  40. [40]

    Zhenguo Zhang, Haohan Zheng, Yishen Wang, Le Xu, Tianchen Deng, Xue- feng Chen, Qu Chen, Bo Zhang, and Wuxiong Huang. 2025. OmniDrive-R1: Reinforcement-driven Interleaved Multi-modal Chain-of-Thought for Trustwor- thy Vision-Language Autonomous Driving.arXiv preprint arXiv:2512.14044 (2025)

    Pith/arXiv arXiv 2025

  41. [41]

    Weicheng Zheng, Xiaofei Mao, Nanfei Ye, Pengxiang Li, Kun Zhan, Xian- peng Lang, and Hang Zhao. 2025. DriveAgent-R1: Advancing VLM-based Au- tonomous Driving with Active Perception and Hybrid Thinking.arXiv preprint arXiv:2507.20879(2025)

    Pith/arXiv arXiv 2025

  42. [42]

    Xingcheng Zhou, Xuyuan Han, Feng Yang, Yunpu Ma, Volker Tresp, and Alois Knoll. 2026. Opendrivevla: Towards end-to-end autonomous driving with large vision language action model. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 40. 13782–13790

    2026

  43. [43]

    Zewei Zhou, Tianhui Cai, Seth Z Zhao, Yun Zhang, Zhiyu Huang, Bolei Zhou, and Jiaqi Ma. 2025. Autovla: A vision-language-action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine-tuning. arXiv preprint arXiv:2506.13757(2025). 10

    Pith/arXiv arXiv 2025

  44. [2025]

    ColaVLA: Leveraging Cognitive Latent Reasoning for Hierarchical Parallel Trajectory Planning in Autonomous Driving.arXiv preprint arXiv:2512.22939 (2025)

    arXiv 2025