Pith. sign in

REVIEW 2 major objections 2 minor 60 cited by

Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving

T0 review · 2 major / 2 minor · reviewed 2026-05-15 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Senna uses a large vision-language model for natural language driving plans that an end-to-end model converts into precise trajectories.

desk verdict Senna splits LVLM language planning from E2E trajectories and reports solid gains after DriveX pre-training, but the translation step between them lacks isolating evidence. read the letter →

arxiv 2410.22313 v1 pith:P4KL45MZ submitted 2024-10-29 cs.CV cs.RO

Bo Jiang , Shaoyu Chen , Bencheng Liao , Xingyu Zhang , Wei Yin , Qian Zhang , Chang Huang , Wenyu Liu
show 1 more author
Xinggang Wang
This is my paper
classification cs.CVcs.RO
keywords autonomousdrivingvision-languagemodelsend-to-endtrajectorypredictionplanningdecisionsnuScenespre-training
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper seeks to merge the commonsense reasoning of large vision-language models with the data-driven planning of end-to-end autonomous driving systems. It proposes decoupling high-level decisions expressed in language from the generation of precise numerical trajectories to address limitations in complex and rare scenarios. A sympathetic reader would care because current approaches either lack reasoning or struggle with precise control, and successful integration could advance toward fully autonomous vehicles. The work shows concrete gains through pre-training on large datasets followed by fine-tuning on nuScenes.

What carries the argument

The hybrid architecture that separates natural language planning decisions from trajectory prediction, with the LVLM handling scene understanding and reasoning.

What would settle it

A high rate of collisions in rare scenarios where the generated language plan is accurate but the resulting trajectory deviates would show the translation step fails.

Watch

Extended reading notes

Core claim

Senna decouples high-level planning from low-level trajectory prediction by having its vision-language component generate natural language planning decisions using multi-image encoding and multi-view prompts, while the end-to-end component predicts precise trajectories, supported by planning-oriented question-answering data and a three-stage training strategy. This yields state-of-the-art performance, including a 27.12% reduction in average planning error and 33.33% in collision rate after pre-training on DriveX and fine-tuning on nuScenes.

Load-bearing premise

Natural language planning outputs from the LVLM can be translated into low-level trajectories by the E2E model without introducing critical errors or losing necessary detail.

Editorial extensions

If this is right

  • Senna achieves state-of-the-art planning performance on standard benchmarks.
  • Pre-training on large-scale data like DriveX followed by fine-tuning significantly improves results over no pre-training.
  • The system demonstrates improved handling of complex and rare scenarios through commonsense from the language model.
  • Cross-scenario generalization and transferability support progress toward fully autonomous driving.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • This bridging strategy might apply to other domains needing both high-level reasoning and low-level precision, such as robotics manipulation.
  • Further work could test whether the language-to-trajectory translation holds in real-time, safety-critical driving without simulation artifacts.
  • Exploring richer prompt engineering or additional sensor inputs could enhance the multi-view scene understanding.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper proposes Senna, which decouples high-level planning from low-level control in autonomous driving by pairing an LVLM (Senna-VLM) that outputs natural-language planning decisions with an E2E model (Senna-E2E) that predicts precise trajectories. Senna-VLM uses multi-image encoding, multi-view prompts, and planning-oriented QAs trained in three stages; experiments on DriveX pre-training followed by nuScenes fine-tuning report SOTA planning results, including a 27.12% drop in average planning error and 33.33% reduction in collision rate relative to the no-pretrain baseline.

Significance. If the empirical gains hold under rigorous controls, the work offers a practical route to inject commonsense reasoning into E2E driving pipelines, addressing their documented weakness in rare or complex scenes. The public release of code and models is a concrete strength that supports reproducibility and follow-on research.

major comments (2)
  1. [Abstract and §4] Abstract and §4 (Experiments): the reported 27.12% planning-error and 33.33% collision-rate reductions are presented without error bars, without an ablation isolating the language-to-trajectory translation step, and without a full table of competing baselines; these omissions make it impossible to attribute the gains specifically to the LVLM component rather than to scale or training schedule.
  2. [§3.2] §3.2 (Senna-VLM to Senna-E2E interface): the central architectural claim rests on the assumption that natural-language plans can be mapped to E2E trajectory inputs without critical loss of spatial or temporal detail; no quantitative metric (e.g., conversion accuracy on held-out ambiguous prompts or rare-scenario subsets) is provided to validate this interface, which directly undermines the attribution of performance improvements to the LVLM.
minor comments (2)
  1. [Abstract] The abstract states that LVLMs are ill-suited for numerical outputs yet does not quantify how the chosen conditioning mechanism (tokenization of planning QAs into E2E inputs) mitigates this limitation.
  2. [§4] Figure captions and §4 should explicitly list the exact metrics (e.g., L2 error, collision rate) and the precise definition of the “model without pre-training” baseline for each reported number.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on our manuscript. The comments highlight important aspects for strengthening the experimental validation and interface analysis. We address each point below and commit to revisions that improve clarity and rigor without altering the core contributions.

read point-by-point responses

  1. Referee: [Abstract and §4] Abstract and §4 (Experiments): the reported 27.12% planning-error and 33.33% collision-rate reductions are presented without error bars, without an ablation isolating the language-to-trajectory translation step, and without a full table of competing baselines; these omissions make it impossible to attribute the gains specifically to the LVLM component rather than to scale or training schedule.

    Authors: We agree that error bars and expanded baselines would improve attribution. In the revised version we will add standard deviations computed over three independent runs for the reported metrics on nuScenes. We will also expand the main results table to include all relevant competing methods with their full metrics. For isolating the language-to-trajectory step, we will insert a new controlled ablation that feeds ground-truth parsed plans versus Senna-VLM-generated plans into the identical Senna-E2E backbone; this directly quantifies the contribution of the LVLM component beyond scale or schedule. These additions will appear in the updated §4 and supplementary material. revision: yes

  2. Referee: [§3.2] §3.2 (Senna-VLM to Senna-E2E interface): the central architectural claim rests on the assumption that natural-language plans can be mapped to E2E trajectory inputs without critical loss of spatial or temporal detail; no quantitative metric (e.g., conversion accuracy on held-out ambiguous prompts or rare-scenario subsets) is provided to validate this interface, which directly undermines the attribution of performance improvements to the LVLM.

    Authors: The interface uses a lightweight deterministic parser that extracts structured fields (target velocity, lane-change flag, stop/go decision, and approximate waypoint offsets) from the generated natural-language sentence; these fields are tokenized and concatenated to the E2E visual features. While the current manuscript provides qualitative examples, we acknowledge the absence of a quantitative fidelity metric. In revision we will add a parser-accuracy evaluation on a held-out set of 500 Senna-VLM outputs (including 100 rare-scenario cases), reporting exact-match accuracy for each extracted field against human annotations. This metric will be reported in an expanded §3.2 and will support the claim that critical spatial-temporal information is preserved. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

Empirical benchmark results with no self-referential derivations

full rationale

The paper reports measured planning error and collision reductions (27.12% and 33.33%) from pre-training on DriveX followed by fine-tuning on nuScenes, compared against an internal no-pretrain baseline. These are direct empirical outcomes on external datasets rather than any mathematical derivation, fitted parameter, or prediction that reduces to a quantity defined inside the paper. No equations, uniqueness theorems, or ansatzes are invoked in a load-bearing way; the architecture (multi-image encoding, planning-oriented QAs, three-stage training) is described independently of the final metrics, which remain falsifiable on the cited benchmarks.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

The central claim rests on the domain assumption that LVLMs can produce reliable high-level plans in language and that these plans integrate cleanly with an E2E trajectory predictor. No new physical entities or ad-hoc constants are introduced beyond standard neural-network training.

assumptions (1)
  • domain assumption LVLMs excel in scene understanding and reasoning while end-to-end models struggle with commonsense in rare scenarios
    Stated directly in the opening of the abstract as the motivation for the hybrid design.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving." pith.science (2026). https://pith.science/paper/P4KL45MZ

@misc{pith2026241022313,
  author       = {Pith},
  title        = {Pith review of: Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P4KL45MZ}},
  note         = {Machine review of arXiv:2410.22313}
}
read the original abstract

End-to-end autonomous driving demonstrates strong planning capabilities with large-scale data but still struggles in complex, rare scenarios due to limited commonsense. In contrast, Large Vision-Language Models (LVLMs) excel in scene understanding and reasoning. The path forward lies in merging the strengths of both approaches. Previous methods using LVLMs to predict trajectories or control signals yield suboptimal results, as LVLMs are not well-suited for precise numerical predictions. This paper presents Senna, an autonomous driving system combining an LVLM (Senna-VLM) with an end-to-end model (Senna-E2E). Senna decouples high-level planning from low-level trajectory prediction. Senna-VLM generates planning decisions in natural language, while Senna-E2E predicts precise trajectories. Senna-VLM utilizes a multi-image encoding approach and multi-view prompts for efficient scene understanding. Besides, we introduce planning-oriented QAs alongside a three-stage training strategy, which enhances Senna-VLM's planning performance while preserving commonsense. Extensive experiments on two datasets show that Senna achieves state-of-the-art planning performance. Notably, with pre-training on a large-scale dataset DriveX and fine-tuning on nuScenes, Senna significantly reduces average planning error by 27.12% and collision rate by 33.33% over model without pre-training. We believe Senna's cross-scenario generalization and transferability are essential for achieving fully autonomous driving. Code and models will be released at https://github.com/hustvl/Senna.

Discussion (0). Sign in to comment.

Forward citations

Cited by 60 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMs

    cs.AI 2026-08 conditional novelty 7.0 of 10

    Hiding future trajectory information until after a driving model forms its decision reduces rationalization and improves verifiable autonomous-driving reasoning in the proposed AD-MCQ and DEFT-RLVR framework.

  2. VLADriveBench: Evaluating CoT-Action Relationship in VLA for Autonomous Driving

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    VLADriveBench combines observational metrics and CoT intervention protocols to evaluate the relevance and causality of reasoning in vision-language-action models for autonomous driving, revealing divergent model behaviors.

  3. 4DLidarOpen: An Open 4D FMCW Lidar Dataset for Motion-Aware Autonomous Driving

    cs.RO 2026-05 unverdicted novelty 7.0 of 10

    4DLidarOpen is a new open dataset providing synchronized 4D FMCW Lidar velocity measurements, multi-Lidar and camera data, and 3D bounding-box annotations with track IDs to support benchmarks on 3D detection, BEV segm...

  4. MindVLA-U1: VLA Beats VA with Unified Streaming Architecture for Autonomous Driving

    cs.RO 2026-05 unverdicted novelty 7.0 of 10

    MindVLA-U1 introduces a unified streaming VLA with shared backbone, framewise memory, and language-guided action diffusion that surpasses human drivers on WOD-E2E planning metrics.

  5. VECTOR-Drive: Tightly Coupled Vision-Language and Trajectory Expert Routing for End-to-End Autonomous Driving

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    VECTOR-DRIVE couples vision-language reasoning and trajectory planning in one Transformer via semantic expert routing and flow-matching, reaching 88.91 driving score on Bench2Drive.

  6. ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving

    cs.CV 2025-06 unverdicted novelty 7.0 of 10

    ReCogDrive unifies VLM scene understanding with a diffusion planner reinforced by DiffGRPO to reach state-of-the-art results on NAVSIM and Bench2Drive benchmarks.

  7. AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoning

    cs.CV 2025-03 unverdicted novelty 7.0 of 10

    AlphaDrive uses GRPO-based RL rewards and two-stage SFT+RL training on VLMs to improve autonomous driving planning performance and efficiency while producing emergent multimodal capabilities.

  8. BEVLM: Distilling Semantic Knowledge from LLMs into Bird's-Eye View Representations

    cs.CV 2026-03 conditional novelty 6.5 of 10

    BEV tokens give LLMs stronger cross-view spatial reasoning than multi-view image tokens, and reverse-distilling LLM semantics into BEV encoders measurably improves closed-loop safety-critical driving.

  9. WAM-Diff2: Hierarchical AR-to-Diffusion Distillation for Highly Efficient Autonomous Driving VLA

    cs.RO 2026-08 conditional novelty 6.0 of 10

    A three-stage distillation converts an autoregressive driving VLA into a block-causal masked diffusion model, preserving planning accuracy while decoding 2.8x faster (15.1x with optimized kernels).

  10. Latent-Centroid Steering: Single-Pass Classifier-Free Guidance for Command-Aligned Autonomous Driving

    cs.CV 2026-07 conditional novelty 6.0 of 10

    LCS steers a VLA driving model's latent features toward command-specific centroids to improve command following at roughly half the inference cost of two-pass CFG.

  11. Cognitive Dual-Process Planning for Autonomous Driving with Structured Scene Knowledge and Verifiable Reasoning-Action Consistency

    cs.RO 2026-07 conditional novelty 6.0 of 10

    A dual-process VLM planner routes easy scenes to fast prediction and hard scenes to structured reasoning with rule-verified consistency rewards, reaching 80.14% planning accuracy and 97.20% LCS on a manually verified ...

  12. WorkDrive: Roadwork Chain of Causation for Autonomous Driving

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Perception-grounded Chain-of-Causation labels plus a consistency-only GRPO reward lower ROADWork trajectory ADE@15 from 12.10 to 10.68 pixels (about -12%) over a trajectory-only SFT baseline.

  13. WCog-VLA: A Dual-Level World-Cognitive Vision-Language-Action Model for End-to-End Autonomous Driving

    cs.CV 2026-07 conditional novelty 6.0 of 10

    WCog-VLA couples Game-CoT semantic reasoning with an aligned decoupled diffusion transformer to generate joint multi-agent trajectories and reaches 92.9 PDMS on NAVSIM.

  14. DriveVer: Lightweight Trajectory Evaluator as Test-Time Verifier for Autonomous Driving

    cs.CV 2026-07 unverdicted novelty 6.0 of 10

    DriveVer is a lightweight dual-head test-time verifier that predicts safety confidence scores and geometric refinement vectors for candidate trajectories, improving base planners on the NAVSIM benchmark.

  15. MVPruner: Dynamic Token Pruning for Accelerating Multi-view Vision-Language Models in Autonomous Driving

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    MVPruner is a two-stage dynamic token pruning technique that uses view diversity for initial budget allocation and instruction text for task-aligned selection, delivering 87.3% FLOPs reduction and 4.97x prefilling spe...

  16. Neuro-Symbolic Drive: Rule-Grounded Faithful Reasoning for Driving VLAs

    cs.AI 2026-06 unverdicted novelty 6.0 of 10

    The paper fine-tunes Qwen3.5-4B as a driving VLA using serialized decision traces from rule-based planners, reporting reduced ADE and miss rate on a simulator benchmark with camera inputs.

  17. OmniSpace: Efficient Geometry Awareness for Autonomous Vehicles MLLMs

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    OmniSpace is a plug-and-play method that improves spatial reasoning in MLLMs for AV by injecting camera pose, using epipolar attention across views, and distilling 3D geometric knowledge to overcome weak cross-view co...

  18. VLGA: Vision-Language-Geometry-Action Models for Autonomous Driving

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    VLGA introduces geometry as a fourth modality in VLA models via pointmap regression loss, reporting SOTA open-loop and closed-loop driving metrics on nuScenes and Bench2Drive.

  19. An Exposure-Time-Aligned Primary-Path Architecture for Autonomous-Driving ECUs

    cs.RO 2026-06 unverdicted novelty 6.0 of 10

    Proposes Primary-Path, Exposure-Time-Aligned, and Co-Path Coexistence principles enabling modular and E2E paths to run together on production AD-ECUs with 296 ms mean latency.

  20. An Exposure-Time-Aligned Primary-Path Architecture for Autonomous-Driving ECUs

    cs.RO 2026-06 conditional novelty 6.0 of 10

    Primary-path enclosure plus τ_exp-aligned fusion on a Dual-SoC AD-ECU yields 296 ms mean shutter-to-planner latency within a 350 ms budget while co-running modular and E2E paths.

  21. Test-Time Trajectory Optimization for Autonomous Driving

    cs.RO 2026-06 unverdicted novelty 6.0 of 10

    TOAD applies test-time Cross-Entropy Method optimization to refine trajectories using the planner's scorer as a reward function, improving end-to-end autonomous driving performance without retraining.

  22. DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Driving

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    DriveWAM converts video generative priors into a unified video-action policy for driving, reporting strong benchmark performance and positive scaling from 4k to 100k clips.

  23. Distill to Think, Foresee to Act: Cognitive-Physical Reinforcement Learning for Autonomous Driving

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    CoPhy distills VLM knowledge into a BEV encoder and uses an action-conditioned auto-regressive BEV world model inside GRPO with dual physical-cognitive rewards to reach SOTA on NAVSIM v1/v2 while adding language-based...

  24. MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving

    cs.RO 2026-05 unverdicted novelty 6.0 of 10

    MAPLE performs closed-loop multi-agent training of VLA driving models entirely in latent space using supervised fine-tuning followed by RL with safety, progress, and diversity rewards, reaching SOTA on Bench2Drive.

  25. MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving

    cs.RO 2026-05 unverdicted novelty 6.0 of 10

    MAPLE proposes latent multi-agent rollouts with supervised fine-tuning followed by reinforcement learning using safety, progress, interaction, and diversity rewards to enable scalable closed-loop training for end-to-e...

  26. MindVLA-U1: VLA Beats VA with Unified Streaming Architecture for Autonomous Driving

    cs.RO 2026-05 unverdicted novelty 6.0 of 10

    MindVLA-U1 is the first unified streaming VLA architecture that surpasses human drivers on WOD-E2E planning metrics while matching VA latency and preserving language interfaces.

  27. CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Driving

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    CoWorld-VLA encodes world information into four expert tokens that condition a diffusion-based planner, yielding competitive collision avoidance and trajectory accuracy on the NAVSIM benchmark.

  28. CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Driving

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    CoWorld-VLA extracts semantic, geometric, dynamic, and trajectory expert tokens from multi-source supervision and feeds them into a diffusion-based hierarchical planner, achieving competitive collision avoidance and t...

  29. GuardAD: Safeguarding Autonomous Driving MLLMs via Markovian Safety Logic

    cs.AI 2026-05 unverdicted novelty 6.0 of 10

    GuardAD reduces accident rates by 32% in autonomous driving MLLMs by using n-th order Markovian logic to infer latent hazards and revise actions.

  30. DriveFuture: Future-Aware Latent World Models for Autonomous Driving

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    DriveFuture achieves SOTA results on NAVSIM by conditioning latent world model states on future predictions to directly inform trajectory planning.

  31. VECTOR-Drive: Tightly Coupled Vision-Language and Trajectory Expert Routing for End-to-End Autonomous Driving

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    VECTOR-DRIVE uses shared self-attention with semantic-aware expert routing of tokens to VL and trajectory experts plus flow-matching action decoding to reach 88.91 driving score on Bench2Drive.

  32. VLADriver-RAG: Retrieval-Augmented Vision-Language-Action Models for Autonomous Driving

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    VLADriver-RAG reaches a new state-of-the-art Driving Score of 89.12 on Bench2Drive by retrieving structure-aware historical knowledge through spatiotemporal semantic graphs and Graph-DTW alignment.

  33. ST-Prune: Training-Free Spatio-Temporal Token Pruning for Vision-Language Models in Autonomous Driving

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    ST-Prune is a training-free spatio-temporal token pruning framework for VLMs in autonomous driving that achieves near-lossless results at 90% token reduction by exploiting motion volatility, temporal recency, and mult...

  34. OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    OneDrive unifies heterogeneous decoding in a single VLM transformer decoder for end-to-end driving, achieving 0.28 L2 error and 0.18 collision rate on nuScenes plus 86.8 PDMS on NAVSIM.

  35. FeaXDrive: Feasibility-aware Trajectory-Centric Diffusion Planning for End-to-End Autonomous Driving

    cs.RO 2026-04 unverdicted novelty 6.0 of 10

    FeaXDrive improves end-to-end autonomous driving by shifting diffusion planning to a trajectory-centric formulation with curvature-constrained training, drivable-area guidance, and GRPO post-training, yielding stronge...

  36. DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scale

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    DVGT-2 is a streaming vision-geometry-action model that jointly reconstructs dense 3D geometry and plans trajectories online, achieving better reconstruction than prior batch methods while transferring directly to pla...

  37. CausalVAD: De-confounding End-to-End Autonomous Driving via Causal Intervention

    cs.CV 2026-03 unverdicted novelty 6.0 of 10

    CausalVAD applies sparse causal intervention to remove spurious correlations from end-to-end autonomous driving models, reporting state-of-the-art planning accuracy and robustness on nuScenes.

  38. From Representational Complementarity to Dual Systems: Synergizing VLM and Vision-Only Backbones for End-to-End Driving

    cs.RO 2026-02 conditional novelty 6.0 of 10

    A VLM-based and a vision-only end-to-end planner are behaviorally complementary in a long tail of driving scenarios; selecting the better trajectory lifts NAVSIM PDMS from 90.80 to 92.10 at modest compute.

  39. Pseudo-Expert Regularized Offline RL for End-to-End Autonomous Driving in Photorealistic Closed-Loop Environments

    cs.RO 2025-12 conditional novelty 6.0 of 10

    Pseudo-expert regularized offline RL reduces collisions and improves route completion for camera-based driving models trained on fixed simulator datasets from nuScenes.

  40. SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving

    cs.CV 2025-12 conditional novelty 6.0 of 10

    SpaceDrive replaces textual coordinate tokens with shared 3D positional encodings in a VLM driving planner, achieving state-of-the-art open-loop planning on nuScenes and 78.02 Driving Score on Bench2Drive.

  41. SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving

    cs.CV 2025-12 conditional novelty 6.0 of 10

    SpaceDrive integrates 3D positional encodings derived from depth and ego-states into VLMs, replacing digit tokens to improve spatial reasoning and trajectory regression in autonomous driving.

  42. E3AD: An Emotion-Aware Vision-Language-Action Model for Human-Centric End-to-End Autonomous Driving

    cs.CV 2025-12 conditional novelty 6.0 of 10

    An emotion-aware vision-language-action driving model estimates VAD emotion from commands and uses it to improve grounding and waypoint planning.

  43. Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail

    cs.RO 2025-10 conditional novelty 6.0 of 10

    Alpamayo-R1 introduces a VLA model with a Chain of Causation dataset and multi-stage SFT-plus-RL training that reports 12% better planning accuracy and 35% fewer close encounters versus trajectory-only baselines in dr...

  44. CogDriver: Integrating Cognitive Inertia for Temporally Coherent Planning in Autonomous Driving

    cs.CV 2025-08 unverdicted novelty 6.0 of 10

    CogDriver-Agent with sparse temporal memory and spatiotemporal distillation on CogDriver-Data achieves 22% higher closed-loop Driving Score on Bench2Drive and 21% lower mean L2 error on nuScenes.

  45. IRL-VLA: Training an Vision-Language-Action Policy via Reward World Model

    cs.AI 2025-08 conditional novelty 6.0 of 10

    IRL-VLA fine-tunes a vision-language-action driving policy with PPO against a learned reward world model trained on NAVSIM's EPDMS metrics, reaching 74.9 EPDMS on navhard-real.

  46. RoboTron-Sim: Improving Real-World Driving via Simulated Hard-Case

    cs.RO 2025-08 conditional novelty 6.0 of 10

    A simulation-to-real pipeline (HASS synthetic hard cases, scenario-aware prompts, and an image-to-ego geometry encoder) improves an MLLM's open-loop planning on nuScenes, especially in hard scenarios.

  47. Box-QAymo: Box-Referring VQA Dataset for Autonomous Driving

    cs.CV 2025-07 conditional novelty 6.0 of 10

    Box-QAymo introduces a box-referring VQA benchmark for autonomous driving, with hierarchical binary, attribute, and motion reasoning questions built from Waymo data and crowd-sourced labels.

  48. AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning

    cs.CV 2025-06 unverdicted novelty 6.0 of 10

    AutoVLA unifies semantic reasoning and trajectory planning in one autoregressive VLA model for end-to-end autonomous driving by tokenizing trajectories into discrete actions and using GRPO reinforcement fine-tuning to...

  49. ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generation

    cs.CV 2025-03 unverdicted novelty 6.0 of 10

    ORION reports 77.74 Driving Score and 54.62% Success Rate on Bench2Drive, outperforming prior end-to-end methods by 14.28 DS and 19.61% SR through unified VQA and planning optimization.

  50. CLEAR: Closed-Loop Reinforcement Learning at Scale for End-to-End Autonomous Driving

    cs.RO 2026-07 conditional novelty 5.5 of 10

    Residual waypoint RL around a frozen VLA prior, scaled via heterogeneous CARLA/H100 infrastructure, raises closed-loop driving score and success rate on longest6 v2 and Bench2Drive.

  51. MVPruner: Dynamic Token Pruning for Accelerating Multi-view Vision-Language Models in Autonomous Driving

    cs.CV 2026-06 unverdicted novelty 5.0 of 10

    MVPruner is a two-stage adaptive token pruning technique for multi-view VLMs that achieves 87.3% FLOPs reduction and 4.97x prefilling speedup while retaining 98.5% accuracy on DriveLM.

  52. DriveReward: A Comprehensive Dataset and Generative Vision-Language Reward Model for Autonomous Driving

    cs.CV 2026-06 unverdicted novelty 5.0 of 10

    Creates DriveReward dataset with counterfactual annotations and a 1B VLM reward model that outperforms larger VLMs on driving tasks and matches rule-based rewards in RL and trajectory scoring.

  53. Decision-Making with Lightweight Confidence-Aware Language Model for Autonomous Driving

    cs.RO 2026-05 unverdicted novelty 5.0 of 10

    Lightweight confidence-aware LM distilled from multi-agent CoT demonstrations achieves SOTA success rates on nuPlan benchmark for AD decision-making with low inference latency.

  54. LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Model

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    LVDrive improves closed-loop driving on Bench2Drive by adding latent future scene prediction to VLA models via unified embedding space processing and two-stage trajectory decoding.

  55. Distill to Think, Foresee to Act: Cognitive-Physical Reinforcement Learning for Autonomous Driving

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    CoPhy is a new RL framework that distills VLM cognition into BEV encoders, adds an auto-regressive BEV world model for action-conditioned future prediction, and optimizes policies via GRPO with dual physical-cognitive...

  56. SafeAlign-VLA: A Negative-Enhanced Safe Alignment Framework for Risk-Aware Autonomous Driving

    cs.RO 2026-05 unverdicted novelty 5.0 of 10

    SafeAlign-VLA uses counterfactual safety pairing and anchor-based group relative policy optimization to incorporate negative data for safer VLA-based autonomous driving.

  57. C-CoT: Counterfactual Chain-of-Thought with Vision-Language Models for Safe Autonomous Driving

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    C-CoT applies VLMs to autonomous driving via five-stage reasoning with a meta-action tree for counterfactuals, yielding 81.9% risk recall, 3.52% collision rate, and 1.98 m L2 error on a new dataset.

  58. Latency Analysis and Optimization of Alpamayo 1 via Efficient Trajectory Generation

    cs.AI 2026-05 unverdicted novelty 5.0 of 10

    Redesigning Alpamayo 1 to single-reasoning and optimizing diffusion action generation cuts inference latency by 69.23% while preserving trajectory diversity and prediction quality.

  59. VLADriver-RAG: Retrieval-Augmented Vision-Language-Action Models for Autonomous Driving

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    VLADriver-RAG achieves state-of-the-art performance on Bench2Drive by grounding VLA planning in structure-aware retrieved priors via spatiotemporal semantic graphs and Graph-DTW alignment.

  60. SpanVLA: Efficient Action Bridging and Learning from Negative-Recovery Samples for Vision-Language-Action Model

    cs.CV 2026-04 unverdicted novelty 5.0 of 10

    SpanVLA reduces action generation latency via flow-matching conditioned on history and improves robustness by training on negative-recovery samples with GRPO and a dedicated reasoning dataset.

Reference graph

Works this paper leans on

73 extracted references · 73 canonical work pages · cited by 60 Pith papers

  1. [1]

    Detr3d: 3d object detection from multi-view images via 3d-to-2d queries,

    Y . Wang, V . C. Guizilini, T. Zhang, Y . Wang, H. Zhao, and J. Solomon, “Detr3d: 3d object detection from multi-view images via 3d-to-2d queries,” in CoRL, 2022

    work page 2022

  2. [2]

    Planning-oriented autonomous driving,

    Y . Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wang et al., “Planning-oriented autonomous driving,” in CVPR, 2023

    work page 2023

  3. [3]

    Vad: Vectorized scene representation for efficient autonomous driving,

    B. Jiang, S. Chen, Q. Xu, B. Liao, J. Chen, H. Zhou, Q. Zhang, W. Liu, C. Huang, and X. Wang, “Vad: Vectorized scene representation for efficient autonomous driving,” in ICCV, 2023

    work page 2023

  4. [4]

    Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d,

    J. Philion and S. Fidler, “Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d,” in ECCV, 2020

    work page 2020

  5. [5]

    BEVFormer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers,

    Z. Li, W. Wang, H. Li, E. Xie, C. Sima, T. Lu, Q. Yu, and J. Dai, “Bev- former: Learning bird’s-eye-view representation from multi-camera im- ages via spatiotemporal transformers,” arXiv preprint arXiv:2203.17270 , 2022

    work page arXiv 2022

  6. [6]

    MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction

    B. Liao, S. Chen, X. Wang, T. Cheng, Q. Zhang, W. Liu, and C. Huang, “Maptr: Structured modeling and learning for online vectorized hd map construction,” arXiv preprint arXiv:2208.14437 , 2022

    work page Pith review arXiv 2022

  7. [7]

    Multipath: Multiple probabilistic anchor trajectory hypotheses for behavior prediction.arXiv preprint arXiv:1910.05449, 2019

    Y . Chai, B. Sapp, M. Bansal, and D. Anguelov, “Multipath: Multiple probabilistic anchor trajectory hypotheses for behavior prediction,” arXiv preprint arXiv:1910.05449, 2019

    work page arXiv 1910

  8. [8]

    Vip3d: End-to- end visual trajectory prediction via 3d agent queries

    J. Gu, C. Hu, T. Zhang, X. Chen, Y . Wang, Y . Wang, and H. Zhao, “Vip3d: End-to-end visual trajectory prediction via 3d agent queries,” arXiv preprint arXiv:2208.01582 , 2022

    work page arXiv 2022

Show all 73 references
  1. [9]

    Perceive, interact, predict: Learning dynamic and static clues for end-to-end motion prediction,

    B. Jiang, S. Chen, X. Wang, B. Liao, T. Cheng, J. Chen, H. Zhou, Q. Zhang, W. Liu, and C. Huang, “Perceive, interact, predict: Learning dynamic and static clues for end-to-end motion prediction,” arXiv preprint arXiv:2212.02181, 2022

    2022

  2. [10]

    End-to-end model-free reinforcement learning for urban driving using implicit affordances,

    M. Toromanoff, E. Wirbel, and F. Moutarde, “End-to-end model-free reinforcement learning for urban driving using implicit affordances,” in CVPR, 2020

    2020

  3. [11]

    Multi-modal fusion transformer for end-to-end autonomous driving,

    A. Prakash, K. Chitta, and A. Geiger, “Multi-modal fusion transformer for end-to-end autonomous driving,” in CVPR, 2021

    2021

  4. [12]

    St-p3: End-to-end vision-based autonomous driving via spatial-temporal feature learning,

    S. Hu, L. Chen, P. Wu, H. Li, J. Yan, and D. Tao, “St-p3: End-to-end vision-based autonomous driving via spatial-temporal feature learning,” in ECCV, 2022

    2022

  5. [13]

    Visual instruction tuning,

    H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual instruction tuning,” in NeurIPS, 2024. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, OCTOBER 2024 10 Scene Description The image shows a daytime urban driving scene with clear visibility. There are several cyclists on the road and ...

    2024

  6. [14]

    Qwen-vl: A frontier large vision-language model with versatile abilities,

    J. Bai, S. Bai, S. Yang, S. Wang, S. Tan, P. Wang, J. Lin, C. Zhou, and J. Zhou, “Qwen-vl: A frontier large vision-language model with versatile abilities,” arXiv preprint arXiv:2308.12966 , 2023

    2023 arXiv

  7. [15]

    Cogvlm: Visual expert for pretrained language models,

    W. Wang, Q. Lv, W. Yu, W. Hong, J. Qi, Y . Wang, J. Ji, Z. Yang, L. Zhao, X. Song et al., “Cogvlm: Visual expert for pretrained language models,” arXiv preprint arXiv:2311.03079 , 2023

    2023

  8. [16]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks,

    Z. Chen, J. Wu, W. Wang, W. Su, G. Chen, S. Xing, M. Zhong, Q. Zhang, X. Zhu, L. Lu et al., “Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks,” in CVPR, 2024

    2024

  9. [17]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al. , “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774 , 2023

    2023 arXiv

  10. [18]

    Hierarchical models of behavior and prefrontal function,

    M. M. Botvinick, “Hierarchical models of behavior and prefrontal function,” Trends in cognitive sciences , 2008

    2008

  11. [19]

    The architecture of cognitive control in the human prefrontal cortex,

    E. Koechlin, C. Ody, and F. Kouneiher, “The architecture of cognitive control in the human prefrontal cortex,” Science, 2003

    2003

  12. [20]

    Cognitive control, hierarchy, and the rostro–caudal organiza- tion of the frontal lobes,

    D. Badre, “Cognitive control, hierarchy, and the rostro–caudal organiza- tion of the frontal lobes,” Trends in cognitive sciences , 2008

    2008

  13. [21]

    Drivegpt4: Interpretable end-to-end autonomous driving via large language model,

    Z. Xu, Y . Zhang, E. Xie, Z. Zhao, Y . Guo, K. K. Wong, Z. Li, and H. Zhao, “Drivegpt4: Interpretable end-to-end autonomous driving via large language model,” arXiv preprint arXiv:2310.01412 , 2023

    2023

  14. [22]

    Driving with llms: Fusing object- level vector modality for explainable autonomous driving,

    L. Chen, O. Sinavski, J. Hünermann, A. Karnsund, A. J. Willmott, D. Birch, D. Maund, and J. Shotton, “Driving with llms: Fusing object- level vector modality for explainable autonomous driving,” arXiv preprint arXiv:2310.01957, 2023

    2023

  15. [23]

    Drivemlm: Aligning multi-modal large language models with behavioral planning states for autonomous driving,

    W. Wang, J. Xie, C. Hu, H. Zou, J. Fan, W. Tong, Y . Wen, S. Wu, H. Deng, Z. Li et al., “Drivemlm: Aligning multi-modal large language models with behavioral planning states for autonomous driving,” arXiv preprint arXiv:2312.09245, 2023

    2023

  16. [24]

    Omnidrive: A holistic llm-agent framework for autonomous driving with 3d perception, reasoning and planning,

    S. Wang, Z. Yu, X. Jiang, S. Lan, M. Shi, N. Chang, J. Kautz, Y . Li, and J. M. Alvarez, “Omnidrive: A holistic llm-agent framework for autonomous driving with 3d perception, reasoning and planning,” arXiv preprint arXiv:2405.01533, 2024

    2024

  17. [25]

    Mathematical capabilities of chatgpt,

    S. Frieder, L. Pinchetti, R.-R. Griffiths, T. Salvatori, T. Lukasiewicz, P. Petersen, and J. Berner, “Mathematical capabilities of chatgpt,” in NeurIPS, 2024

    2024

  18. [26]

    Measuring mathematical problem solving with the math dataset,

    D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt, “Measuring mathematical problem solving with the math dataset,” arXiv preprint arXiv:2103.03874 , 2021

    2021 arXiv

  19. [27]

    Drivevlm: The convergence of autonomous driving and large vision-language models,

    X. Tian, J. Gu, B. Li, Y . Liu, C. Hu, Y . Wang, K. Zhan, P. Jia, X. Lang, and H. Zhao, “Drivevlm: The convergence of autonomous driving and large vision-language models,” arXiv preprint arXiv:2402.12289 , 2024

    2024 arXiv

  20. [28]

    Vila: On pre-training for visual language models,

    J. Lin, H. Yin, W. Ping, P. Molchanov, M. Shoeybi, and S. Han, “Vila: On pre-training for visual language models,” in CVPR, 2024

    2024

  21. [29]

    Embodied understanding of driving scenarios,

    Y . Zhou, L. Huang, Q. Bu, J. Zeng, T. Li, H. Qiu, H. Zhu, M. Guo, Y . Qiao, and H. Li, “Embodied understanding of driving scenarios,”arXiv preprint arXiv:2403.04593, 2024

    2024

  22. [30]

    Improved baselines with visual instruction tuning,

    H. Liu, C. Li, Y . Li, and Y . J. Lee, “Improved baselines with visual instruction tuning,” in CVPR, 2024

    2024

  23. [31]

    Drivelm: Driving with graph visual question answering,

    C. Sima, K. Renz, K. Chitta, L. Chen, H. Zhang, C. Xie, P. Luo, A. Geiger, and H. Li, “Drivelm: Driving with graph visual question answering,” arXiv preprint arXiv:2312.14150 , 2023

    2023

  24. [32]

    Language prompt for autonomous driving,

    D. Wu, W. Han, T. Wang, Y . Liu, X. Zhang, and J. Shen, “Language prompt for autonomous driving,” arXiv preprint arXiv:2309.04379, 2023

    2023

  25. [33]

    Nuscenes-qa: A multi-modal visual question answering benchmark for autonomous driving scenario,

    T. Qian, J. Chen, L. Zhuo, Y . Jiao, and Y .-G. Jiang, “Nuscenes-qa: A multi-modal visual question answering benchmark for autonomous driving scenario,” in AAAI, 2024

    2024

  26. [34]

    nuscenes: A multimodal dataset for autonomous driving,

    H. Caesar, V . Bankiti, A. H. Lang, S. V ora, V . E. Liong, Q. Xu, A. Krishnan, Y . Pan, G. Baldan, and O. Beijbom, “nuscenes: A multimodal dataset for autonomous driving,” in CVPR, 2020

    2020

  27. [35]

    A survey of motion planning and control techniques for self-driving urban vehicles,

    B. Paden, M. ˇCáp, S. Z. Yong, D. Yershov, and E. Frazzoli, “A survey of motion planning and control techniques for self-driving urban vehicles,” IEEE Transactions on intelligent vehicles , 2016

    2016

  28. [36]

    Stanley: The robot that won the darpa grand challenge,

    S. Thrun, M. Montemerlo, H. Dahlkamp, D. Stavens, A. Aron, J. Diebel, P. Fong, J. Gale, M. Halpenny, G. Hoffmann et al., “Stanley: The robot that won the darpa grand challenge,” Journal of field Robotics , 2006

    2006

  29. [37]

    Autonomous driving in urban environments: Boss and the urban challenge,

    C. Urmson, J. Anhalt, D. Bagnell, C. Baker, R. Bittner, M. Clark, J. Dolan, D. Duggins, T. Galatali, C. Geyer et al., “Autonomous driving in urban environments: Boss and the urban challenge,” Journal of field Robotics , 2008

    2008

  30. [38]

    Exploring the limitations of behavior cloning for autonomous driving,

    F. Codevilla, E. Santana, A. M. López, and A. Gaidon, “Exploring the limitations of behavior cloning for autonomous driving,” in ICCV, 2019

    2019

  31. [39]

    Alvinn: An autonomous land vehicle in a neural network,

    D. A. Pomerleau, “Alvinn: An autonomous land vehicle in a neural network,” in NeurIPS, 1988

    1988

  32. [40]

    End-to-end urban driving by imitating a reinforcement learning coach,

    Z. Zhang, A. Liniger, D. Dai, F. Yu, and L. Van Gool, “End-to-end urban driving by imitating a reinforcement learning coach,” in ICCV, 2021

    2021

  33. [41]

    Driveadapter: Breaking the coupling barrier of perception and planning in end-to-end autonomous driving,

    X. Jia, Y . Gao, L. Chen, J. Yan, P. L. Liu, and H. Li, “Driveadapter: Breaking the coupling barrier of perception and planning in end-to-end autonomous driving,” in ICCV, 2023

    2023

  34. [42]

    Maptrv2: An end-to-end framework for online vectorized hd map construction,

    B. Liao, S. Chen, Y . Zhang, B. Jiang, Q. Zhang, W. Liu, C. Huang, and X. Wang, “Maptrv2: An end-to-end framework for online vectorized hd map construction,” arXiv preprint arXiv:2308.05736 , 2023

    2023

  35. [43]

    Vadv2: End-to-end vectorized autonomous driving via probabilistic planning,

    S. Chen, B. Jiang, H. Gao, B. Liao, Q. Xu, Q. Zhang, C. Huang, W. Liu, and X. Wang, “Vadv2: End-to-end vectorized autonomous driving via probabilistic planning,” arXiv preprint arXiv:2402.13243 , 2024

    2024 arXiv

  36. [44]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in NeurIPS, 2017

    2017

  37. [45]

    Language models are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language models are few-shot learners,” in NeurIPS, 2020

    2020

  38. [46]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al., “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971 , 2023

    2023 arXiv

  39. [47]

    Palm 2 technical report,

    R. Anil, A. M. Dai, O. Firat, M. Johnson, D. Lepikhin, A. Passos, S. Shakeri, E. Taropa, P. Bailey, Z. Chen et al., “Palm 2 technical report,” arXiv preprint arXiv:2305.10403 , 2023

    2023 arXiv

  40. [48]

    Qwen2 technical report,

    A. Yang, B. Yang, B. Hui, B. Zheng, B. Yu, C. Zhou, C. Li, C. Li, D. Liu, F. Huang et al. , “Qwen2 technical report,” arXiv preprint arXiv:2407.10671, 2024

    2024 arXiv

  41. [49]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models,

    D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny, “Minigpt-4: Enhancing vision-language understanding with advanced large language models,” arXiv preprint arXiv:2304.10592 , 2023

    2023 arXiv

  42. [50]

    Instructblip: Towards general-purpose vision-language models with instruction tuning,

    W. Dai, J. Li, D. Li, A. M. H. Tiong, J. Zhao, W. Wang, B. Li, P. Fung, and S. Hoi, “Instructblip: Towards general-purpose vision-language models with instruction tuning,” 2023

    2023

  43. [51]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in ICML, 2021

    2021

  44. [52]

    Eva: Exploring the limits of masked visual representation learning at scale,

    Y . Fang, W. Wang, B. Xie, Q. Sun, L. Wu, X. Wang, T. Huang, X. Wang, and Y . Cao, “Eva: Exploring the limits of masked visual representation learning at scale,” in CVPR, 2023

    2023

  45. [53]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation,

    J. Li, D. Li, C. Xiong, and S. Hoi, “Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation,” in ICML, 2022

    2022

  46. [54]

    Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models,” in ICML, 2023

    2023

  47. [55]

    Scaling up visual and vision-language representation learning with noisy text supervision,

    C. Jia, Y . Yang, Y . Xia, Y .-T. Chen, Z. Parekh, H. Pham, Q. Le, Y .-H. Sung, Z. Li, and T. Duerig, “Scaling up visual and vision-language representation learning with noisy text supervision,” in ICML, 2021

    2021

  48. [56]

    Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks,

    J. Lu, D. Batra, D. Parikh, and S. Lee, “Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks,” in NeurIPS, 2019

    2019

  49. [57]

    Lxmert: Learning cross-modality encoder representations from transformers,

    H. Tan and M. Bansal, “Lxmert: Learning cross-modality encoder representations from transformers,” arXiv preprint arXiv:1908.07490 , 2019

    1908

  50. [58]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution,

    P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge et al., “Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution,” arXiv preprint arXiv:2409.12191 , 2024

    2024 arXiv

  51. [59]

    Languagempc: Large language models as deci- sion makers for autonomous driving,

    H. Sha, Y . Mu, Y . Jiang, L. Chen, C. Xu, P. Luo, S. E. Li, M. Tomizuka, W. Zhan, and M. Ding, “Languagempc: Large language models as deci- sion makers for autonomous driving,” arXiv preprint arXiv:2310.03026 , 2023

    2023

  52. [60]

    Carla: An open urban driving simulator,

    A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V . Koltun, “Carla: An open urban driving simulator,” in CoRL, 2017

    2017

  53. [61]

    Referring multi-object tracking,

    D. Wu, W. Han, T. Wang, X. Dong, X. Zhang, and J. Shen, “Referring multi-object tracking,” in CVPR, 2023

    2023

  54. [62]

    Talk2car: Taking control of your self-driving car,

    T. Deruyttere, S. Vandenhende, D. Grujicic, L. Van Gool, and M.-F. Moens, “Talk2car: Taking control of your self-driving car,”arXiv preprint arXiv:1909.10838, 2019

    1909

  55. [63]

    Textual explanations for self-driving vehicles,

    J. Kim, A. Rohrbach, T. Darrell, J. Canny, and Z. Akata, “Textual explanations for self-driving vehicles,” in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 563–578

    2018

  56. [64]

    Judging llm-as-a-judge with mt-bench and chatbot arena,

    L. Zheng, W.-L. Chiang, Y . Sheng, S. Zhuang, Z. Wu, Y . Zhuang, Z. Lin, Z. Li, D. Li, E. Xing et al., “Judging llm-as-a-judge with mt-bench and chatbot arena,” in NeurIPS, 2023

    2023

  57. [65]

    Gaussian error linear units (gelus),

    D. Hendrycks and K. Gimpel, “Gaussian error linear units (gelus),” arXiv preprint arXiv:1606.08415, 2016

    2016 arXiv

  58. [66]

    Maximum margin planning,

    N. D. Ratliff, J. A. Bagnell, and M. A. Zinkevich, “Maximum margin planning,” in Proceedings of the 23rd international conference on Machine learning, 2006, pp. 729–736. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, OCTOBER 2024 13

    2006

  59. [67]

    End-to-end interpretable neural motion planner,

    W. Zeng, W. Luo, S. Suo, A. Sadat, B. Yang, S. Casas, and R. Urtasun, “End-to-end interpretable neural motion planner,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 8660–8669

    2019

  60. [68]

    Safe local motion planning with self-supervised freespace forecasting,

    P. Hu, A. Huang, J. Dolan, D. Held, and D. Ramanan, “Safe local motion planning with self-supervised freespace forecasting,” in CVPR, 2021

    2021

  61. [69]

    Differentiable raycasting for self-supervised occupancy forecasting,

    T. Khurana, P. Hu, A. Dave, J. Ziglar, D. Held, and D. Ramanan, “Differentiable raycasting for self-supervised occupancy forecasting,” in ECCV, 2022

    2022

  62. [70]

    Bleu: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in ACL, 2002

    2002

  63. [71]

    Cider: Consensus- based image description evaluation,

    R. Vedantam, C. Lawrence Zitnick, and D. Parikh, “Cider: Consensus- based image description evaluation,” in CVPR, 2015

    2015

  64. [72]

    Meteor: An automatic metric for mt evaluation with improved correlation with human judgments,

    S. Banerjee and A. Lavie, “Meteor: An automatic metric for mt evaluation with improved correlation with human judgments,” in Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization , 2005

    2005

  65. [73]

    Is ego status all you need for open-loop end-to-end autonomous driving?

    Z. Li, Z. Yu, S. Lan, J. Li, J. Kautz, T. Lu, and J. M. Alvarez, “Is ego status all you need for open-loop end-to-end autonomous driving?” in CVPR, 2024. Bo Jiang received his B.E. degree in data science and big data technology from Central South University, China, in 2021. Cu...

    2024

Pith tools

Reviewed May 15, 2026 · model on record in the stance chip above.