Pith. sign in

REVIEW 4 major objections 6 minor 56 references

Letting action, image, and LiDAR tokens update each other in every block removes the cascaded perception bottleneck and plans without anchors.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 20:29 UTC pith:PQ2PMH6X

load-bearing objection Solid unified E2E driving system with real NAVSIM gains and released code; the “bottleneck removed” story is only partly isolated from the backbone upgrade. the 4 major comments →

arxiv 2607.23511 v1 pith:PQ2PMH6X submitted 2026-07-26 cs.CV

MOJITO: Modal Joint Learning for Unified End-to-End Autonomous Driving

Zhijing Cheng , Xuancheng Zhang , Donglin Di , Lei Fan , Baorui Ma , Hao Li , Xun Yang This is my paper
classification cs.CV
keywords Autonomous DrivingEnd-to-End PlanningMulti-modal Joint LearningModal Joint AttentionDiffusion TransformerLiDAR TokenizationNAVSIM
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Most end-to-end driving stacks still squeeze cameras and LiDAR into a compact context, then hand that summary to a separate planner. The authors argue that squeeze discards the fine geometric detail trajectory planning needs and also keeps modern vision transformers from helping the planner directly. MOJITO instead runs three parallel transformer branches—image, LiDAR, and a diffusion action planner—and, in every block, concatenates their tokens so a shared self-attention layer updates all three at once. Because the planner can query dense sensor tokens while it denoises waypoints, the system needs neither hand-crafted trajectory anchors nor auxiliary 3D detection losses. On the NAVSIM benchmarks this yields 88.9 PDMS and 88.4 EPDMS, ahead of recent cascaded and vision-language baselines, while remaining a compact 127M-parameter model that also follows high-level commands and produces diverse paths.

Core claim

A fully unified sensor-to-action architecture, built on block-wise Modal Joint Attention among image, LiDAR, and diffusion action tokens, can generate stable anchor-free trajectories without auxiliary supervised tasks and reaches state-of-the-art closed-loop scores on NAVSIM v1 and v2 by letting the planner read fine-grained multi-modal features at every layer instead of a compressed perception context.

What carries the argument

Modal Joint Attention: inside each transformer block the image, LiDAR, and action token sequences are concatenated, given shared positional embeddings, and passed through one multi-head self-attention layer so action tokens query dense sensor features while perception features adapt to the current planning state; the joint sequence is then split and fed to each branch’s own FFN.

Load-bearing premise

That higher NAVSIM non-reactive scores prove the joint attention truly preserves the fine sensor detail the authors say cascaded pipelines lose, rather than gains coming mainly from backbone choice or diffusion setup.

What would settle it

Train an otherwise identical MOJITO that freezes perception tokens from attending to action tokens (or replaces joint self-attention with one-way cross-attention from action to sensors only) and check whether PDMS/EPDMS and the qualitative failure modes collapse back toward strong cascaded baselines such as DiffusionDrive on the same navtest split.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • End-to-end planners can drop predefined trajectory anchors and auxiliary 3D box or map losses if action tokens continually attend to dense multi-modal features.
  • Standard ViT and DiT stacks with pretrained vision and 3D weights can be wired straight into planning without a separate compressed context interface.
  • Scaling aligned depth across image, LiDAR, and action branches should continue to lift closed-loop scores when more data are available.
  • High-level command conditioning plus diffusion sampling yields instruction-following and continuous trajectory diversity without a large vision-language backbone.
  • A metric-preserving pillar grid for LiDAR is required for joint attention to help; relative FPS+KNN tokenization does not.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If bidirectional perception–action attention is the real driver, the same joint-token pattern should transfer to other continuous control domains (manipulation, drone flight) that today still cascade a frozen encoder into a policy head.
  • The camera-only drop on navhard Stage II suggests joint attention still depends on complementary geometry; pure vision variants may need explicit depth or temporal cues to close that gap.
  • Because the action branch is trained from scratch while perception starts from foundation weights, further gains may come from co-pretraining the diffusion planner on large unlabeled trajectory corpora before NAVSIM fine-tuning.
  • Latency near 188 ms with two diffusion steps implies real-time use will hinge on fewer steps or distilled samplers rather than on shrinking the joint attention itself.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes MOJITO, a unified end-to-end driving architecture in which image tokens (DINOv3-S+), LiDAR tokens (Uni3D-S with a new "PillarGroup" BEV grid tokenizer), and diffusion action tokens are concatenated and processed by shared block-wise self-attention ("Modal Joint Attention"), eliminating the cascaded perception→planning interface, predefined anchors, and auxiliary supervision. The authors argue this removes an information bottleneck that discards fine-grained detail. Empirically, MOJITO reports 88.9 PDMS on NAVSIM-v1 navtest, 88.4 EPDMS on NAVSIM-v2 navtest, and 29.0 EPDMS on the navhard two-stage split (camera-only), each at or near the top of the respective tables, plus ablations over modality, tokenizer, and attention type, a depth-scaling study, attention visualizations, and latency numbers. Code and models are promised at a public repository.

Significance. If the results hold, the paper is a useful data point for the field: a compact (127M) anchor-free imitation-only model reaching 88.9/88.4 on NAVSIM-v1/v2 without RL, VLM backbones, or auxiliary tasks is a genuinely competitive and parameter-efficient result, and the anchor-free diffusion design with only 2 denoising steps at 187.65 ms latency is practically relevant. Strengths worth naming: head-to-head tables on both NAVSIM versions including the harder navhard two-stage protocol; an internal ablation (Table 5) that isolates attention type (cross- vs. self-attention) with all other factors fixed, showing a 3.2 PDMS gap in favor of joint attention; a scaling study with from-scratch-planner controls; a public code/model release; and a falsifiable architectural claim. The PillarGroup tokenizer with absolute metric scale is a simple, transferable contribution independent of the main thesis.

major comments (4)
  1. [Abstract; §5.3, Tables 1-2] Abstract and §5.3, Tables 1-2: the claim of 'setting a new state-of-the-art' is not supported by the paper's own tables. Table 2 lists ReCogDrive-Base-RL at 90.8, ReCogDrive-Large-RL at 90.4, AdaThinkDrive-RL at 90.3, and AutoVLA-RL at 89.1 — all above MOJITO's 88.9. The text in §5.3 is careful ('competitive score compared to VLM-RL-based methods'), but the abstract and §1 are not. The accurate claim — best among imitation-only, anchor-free, non-RL methods — is still strong and should be stated precisely in the abstract, introduction, and conclusion.
  2. [§1, §3.2, §5.6 Table 5] The central causal framing (§1, §3.2 Challenge 1) — that the PDMS gains come from removing the compressed-context bottleneck via joint attention — is confounded in all cross-method comparisons by the simultaneous backbone upgrade to pretrained DINOv3-S+/Uni3D-S, the anchor-free diffusion formulation, and the training recipe. Table 5 ID 1 sharpens this: camera-only MOJITO scores 86.8, below the cascaded DiffusionDrive (88.1) and WoTE (88.3), so the unified interface alone does not dominate cascaded designs when the modality set is held to camera. Table 5 ID 3 does isolate attention topology (85.7 cross-attn vs 88.9 self-attn, same backbones) and is the paper's best evidence for the mechanism — but cross-attention still lets action tokens query dense sensor tokens every block, so the 'compressed context' hypothesis itself is tested nowhere. A controlled comparison (e.g., a cascaded variant
  3. [§5.3-5.6, Tables 1, 3, 5] No variance estimates anywhere. The headline margin over WoTE is 0.6 PDMS (88.9 vs 88.3) and over DiffusionDrive 0.8, with single-run numbers and no seeds reported. NAVSIM PDMS is known to have non-trivial run-to-run variation under imitation training, and the ablation deltas in Table 5 (e.g., ID 2 at 86.1 vs ID 1 at 86.8) are interpreted directionally without any notion of noise. At minimum: report the main result and Table 5 over ≥3 seeds with std, or state explicitly the observed seed variance. This is a standard expectation for margins of this size.
  4. [§4.2, §5.6 Table 5] PillarGroup is one of the paper's load-bearing components (Table 5 ID 2 vs ID 4 attributes a 2.8 PDMS gap to tokenization), yet it is specified only as 'a fixed grid (e.g., 32×32 pillars), covering a specific metric range', top-K=512, N=64. The metric crop range, the ranking criterion in step C of Fig. 2, padding behavior, and z/height handling are unspecified, and none of (grid size, K, N) is ablated. Since the FPS+KNN degradation is attributed entirely to tokenization while backbone, point budget, and density also differ between ID 2 and ID 4, the paper should (i) fully specify PillarGroup and (ii) add at least a small sensitivity sweep over grid resolution and K, or soften the attribution.
minor comments (6)
  1. [§4.3, Eq. (2)] Eq. (2): the network is denoted εθ but the regression target is x(0) — this is x0-prediction, not ε-prediction; the notation is misleading. Also k is called a 'diffusion step' yet sampled as k∼U(0,1); please clarify continuous vs discrete time and state the sampler and the number of inference steps (2 steps appears only in §7, with no sensitivity to step count).
  2. [§5.5 Fig. 3; §6 Fig. 5] Fig. 3 caption says 'the red line represents the trajectory generated by DiffusionDrive, ReCogDrive, and MOJITO, respectively' — three methods cannot share one color; the figure needs distinct colors/labels per method. Fig. 5's attention maps are averaged over 8 waypoints and 3 camera views; specifying the averaging and normalization would aid interpretation.
  3. [§5.8, Fig. 4] §5.8: 'instruction following' is implemented by mapping language to a fixed NAVSIM command vocabulary via Qwen3-1.7B, i.e., the model itself is conditioned only on discrete commands. The framing in §5.8 and the contributions list should be tempered accordingly; a quantitative instruction-compliance metric would strengthen the claim.
  4. [§7] §7 latency: measured on a single H200 with batch size unspecified; please report batch size, whether backbone precompute is amortized, and the latency breakdown across branches, since the joint-attention sequence length scales with NI+NL+NA.
  5. [References; §5.2; Table 4] Reference and text glitches: [24] arXiv id '2512.107192' appears malformed; [55] author list contains 'Zhao, Seth Z.and Zhang' (missing space/formatting); §5.2 'follow Transfuser for image pre-processing' should specify exactly which augmentations/crops; Table 4 Stage I/II rows share one EPDMS column — clarify that EPDMS aggregates both stages.
  6. [§6] §6 is titled 'Cross Modal Learning' but contains only an attention-map visualization analysis; either rename (e.g., 'Attention Analysis') or add the promised analysis of cross-modal learning dynamics. Also consider noting that attention weights are not, by themselves, evidence of information flow — a sentence of caveat would suffice.

Circularity Check

0 steps flagged

No circularity: standard supervised diffusion training evaluated on external NAVSIM metrics against independent baselines.

full rationale

MOJITO’s load-bearing claims are architectural (block-wise Modal Joint Attention fusing image, LiDAR, and DiT action tokens) and empirical (88.9 PDMS / 88.4 EPDMS on NAVSIM v1/v2 navtest). The training objective (Eq. 2) is ordinary diffusion denoising of ground-truth waypoints; PDMS/EPDMS are community-defined closed-loop scores from the external NAVSIM benchmark, not quantities algebraically forced by the model’s own normalizations or fits. Baselines (DiffusionDrive, WoTE, Transfuser, VLA methods) are independent systems. Ablations (Table 5) and scaling (Table 6) vary design choices under the same external metric. Self-citations in the bibliography are peripheral (prior generative/video work) and do not underwrite uniqueness theorems, forced ansatze, or the SOTA numbers. Confounding of backbone choice with joint-attention topology is a causal-attribution concern, not circularity. Derivation chain is self-contained imitation learning plus external evaluation; score 0.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 2 invented entities

Load-bearing content is architectural and empirical. The claim rests on standard Transformer/diffusion machinery, pretrained vision/3D encoders used as initialization, NAVSIM’s non-reactive metric suite as a proxy for planning quality, and several hand-chosen tokenization and training hyperparameters. No physical constants; invented pieces are named modules rather than new ontological entities.

free parameters (4)
  • PillarGroup grid and sampling (e.g. 32×32 pillars, top-K=512, N=64 points/pillar) = K=512, N=64 (grid example 32×32)
    Hand-chosen LiDAR tokenization that defines the metric tokens the joint attention sees; ablation shows FPS+KNN alternative hurts, so results depend on this choice.
  • Model depth/width and training hyperparameters = 12 blocks, D=384, LR=6e-4, bs=512
    12 blocks, dim 384, AdamW LR 6e-4, batch 512, 8 H200s; scaling table shows PDMS highly sensitive to block count.
  • Diffusion inference steps and trajectory horizon = 2 steps, Tf=8
    Inference uses 2 diffusion steps and Tf=8 waypoints / 4s; latency and trajectory quality depend on these settings.
  • Camera crop/layout (3 views → 1024×256) = 1024×256 concatenated front/left/right
    Input geometry follows Transfuser-style preprocessing; performance is conditioned on this reduced multi-view setup rather than full 360° native resolution.
axioms (5)
  • ad hoc to paper Shared multi-head self-attention over concatenated image, LiDAR, and action tokens implements bidirectional perception–planning information flow without a compact context bottleneck.
    Core design thesis of §4.4 Modal Joint Attention; supported by ablation ID3 vs ID4 but not derived from a formal information-bottleneck bound.
  • domain assumption NAVSIM PDMS/EPDMS under non-reactive closed-loop simulation is an adequate primary measure of planning quality and ‘driving stability in complex scenarios.’
    Entire evaluation §5 relies on NAVSIM v1/v2 protocols; common in the subfield but still a proxy assumption for real interactive driving.
  • domain assumption Trajectory denoising loss alone (plus high-level command via adaLN) is sufficient supervision—no anchors or auxiliary 3D detection/map losses required for stable generation.
    Stated in §3.1 and §4.3; contrasts with cited baselines that use anchors/aux tasks.
  • domain assumption Pretrained DINOv3 and Uni3D representations remain useful when fine-tuned inside joint attention with a from-scratch DiT action branch.
    Initialization choice in §5.2; compatibility with foundation models is part of the contribution narrative in §1.
  • standard math Standard Transformer residual attention and diffusion ε-prediction training objectives behave as in the cited ViT/DiT literature.
    Equations (2)–(4) and architecture follow Peebles & Xie DiT / standard MHSA practice.
invented entities (2)
  • Modal Joint Attention (block-wise joint MHSA over image∥LiDAR∥action tokens) no independent evidence
    purpose: Replace cascaded context interface with per-block multi-modal token interaction for planning.
    Named core mechanism; empirically ablated but not an externally measured physical entity—architectural construct validated only inside this training setup.
  • PillarGroup BEV grid tokenizer for Uni3D-style LiDAR branch no independent evidence
    purpose: Produce metric-scale LiDAR tokens compatible with driving scenes, replacing FPS+KNN shape tokenization.
    Custom tokenization module; ablation favors it over FPS+KNN, but no external benchmark of the tokenizer alone outside MOJITO.

pith-pipeline@v1.2.0-grok45-kimik3 · 20455 in / 3945 out tokens · 92515 ms · 2026-07-30T20:29:55.871542+00:00 · methodology

0 comments
read the original abstract

End-to-end autonomous driving systems commonly follow a cascaded two-stage pipeline where a perception stage compresses multi-modal sensor inputs into a compact context and a downstream planner predicts trajectories conditioned on this context. We argue that this one-way perception-to-planning interface forces sensor inputs into a compact representation, losing the fine-grained details critical for planning. Moreover, by constraining the planner to this compressed context, it is difficult to leverage the rich representations offered by modern vision foundation models. To address these issues, we propose MOJITO, a unified sensor-to-action framework for end-to-end autonomous driving built on modal joint learning. MOJITO removes the cascaded interface and instead performs block-wise Modal Joint Attention that simultaneously updates action, image, and LiDAR features, allowing the planner to directly access multi-modal features during action generation. MOJITO achieves 88.9 PDMS on the NAVSIM v1 dataset and 88.4 EPDMS on the more challenging NAVSIM v2 dataset, setting a new state-of-the-art. Extensive experiments further demonstrate strong scalability, instruction following, and diverse trajectory generation. Code and models are available at https://github.com/mumucc01/MOJITO.

Figures

Figures reproduced from arXiv: 2607.23511 by Baorui Ma, Donglin Di, Hao Li, Lei Fan, Xuancheng Zhang, Xun Yang, Zhijing Cheng.

Figure 1
Figure 1. Figure 1: This figure outlines the core differences between a cascaded two-stage frame￾work and our unified E2E architecture. (a) shows that the “perception-to-planning” methods compress the fine-grained sensor information and suffer from the information bottleneck. (b) We propose a unified multimodal architecture comprising three parallel branches. Benefiting from the multi-modal fusion, MOJITO can generate feasibl… view at source ↗
Figure 2
Figure 2. Figure 2: MOJITO Architecture. To avoid the information bottleneck inherent in cas￾caded frameworks, we propose MOJITO, a unified multi-modal framework with three branches: Image, LiDAR, and the Action Planner branch. To preserve the 3D spatial understanding of the driving scene, we introduce a BEV Grid Patchify module named PillarGroup. The Modal Joint Attention mechanism enables the deep interaction be￾tween the p… view at source ↗
Figure 3
Figure 3. Figure 3: This figure presents a qualitative comparison between our method and existing methods on NAVSIM-v1. Our proposed MOJITO achieves more stable and reliable decision-making in complex scenarios. The green line denotes the ground-truth trajec￾tory, while the red line represents the trajectory generated by DiffusionDrive, ReCog￾Drive, and MOJITO, respectively. We additionally report results on NAVSIM-v2 navhard… view at source ↗
Figure 4
Figure 4. Figure 4: This figure demonstrates the instruction-following and diverse trajectory gen￾eration capability of MOJITO. (a) The blue, green, and red lines represent the tra￾jectories generated for the instructions “turn left”, “go straight”, and “turn right”, re￾spectively. (b) shows our ability to follow more complex instructions. Benefiting from our diffusion-based planner, our method can generate trajectories with … view at source ↗
Figure 5
Figure 5. Figure 5: Visualization of the trajectory-to-image attention maps. The attention weights are averaged over the 8 future waypoints to produce a holistic view of how the planning branch attends to the multi-view image patches. 7 Inference Latency In this section, we measure per-sample latency on a single H200 GPU using BF16 precision. MOJITO employs 2 diffusion steps and achieves an inference latency of 187.65 ms, whi… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

56 extracted references · 17 linked inside Pith

  1. [1]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Bi, H., Tan, H., Xie, S., Wang, Z., Huang, S., Liu, H., Zhao, R., Feng, Y., Xiang, C., Rong, Y., et al.: Motus: A Unified Latent Action World Model. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 35101–35113 (2026)

    2026

  2. [2]

    arXiv preprint arXiv:1604.07316 (2016)

    Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L.D., Monfort, M., Muller, U., Zhang, J., et al.: End to End Learning for Self-Driving Cars. arXiv preprint arXiv:1604.07316 (2016)

    Pith/arXiv arXiv 2016

  3. [3]

    arXiv preprint arXiv:2106.11810 (2021)

    Caesar, H., Kabzan, J., Tan, K.S., Fong, W.K., Wolff, E., Lang, A., Fletcher, L., Beijbom, O., Omari, S.: NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles. arXiv preprint arXiv:2106.11810 (2021)

    Pith/arXiv arXiv 2021

  4. [4]

    arXiv preprint arXiv:2402.13243 (2024)

    Chen, S., Jiang, B., Gao, H., Liao, B., Xu, Q., Zhang, Q., Huang, C., Liu, W., Wang, X.: VaDv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning. arXiv preprint arXiv:2402.13243 (2024)

    Pith/arXiv arXiv 2024

  5. [5]

    In: The Fourteenth International Conference on Learning Representations

    Cheng, Z., Zhang, X., Di, D., Wei, C., Li, H., Yang, X.: MoCa: Modeling Ob- ject Consistency for 3D Camera Control in Video Generation. In: The Fourteenth International Conference on Learning Representations

  6. [6]

    arXiv preprint arXiv:2605.05092 (2026)

    Chi, H., Qiu, D., Su, H., Liu, H., Li, Z., Zhang, H., Lv, C.: Driver-WM: A Driver- Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rollout. arXiv preprint arXiv:2605.05092 (2026)

    Pith/arXiv arXiv 2026

  7. [7]

    IEEE Transactions on Pattern Analysis and Machine Intelligence45(11), 12878–12895 (2022)

    Chitta, K., Prakash, A., Jaeger, B., Yu, Z., Renz, K., Geiger, A.: TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving. IEEE Transactions on Pattern Analysis and Machine Intelligence45(11), 12878–12895 (2022)

    2022

  8. [8]

    In: Proceedings of the Conference on Com- puter Vision and Pattern Recognition, Vancouver, Canada

    Contributors, O.: OpenScene: The Largest Up-to-Date 3D Occupancy Prediction Benchmark in Autonomous Driving. In: Proceedings of the Conference on Com- puter Vision and Pattern Recognition, Vancouver, Canada. pp. 18–22 (2023)

    2023

  9. [9]

    In: Ad- vances in Neural Information Processing Systems (2024)

    Dauner, D., Hallgarten, M., Li, T., Weng, X., Huang, Z., Yang, Z., Li, H., Gilitschenski, I., Ivanovic, B., Pavone, M., Geiger, A., Chitta, K.: NAVSIM: Data- Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking. In: Ad- vances in Neural Information Processing Systems (2024)

    2024

  10. [10]

    arXiv preprint arXiv:2505.14683 (2025)

    Deng, C., Zhu, D., Li, K., Gou, C., Li, F., Wang, Z., Zhong, S., Yu, W., Nie, X., Song, Z., Shi, G., Fan, H.: Emerging Properties in Unified Multimodal Pretraining. arXiv preprint arXiv:2505.14683 (2025)

    Pith/arXiv arXiv 2025

  11. [11]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Deng, T., Chen, X., Chen, Y., Chen, Q., Xu, Y., Yang, L., Xu, L., Zhang, Y., Zhang, B., Huang, W., et al.: GaussianDWM: 3D Gaussian Driving World Model for Unified Scene Understanding and Multi-Modal Generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 10656–10667 (2026)

    2026

  12. [12]

    arXiv preprint arXiv:2512.03422 (2025)

    Deng, T., Pan, Y., Yuan, S., Li, D., Wang, C., Li, M., Chen, L., Xie, L., Wang, D., Wang, J., et al.: What Is The Best 3D Scene Representation for Robotics? From Geometric to Foundation Models. arXiv preprint arXiv:2512.03422 (2025)

    arXiv 2025

  13. [13]

    In- ternational Journal of Computer Vision134(4), 152 (2026)

    Du, S., Zou, Y., Wang, Z., Li, X., Li, Y., Shang, C., Shen, Q.: Unsupervised Hy- perspectral Image Super-Resolution via Self-Supervised Modality Decoupling. In- ternational Journal of Computer Vision134(4), 152 (2026)

    2026

  14. [14]

    IEEE Transactions on Image Processing6(9), 1305–1315 (1997) MOJITO 17

    Eldar, Y., Lindenbaum, M., Porat, M., Zeevi, Y.Y.: The Farthest Point Strategy for Progressive Image Sampling. IEEE Transactions on Image Processing6(9), 1305–1315 (1997) MOJITO 17

    1997

  15. [15]

    IEEE Robotics and Automation Letters11(1), 226–233 (2025)

    Feng, R., Xi, N., Chu, D., Wang, R., Deng, Z., Wang, A., Lu, L., Wang, J., Huang, Y.: ARTEMIS: Autoregressive End-to-End Trajectory Planning with Mixture of Experts for Autonomous Driving. IEEE Robotics and Automation Letters11(1), 226–233 (2025)

    2025

  16. [16]

    On the Move to Meaningful Internet Systems

    Guo, G., Wang, H., Bell, D., Bi, Y., Greer, K.: KNN Model-based Approach in Classification. In: OTM Confederated International Conferences" On the Move to Meaningful Internet Systems". pp. 986–996. Springer (2003)

    2003

  17. [17]

    In: European Conference on Computer Vision

    Hu, S., Chen, L., Wu, P., Li, H., Yan, J., Tao, D.: ST-P3: End-to-end Vision- based Autonomous Driving via Spatial-Temporal Feature Learning. In: European Conference on Computer Vision. pp. 533–549. Springer (2022)

    2022

  18. [18]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Hu, Y., Yang, J., Chen, L., Li, K., Sima, C., Zhu, X., Chai, S., Du, S., Lin, T., Wang, W., et al.: Planning-oriented Autonomous Driving. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 17853– 17862 (2023)

    2023

  19. [19]

    arXiv preprint arXiv:2203.17054 (2022)

    Huang, J., Huang, G.: BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection. arXiv preprint arXiv:2203.17054 (2022)

    Pith/arXiv arXiv 2022

  20. [20]

    Jia, X., You, J., Zhang, Z., Yan, J.: DriveTransformer: Unified Transformer for Scalable End-to-EndAutonomous Driving.arXiv preprintarXiv:2503.07656(2025)

    Pith/arXiv arXiv 2025

  21. [21]

    arXiv preprint arXiv:2505.19381 (2025)

    Jiang, A., Gao, Y., Sun, Z., Wang, Y., Wang, J., Chai, J., Cao, Q., Heng, Y., Jiang, H., Dong, Y., et al.: DiffVLA: Vision-Language Guided Diffusion Planning for Autonomous Driving. arXiv preprint arXiv:2505.19381 (2025)

    Pith/arXiv arXiv 2025

  22. [22]

    In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Jiang, C., Cornman, A., Park, C., Sapp, B., Zhou, Y., Anguelov, D., et al.: Motion- Diffuser: Controllable Multi-Agent Motion Prediction using Diffusion. In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9644–9653 (2023)

    2023

  23. [23]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Li, B., Ouyang, W., Sheng, L., Zeng, X., Wang, X.: GS3D: An Efficient 3D Object Detection Framework for Autonomous Driving. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 1019–1028 (2019)

    2019

  24. [24]

    arXiv preprint arXiv:2512.107192(2025)

    Li, P., Zhang, Z., Holtz, D., Yu, H., Yang, Y., Lai, Y., Song, R., Geiger, A., Zell, A.: SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving. arXiv preprint arXiv:2512.107192(2025)

    Pith/arXiv arXiv 2025

  25. [25]

    Li, Y., Wang, Y., Liu, Y., He, J., Fan, L., Zhang, Z.: End-to-End Driving with On- lineTrajectoryEvaluationviaBevWorldModel.In:ProceedingsoftheIEEE/CVF International Conference on Computer Vision. pp. 27137–27146 (2025)

    2025

  26. [26]

    arXiv preprint arXiv:2506.08052 (2025)

    Li, Y., Xiong, K., Guo, X., Li, F., Yan, S., Xu, G., Zhou, L., Chen, L., Sun, H., Wang, B., et al.: ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving. arXiv preprint arXiv:2506.08052 (2025)

    Pith/arXiv arXiv 2025

  27. [27]

    arXiv preprint arXiv:2406.06978 (2024)

    Li, Z., Li, K., Wang, S., Lan, S., Yu, Z., Ji, Y., Li, Z., Zhu, Z., Kautz, J., Wu, Z., et al.: Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra- Distillation. arXiv preprint arXiv:2406.06978 (2024)

    Pith/arXiv arXiv 2024

  28. [28]

    arXiv preprint arXiv:2506.06664 (2025)

    Li, Z., Yao, W., Wang, Z., Sun, X., Chen, J., Chang, N., Shen, M., Wu, Z., Lan, S., Alvarez, J.M.: Generalized Trajectory Scoring for End-to-end Multimodal Plan- ning. arXiv preprint arXiv:2506.06664 (2025)

    Pith/arXiv arXiv 2025

  29. [29]

    IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)

    Li, Z., Wang, W., Li, H., Xie, E., Sima, C., Lu, T., Yu, Q., Dai, J.: BEVFormer: Learning Bird’s-Eye-View Representation from Lidar-Camera via Spatiotemporal Transformers. IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)

    2024

  30. [30]

    In: Proceedings of the Computer Vision and Pattern Recog- nition Conference

    Liao, B., Chen, S., Yin, H., Jiang, B., Wang, C., Yan, S., Zhang, X., Li, X., Zhang, Y., Zhang, Q., et al.: DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving. In: Proceedings of the Computer Vision and Pattern Recog- nition Conference. pp. 12037–12047 (2025) 18 Z. Cheng et al

    2025

  31. [31]

    Advances in Neural Information Processing Systems38, 14123–14151 (2026)

    Ling, H., Chen, Z., Chen, Q., Di, D., Ma, Y., Li, H., Wei, C., Tao, Z., Yang, X.: EverybodyDance: Bipartite Graph–Based Identity Correspondence for Multi- Character Animation. Advances in Neural Information Processing Systems38, 14123–14151 (2026)

    2026

  32. [32]

    arXiv preprint arXiv:2511.18729 (2025)

    Liu, L., Jia, C., Yu, G., Song, Z., Li, J., Jia, F., Wu, P., Hao, X., Luo, Y.: Guide- Flow: Constraint-Guided Flow Matching for Planning in End-to-End Autonomous Driving. arXiv preprint arXiv:2511.18729 (2025)

    arXiv 2025

  33. [33]

    arXiv preprint arXiv:2512.09864 (2025)

    Lu, H., Liu, Z., Jiang, G., Luo, Y., Chen, S., Zhang, Y., Chen, Y.C.: UniUGP: Unifying Understanding, Generation, and Planing for End-to-End Autonomous Driving. arXiv preprint arXiv:2512.09864 (2025)

    arXiv 2025

  34. [34]

    arXiv preprint arXiv:2509.13769 (2025)

    Luo, Y., Li, F., Xu, S., Lai, Z., Yang, L., Chen, Q., Luo, Z., Xie, Z., Jiang, S., Liu, J., et al.: AdaThinkDrive: Adaptive Thinking via Reinforcement Learning for Autonomous Driving. arXiv preprint arXiv:2509.13769 (2025)

    arXiv 2025

  35. [35]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Peebles, W., Xie, S.: Scalable Diffusion Models with Transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 4195–4205 (2023)

    2023

  36. [36]

    Siméoni, O., Vo, H.V., Seitzer, M., Baldassarre, F., Oquab, M., Jose, C., Khalidov, V., Szafraniec, M., Yi, S., Ramamonjisoa, M., Massa, F., Haziza, D., Wehrstedt, L., Wang, J., Darcet, T., Moutakanni, T., Sentana, L., Roberts, C., Vedaldi, A., Tolan, J., Brandt, J., Couprie, C., Mairal, J., Jégou, H., Labatut, P., Bojanowski, P.: DINOv3 (2025)

    2025

  37. [37]

    arXiv preprint arXiv:2602.13301 (2026)

    Su, H., Wu, W., Song, F., Zhang, J., Yang, Z., Yan, J.: DriveMamba: Task-Centric Scalable State Space Model for Efficient End-to-End Autonomous Driving. arXiv preprint arXiv:2602.13301 (2026)

    arXiv 2026

  38. [38]

    arXiv preprint arXiv:2405.19620 (2024)

    Sun, W., Lin, X., Shi, Y., Zhang, C., Wu, H., Zheng, S.: SparseDrive: End- to-End Autonomous Driving via Sparse Scene Representation. arXiv preprint arXiv:2405.19620 (2024)

    Pith/arXiv arXiv 2024

  39. [39]

    Tian, X., Gu, J., Li, B., Liu, Y., Wang, Y., Zhao, Z., Zhan, K., Jia, P., Lang, X., Zhao, H.: DriveVLM: The Convergence of Autonomous Driving and Large Vision- LanguageModels.In:ConferenceonRobotLearning.pp.4698–4726.PMLR(2025)

    2025

  40. [40]

    ACM Trans

    Tu, J., Ji, W., Zhao, H., Zhang, C., Zimmermann, R., Qian, H.: DriveDiTFit: Fine- tuning Diffusion Transformers for Autonomous Driving Data Generation. ACM Trans. Multim. Comput. Commun. Appl. pp. 85:1–85:29 (2025)

    2025

  41. [41]

    arXiv preprint arXiv:2503.20314 (2025)

    Wan,T.,Wang,A.,Ai,B.,Wen,B.,Mao,C.,Xie,C.W.,Chen,D.,Yu,F.,Zhao,H., Yang, J., et al.: Wan: Open and Advanced Large-Scale Video Generative Models. arXiv preprint arXiv:2503.20314 (2025)

    Pith/arXiv arXiv 2025

  42. [42]

    In: Proceedings of the computer vision and pattern recognition conference

    Wang, S., Yu, Z., Jiang, X., Lan, S., Shi, M., Chang, N., Kautz, J., Li, Y., Alvarez, J.M.: OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning. In: Proceedings of the computer vision and pattern recognition conference. pp. 22442–22452 (2025)

    2025

  43. [43]

    arXiv preprint arXiv:2503.12170 (2025)

    Wang, T., Zhang, C., Qu, X., Li, K., Liu, W., Huang, C.: DiffAD: A Unified Diffu- sion Modeling Approach for Autonomous Driving. arXiv preprint arXiv:2503.12170 (2025)

    Pith/arXiv arXiv 2025

  44. [44]

    Advances in Neural Information Processing Systems38, 140367–140387 (2026)

    Wang, X., Yang, X., Xu, Y., Wu, Y., Li, Z., Zhao, N.: AffordBot: 3D Fine-grained Embodied Reasoning via Multimodal Large Language Models. Advances in Neural Information Processing Systems38, 140367–140387 (2026)

    2026

  45. [45]

    arXiv preprint arXiv:2505.09388 (2025)

    Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., Zheng, C., Liu, D., Zhou, F., Huang, F., Hu, F., Ge, H., Wei, H., Lin, H., Tang, J., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Zhou, J., Lin, J., Dang, K., Bao, K., Yang, K., Yu, L., Deng, L., Li, M., Xue, M., Li, M., Zhang, MOJITO 19 P., Wang, ...

    Pith/arXiv arXiv 2025

  46. [46]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Yang, B., Su, H., Gkanatsios, N., Ke, T.W., Jain, A., Schneider, J., Fragki- adaki, K.: Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous and Instruction-guided Driving. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 15342–15353 (2024)

    2024

  47. [47]

    International Journal of Computer Vision132(11), 4823–4849 (2024)

    Yang, X., Chang, T., Zhang, T., Wang, S., Hong, R., Wang, M.: Learning Hi- erarchical Visual Transformation for Domain Generalizable Visual Matching and Recognition. International Journal of Computer Vision132(11), 4823–4849 (2024)

    2024

  48. [48]

    In: Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval

    Yang, X., Feng, F., Ji, W., Wang, M., Chua, T.S.: Deconfounded Video Moment Retrieval with Causal Intervention. In: Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval. pp. 1–10 (2021)

    2021

  49. [49]

    IEEE Transactions on Pattern Analysis and Machine Intelligence (2026)

    Yang, X., Pan, H., Wang, X., Cao, Y., Li, J., Wang, M.: FiNE: Fine-grained Neuron-level Model Editing for Reliable and Safe LLMs. IEEE Transactions on Pattern Analysis and Machine Intelligence (2026)

    2026

  50. [50]

    arXiv preprint arXiv:2506.06659 (2025)

    Yao, W., Li, Z., Lan, S., Wang, Z., Sun, X., Alvarez, J.M., Wu, Z.: DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planning. arXiv preprint arXiv:2506.06659 (2025)

    arXiv 2025

  51. [51]

    arXiv preprint arXiv:2408.03601 (2024)

    Yuan, C., Zhang, Z., Sun, J., Sun, S., Huang, Z., Lee, C.D.W., Li, D., Han, Y., Wong, A., Tee, K.P., et al.: DRAMA: An Efficient End-to-End Motion Planner for Autonomous Driving with Mamba. arXiv preprint arXiv:2408.03601 (2024)

    Pith/arXiv arXiv 2024

  52. [52]

    ACM Transactions on Intelligent Systems and Technology8(3), 1–19 (2017)

    Zhang, S., Li, X., Zong, M., Zhu, X., Cheng, D.: Learning K for Knn Classification. ACM Transactions on Intelligent Systems and Technology8(3), 1–19 (2017)

    2017

  53. [53]

    In: The Thirteenth International Conference on Learning Rep- resentations (2025)

    Zheng, Y., Liang, R., ZHENG, K., Zheng, J., Mao, L., Li, J., Gu, W., Ai, R., Li, S.E., Zhan, X., Liu, J.: Diffusion-Based Planning for Autonomous Driving with Flexible Guidance. In: The Thirteenth International Conference on Learning Rep- resentations (2025)

    2025

  54. [54]

    In: International Conference on Learning Rep- resentations (2024)

    Zhou, J., Wang, J., Ma, B., Liu, Y.S., Huang, T., Wang, X.: Uni3D: Exploring Unified 3D Representation at Scale. In: International Conference on Learning Rep- resentations (2024)

    2024

  55. [55]

    Advances in Neural Information Processing Systems (2025)

    Zhou, Z., Cai, T., Zhao, Seth Z.and Zhang, Y., Huang, Z., Zhou, B., Ma, J.: AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning. Advances in Neural Information Processing Systems (2025)

    2025

  56. [56]

    arXiv preprint arXiv:2512.07745 (2025)

    Zou, J., Chen, S., Liao, B., Zheng, Z., Song, Y., Zhang, L., Zhang, Q., Liu, W., Wang, X.: DiffusionDriveV2: Reinforcement Learning-Constrained Trun- cated Diffusion Modeling in End-to-End Autonomous Driving. arXiv preprint arXiv:2512.07745 (2025)

    arXiv 2025