Pith. sign in

REVIEW 4 major objections 6 minor 57 references

BrainWAM: Action-Space Coordination of Semantic Priors and Predictive Dynamics for Autonomous Driving

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read BrainWAM coordinates semantic reasoning and predictive world modeling at the action level, reaching 89.5 PDMS on NAVSIM v1 and 89.6 EPDMS on NAVSIM v2.

desk verdict A plausible systems paper with a useful diagnosis, but the missing ensemble baseline leaves the coordination claim unproven. read the letter →

arxiv 2608.12854 v1 pith:G656RU52 submitted 2026-08-13 cs.RO cs.AIcs.CV

Bing Zhan , Shuyao Shang , Jiahao Gu , Shuo Lu , Yuan Xu , Zhao Wang , Yida Wang , Xueyang Zhang
show 3 more authors
Kun Zhan Lue Fan Zhaoxiang Zhang
This is my paper · ORCID
classification cs.ROcs.AIcs.CV
keywords autonomousdrivingvision-language-actionmodelworldactionaction-spacecoordinationrectifiedflowtrajectoryplanningNAVSIMattentionallocation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that combining a vision-language action model and a world action model by simply mixing all their tokens in one attention space fails, because clean semantic tokens attract action-token attention away from noisier predictive video tokens. It proposes instead that each model first produce its own compact action representation, and that the two action streams be coordinated through a callosum-inspired bridge and a cerebellum-inspired fusion module. This action-space coordination, with an asynchronous denoising schedule for video and action, reaches 89.5 PDMS on NAVSIM v1 and 89.6 EPDMS on NAVSIM v2, outperforming both single-branch planners and the token-level fusion baseline. A sympathetic reader would conclude that specialized-but-coordinated pathways are a better architectural principle for end-to-end driving than a single shared representation.

What carries the argument

Callosal Action Bridge (CAB) is a set of gated bidirectional cross-attention modules inserted at two layers of the two frozen action experts, so prediction-grounded and semantic-grounded action tokens can exchange messages without mixing raw VLM and VGM tokens. Cerebellar Intent Fusion (CIF) is a two-layer transformer that concatenates the two refined action streams and averages them before decoding. Together they carry the claim that coordination at compact action representations, rather than at raw token attention, preserves each pathway's specialization; CAB and CIF are trained in a third stage with both branches frozen, and an asynchronous rectified-flow schedule decouples video denoising from action denoising.

What would settle it

Re-run Stage 3 after randomly permuting the order of action tokens in one frozen branch while keeping the other branch unchanged; if PDMS stays near 89.5, CAB and CIF are not genuinely aligning two specialized representations, and the claimed mechanism would not be the source of the gain.

Watch

Extended reading notes

Core claim

The central discovery is an attention-allocation mismatch in tri-modal joint attention: when VLM tokens, video-generation tokens, and action tokens share one attention space, action tokens attend much more strongly to the semantic VLM tokens than to the VGM tokens in most transformer layers, so the predictive-video signal is suppressed and the fused model underperforms a WAM-only baseline. BrainWAM's remedy is to let the VLA branch and the WAM branch each learn action representations from their own modality, semantic grounding versus future-scene prediction, then exchange information bidirectionally only at the action-token level through the Callosal Action Bridge, fuse the refined streams in the Cerebellar Intent Fusion module, and decode the fused representation into a trajectory with a rectified-flow action expert. The paper reports that this coordination yields 89.5 PDMS on NAVSIM v1 and 89.6 EPDMS on NAVSIM v2, with gains concentrated in drivable-area compliance and ego progress.

Load-bearing premise

The load-bearing premise is that the two separately-trained branches, once frozen, produce action-token spaces that are complementary and stable enough for CAB and CIF to align; the paper argues for this mainly through the freezing ablation in Appendix D, not through any direct representation-geometry analysis.

Editorial extensions

If this is right

  • Action-level coordination should be preferred over token-level fusion when combining a VLM-based planner with a generative world model; the paper's Tri-MoT baseline underperforms WAM-only, while BrainWAM surpasses both.
  • The asynchronous rectified-flow schedule lets one early video denoising step restore most of the predictive context, reaching 89.3 PDMS at 475 ms latency, so the efficiency-accuracy trade-off is favorable.
  • Because both branches are frozen in the joint stage, the coordination modules can in principle be retrained on top of upgraded VLM or video backbones without touching the large pretrained parameters.
  • Gains on NAVSIM v1 are concentrated in DAC and EP, meaning the combined planner improves drivable-area compliance and forward progress while safety metrics stay near saturation.
  • On NAVSIM v2 the same design tops 89.6 EPDMS under the stricter metric with extra rule-compliance penalties, so the action-space result transfers to the more comprehensive protocol.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The attention-allocation mismatch is likely a general phenomenon when a clean semantic encoder is combined with a noisy diffusion- or flow-based video encoder in one attention pool, so action-space coordination may help in other multimodal generative planning settings.
  • The success of the frozen-branch design depends on the action-token geometry of the specific checkpoints; testing Stage 3 with different random seeds or different backbone versions would reveal whether the coordination aligns stable, generalizable representations or overfits to these checkpoints.
  • The asynchronous schedule effectively uses the video generator as an early-exit feature extractor, which suggests that distilling its cached video features directly into the action decoder could remove the need to keep the video backbone in the loop.
  • CAB and CIF add only a small parameter overhead relative to the backbones, so the approach's value lies in the interaction rule rather than in added capacity; similar bridging could be tried for other pairs of specialized planners.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. BrainWAM is a closed-loop planning framework that combines a Vision-Language-Action (VLA) branch (Qwen3-VL-4B) and a World Action Model branch (Wan2.2-TI2V-5B) for autonomous driving. The two branches are trained independently in Stages 1 and 2 with rectified-flow objectives, then frozen in Stage 3 while a Callosal Action Bridge (CAB) and Cerebellar Intent Fusion (CIF) coordinate their action-token streams and decode a final trajectory. An asynchronous rectified-flow inference schedule lets the video stream stop after one or two steps while the action stream continues denoising. Against NAVSIM v1 and v2, the paper reports 89.5 PDMS and 89.6 EPDMS, outperforming VLA-only (86.1), WAM-only (88.1), and Tri-MoT (87.8) on NAVSIM v1. Ablations cover branch complementarity, the number of CAB blocks, CIF architecture and depth, and the Stage-3 freezing strategy.

Significance. The paper proposes a clean and well-motivated alternative to token-level fusion of VLAs and world models: rather than mixing raw VLM and video tokens, it lets each pathway produce action tokens and coordinates them through gated cross-attention (CAB) and a small Transformer fusion head (CIF). The three-stage training recipe is simple, the ablation set is broad and internally consistent, and the asynchronous video-denoising latency analysis (Table 5) is a useful contribution. If the reported numbers are reproducible, the framework is a practical way to bring semantic and predictive modalities together for driving. The main limitations are that all numbers are single runs, the decisive no-coordination ensemble baseline is missing, and the Tri-MoT comparison lacks construction details. The authors do not release code or weights, which weakens the SOTA claim until verified.

major comments (4)
  1. [Section 4.4, Table 3] The paper does not include a no-coordination ensemble baseline in which the two frozen branches are combined by simple averaging of their predicted action velocities or trajectories at inference. Such a baseline is cheap to construct because both branches are available during inference, and it directly tests whether the 1.4 PDMS gain of BrainWAM over WAM-only (89.5 vs 88.1) comes from CAB/CIF coordination or from error cancellation between two independent predictors. Please add this baseline with both uniform and learned-weight averaging to Table 3, and report the corresponding NAVSIM v2 metrics as well.
  2. [Section 4.3, Tables 1-9] All reported PDMS/EPDMS values appear to be from single runs, with no error bars, confidence intervals, or significance tests. The margins in some ablations are small (e.g., Table 5: 89.3 vs 89.5; Table 8: 89.0 vs 89.3), so without run-to-run variance the ranking of configurations and the SOTA claims are not statistically supported. Please provide at least three seeds for the main results and for the key ablations in Tables 3 and 4, or state the expected variance from prior NAVSIM experiments.
  3. [Section 4.4] The claim that Tri-MoT uses identical backbones and comparable parameter counts is not backed by implementation details. The manuscript gives no construction details for Tri-MoT: which backbone produces which token modality, what training schedule and loss weights are used, how many training steps, and what inference timesteps. Since Tri-MoT is the central negative baseline motivating the attention-allocation-mismatch claim, the paper must specify its exact setup and report parameter counts and FLOPs to make the comparison controlled.
  4. [Section 3.3 and Appendix D] The frozen-branch design assumes the two action-token spaces are sufficiently complementary and stable across checkpoints to be alignable by CAB/CIF. The freezing ablation (Table 9) shows that full fine-tuning is worse, but it does not demonstrate that specialization is preserved under coordination, and the results may be tied to the specific Stage-1/Stage-2 checkpoints selected. Please include an analysis of the action-token representations (e.g., cross-stream cosine similarity before and after CAB, or the effect of varying the Stage-1/Stage-2 training length) to support the generality of the coordination mechanism.
minor comments (6)
  1. [Section 4.1] The EPDMS equation is malformed; the product over M_pen and the weighted average are garbled. Please re-typeset it and define all symbols consistently.
  2. [Figure 2] The caption does not specify how the attention ratios are aggregated: which layers, heads, and how many scenes. Please clarify in the caption or in the main text.
  3. [Table 5] For the zero-video-step row, clarify whether the action stream still receives the observation conditioning features or whether it is a pure action-only model; the 79.3 PDMS drop should be interpreted relative to that configuration.
  4. [Abstract and Conclusion] The abstract and conclusion state that the method is practical, but Appendix G reports 475-644 ms inference latency, which the authors themselves note does not satisfy strict real-time requirements. Please soften the claim to indicate that deployment efficiency remains future work.
  5. [Section 3.3] The notation M(Z_pred, Z_sem) is used for element-wise averaging in Eq. (12), but M is also used for the set of penalty metrics in Section 4.1; please disambiguate.
  6. [Appendix B.1 and Eq. (10)] The appendix says each CAB contains two parallel multi-head cross-attention modules, while the main text uses Psi_l_cab for the same operation; please unify the notation between the main text and the appendix.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: training objectives are flow-matching losses on trajectory data, evaluation is on external NAVSIM benchmarks, and self-citations appear only as baselines.

full rationale

The claimed derivation runs from separately pretrained WAM and VLA branches to a frozen-branch Stage-3 coordination module trained with L_fuse = E||u_hat_fuse - u_a||^2, where u_a is the rectified-flow velocity target from expert trajectories; the headline numbers (89.5 PDMS / 89.6 EPDMS) come from NAVSIM, an external non-reactive simulator, not from any fitted constant in the paper. The only element-wise averaging in the architecture is inside CIF after learned projections and a Transformer, so the final fusion is not by construction identical to a simple ensemble of the two branches; the absence of an explicit averaged-trajectories baseline is a control gap, not a circular reduction. Self-citations such as DynVLA and DriveVLA-W0 appear as comparison baselines in Tables 1-2 and are not used to justify the CAB/CIF mechanism or the claimed state-of-the-art result. Architecture choices (CAB count, CIF depth, denoising steps) are tuned on the same benchmark, but this is standard hyperparameter selection and does not make the benchmark result an input to the derivation. No load-bearing step reduces to its own input, and no uniqueness theorem or ansatz is imported from prior author work.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper introduces two named architectural modules, CAB and CIF, but these are trainable components rather than postulated physical entities, so no graviton-style invented entities are present. The free parameters listed above are the design choices that were tuned on the same benchmark used for the final SOTA claim.

free parameters (5)
  • lambda_a_pred (action prediction loss weight in WAM branch) = not reported
    Appears in Eq. 6 as the weight on the action-vector-field loss; no value is given, and it controls how strongly WAM action tokens are grounded in future dynamics.
  • Number of CAB blocks and insertion layers = 2 blocks at layers 9 and 18
    Selected by ablation in Appendix B.2; performance saturates at 2 blocks, but the choice is tuned on NAVSIM v1.
  • Number of CIF Transformer layers = 2
    Selected by ablation in Appendix C.3; a third layer adds no gain.
  • Action and video denoising steps at inference = 3 action steps, 2 video steps
    Selected from Table 5 as a latency/accuracy trade-off; 1 video step gives 89.3 PDMS and 2 steps give 89.5.
  • Peak learning rate and training steps per stage = 5e-5 and 100K steps
    Standard tuned hyperparameters reported in Section 4.2; required to reproduce the training pipeline.
assumptions (5)
  • standard math Rectified flow defines a linear noising path x_t=(1-t)x0+t*eps and supervises velocity u=eps-x0.
    Used throughout Eqs. 2 to 5, 8, and 13 as the training objective for both branches and the fused decoder; this is standard flow-matching theory.
  • domain assumption NAVSIM PDMS and EPDMS scores are a valid proxy for real autonomous-driving planning quality.
    All conclusions about state-of-the-art performance rest on the benchmark's closed-loop simulator rather than on-road deployment; the paper itself notes the system is not real-time.
  • domain assumption Attention weights of action tokens toward VLM versus VGM tokens measure causal suppression of predictive dynamics.
    The motivation for action-space coordination in Figure 2 and Appendix A interprets attention ratios as evidence of modality competition; no intervention or causal test is provided.
  • domain assumption Freezing the pretrained WAM and VLA branches preserves complementary representations and makes Stage 3 optimization stable.
    Appendix D shows full fine-tuning gives 88.8 versus 89.5 PDMS, but the explanation relies on an assumed convergence-speed imbalance between the two branches.
  • domain assumption Qwen3-VL-4B and Wan2.2-TI2V-5B pretrained weights provide the semantic and video priors the method depends on.
    The VLA and WAM branches are initialized from these public models; final performance depends on their quality, and they are treated as fixed inputs from prior literature.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BrainWAM: Action-Space Coordination of Semantic Priors and Predictive Dynamics for Autonomous Driving." pith.science (2026). https://pith.science/paper/G656RU52

@misc{pith2026260812854,
  author       = {Pith},
  title        = {Pith review of: BrainWAM: Action-Space Coordination of Semantic Priors and Predictive Dynamics for Autonomous Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G656RU52}},
  note         = {Machine review of arXiv:2608.12854}
}
read the original abstract

Autonomous driving requires planning under both semantic constraints and predictive dynamics. Existing end-to-end driving approaches, however, typically emphasize only one side of this requirement: Vision-Language-Action (VLA) models exploit VLM priors for semantic reasoning, while World Action Models (WAMs) provide future-aware prediction through generative world modeling. This naturally motivates a unified planner that can leverage both semantic priors and predictive dynamics. However, we find that a naive combination through joint token-level attention suffers from an attention-allocation mismatch, where semantic shortcuts dominate the shared attention space and suppress predictive dynamics. Inspired by neuroscience evidence that complex behavior arises from coordination among functionally specialized systems, we propose BrainWAM, a structured action-space coordination framework that converts semantic reasoning and predictive world modeling into two specialized action-oriented pathways, and aligns them at the level of compact action representations. We further introduce an asynchronous rectified-flow inference strategy with decoupled video and action denoising, which shortens inference latency while preserving planning-relevant predictive context. BrainWAM reaches state-of-the-art performance on both NAVSIM v1 (89.5 PDMS) and NAVSIM v2 (89.6 EPDMS), consistently outperforming VLA-only or WAM-only methods, highlighting BrainWAM as a practical and promising direction for autonomous driving systems.

Figures

Figures reproduced from arXiv: 2608.12854 by the authors.

Figure 2
Figure 2. Attention allocation in Tri-MoT. We compare atten [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overview of the proposed semantic-predictive action architecture. The VLA pathway distills scene semantics, route [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Three-stage training pipeline. Stage 1 trains the WAM branch with video and action rectified-flow objectives, enabling [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Qualitative comparison of VLA-only, WAM-only, and Ours in representative scenarios. Ours produces robust trajectories [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Additional qualitative comparisons among VLA-only, WAM-only, and BrainWAM. Each row presents the predicted [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 17 canonical work pages

  1. [1]

    IEEE Robotics and Automation Letters , volume=

    Drivegpt4: Interpretable end-to-end autonomous driving via large language model , author=. IEEE Robotics and Automation Letters , volume=. 2024 , publisher=

    work page 2024

  2. [2]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Orion: A holistic end-to-end autonomous driving framework by vision-language instructed action generation , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    work page

  3. [3]

    arXiv preprint arXiv:2506.08052 , year=

    Recogdrive: A reinforced cognitive framework for end-to-end autonomous driving , author=. arXiv preprint arXiv:2506.08052 , year=

    arXiv

  4. [4]

    Advances in Neural Information Processing Systems , volume=

    Autovla: A vision-language-action model for end-to-end autonomous driving with adaptive reasoning and reinforcement fine-tuning , author=. Advances in Neural Information Processing Systems , volume=

    work page

  5. [5]

    International Conference on Learning Representations , volume=

    Enhancing end-to-end autonomous driving with latent world model , author=. International Conference on Learning Representations , volume=

    work page

  6. [6]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Drivinggpt: Unifying driving world modeling and planning with multi-modal autoregressive transformers , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  7. [7]

    arXiv preprint arXiv:2506.21539 , year=

    Worldvla: Towards autoregressive action world model , author=. arXiv preprint arXiv:2506.21539 , year=

    arXiv

  8. [8]

    arXiv preprint arXiv:2506.19850 , year=

    Unified vision-language-action model , author=. arXiv preprint arXiv:2506.19850 , year=

    arXiv

Show all 57 references
  1. [9]

    arXiv preprint arXiv:2510.12796 , year=

    DriveVLA-W0: World models amplify data scaling law in autonomous driving , author=. arXiv preprint arXiv:2510.12796 , year=

    arXiv

  2. [10]

    Nature Reviews Neuroscience , volume=

    Forty-five years of split-brain research and still going strong , author=. Nature Reviews Neuroscience , volume=. 2005 , publisher=

    2005

  3. [11]

    Proceedings of the national academy of sciences , volume=

    Functional specificity in the human brain: a window into the functional architecture of the mind , author=. Proceedings of the national academy of sciences , volume=. 2010 , publisher=

    2010

  4. [12]

    Trends in cognitive sciences , volume=

    Internal models in the cerebellum , author=. Trends in cognitive sciences , volume=. 1998 , publisher=

    1998

  5. [13]

    Neuron , volume=

    The cerebellum and cognitive function: 25 years of insight from anatomy and neuroimaging , author=. Neuron , volume=. 2013 , publisher=

    2013

  6. [14]

    Nature Reviews Neuroscience , volume=

    The basal ganglia and the cerebellum: nodes in an integrated network , author=. Nature Reviews Neuroscience , volume=. 2018 , publisher=

    2018

  7. [15]

    arXiv preprint arXiv:2210.02747 , year=

    Flow matching for generative modeling , author=. arXiv preprint arXiv:2210.02747 , year=

    arXiv

  8. [16]

    arXiv preprint arXiv:2209.03003 , year=

    Flow straight and fast: Learning to generate and transfer data with rectified flow , author=. arXiv preprint arXiv:2209.03003 , year=

    arXiv

  9. [17]

    Advances in neural information processing systems , volume=

    Flamingo: a visual language model for few-shot learning , author=. Advances in neural information processing systems , volume=

  10. [18]

    arXiv preprint arXiv:2303.16199 , year=

    Llama-adapter: Efficient fine-tuning of language models with zero-init attention , author=. arXiv preprint arXiv:2303.16199 , year=

    arXiv

  11. [19]

    IEEE transactions on pattern analysis and machine intelligence , volume=

    Transfuser: Imitation with transformer-based sensor fusion for autonomous driving , author=. IEEE transactions on pattern analysis and machine intelligence , volume=. 2022 , publisher=

    2022

  12. [20]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Planning-oriented autonomous driving , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  13. [21]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Para-drive: Parallelized architecture for real-time autonomous driving , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  14. [22]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  15. [23]

    arXiv preprint arXiv:2503.12820 , year=

    Hydra-mdp++: Advancing end-to-end driving via expert-guided hydra-distillation , author=. arXiv preprint arXiv:2503.12820 , year=

    arXiv

  16. [24]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Drivesuprim: Towards precise trajectory selection for end-to-end planning , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  17. [25]

    IEEE Robotics and Automation Letters , volume=

    Artemis: Autoregressive end-to-end trajectory planning with mixture of experts for autonomous driving , author=. IEEE Robotics and Automation Letters , volume=. 2025 , publisher=

    2025

  18. [26]

    arXiv preprint arXiv:2603.11041 , year=

    DynVLA: Learning World Dynamics for Action Reasoning in Autonomous Driving , author=. arXiv preprint arXiv:2603.11041 , year=

  19. [27]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Epona: Autoregressive diffusion world model for autonomous driving , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  20. [28]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    End-to-end driving with online trajectory evaluation via bev world model , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  21. [29]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Drivelaw: Unifying planning and video generation in a latent driving world , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  22. [30]

    arXiv preprint arXiv:2604.01765 , year=

    Drivedreamer-policy: A geometry-grounded world-action model for unified generation and planning , author=. arXiv preprint arXiv:2604.01765 , year=

  23. [31]

    Advances in Neural Information Processing Systems , volume=

    Navsim: Data-driven non-reactive autonomous vehicle simulation and benchmarking , author=. Advances in Neural Information Processing Systems , volume=

  24. [32]

    arXiv preprint arXiv:2506.04218 , year=

    Pseudo-simulation for autonomous driving , author=. arXiv preprint arXiv:2506.04218 , year=

  25. [33]

    arXiv preprint arXiv:2503.20314 , year=

    Wan: Open and advanced large-scale video generative models , author=. arXiv preprint arXiv:2503.20314 , year=

    arXiv

  26. [34]

    arXiv preprint arXiv:2511.21631 , year=

    Qwen3-vl technical report , author=. arXiv preprint arXiv:2511.21631 , year=

    arXiv

  27. [35]

    Proceedings of the Conference on Computer Vision and Pattern Recognition, Vancouver, Canada , pages=

    Openscene: The largest up-to-date 3d occupancy prediction benchmark in autonomous driving , author=. Proceedings of the Conference on Computer Vision and Pattern Recognition, Vancouver, Canada , pages=

  28. [36]

    arXiv preprint arXiv:2106.11810 , year=

    nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles , author=. arXiv preprint arXiv:2106.11810 , year=

    arXiv

  29. [37]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Lmdrive: Closed-loop end-to-end driving with large language models , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  30. [38]

    arXiv preprint arXiv:2402.12289 , year=

    Drivevlm: The convergence of autonomous driving and large vision-language models , author=. arXiv preprint arXiv:2402.12289 , year=

    arXiv

  31. [39]

    European conference on computer vision , pages=

    Drivelm: Driving with graph visual question answering , author=. European conference on computer vision , pages=. 2024 , organization=

    2024

  32. [40]

    European Conference on Computer Vision , pages=

    Dolphins: Multimodal language model for driving , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  33. [41]

    Sensors , volume=

    Drivellava: Human-level behavior decisions via vision language model , author=. Sensors , volume=. 2024 , publisher=

    2024

  34. [42]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Opendrivevla: Towards end-to-end autonomous driving with large vision language action model , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  35. [43]

    arXiv preprint arXiv:2309.17080 , year=

    Gaia-1: A generative world model for autonomous driving , author=. arXiv preprint arXiv:2309.17080 , year=

    arXiv

  36. [44]

    European conference on computer vision , pages=

    Drivedreamer: Towards real-world-drive world models for autonomous driving , author=. European conference on computer vision , pages=. 2024 , organization=

    2024

  37. [45]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Drivedreamer-2: Llm-enhanced world models for diverse driving video generation , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  38. [46]

    arXiv preprint arXiv:2311.13549 , year=

    Adriver-i: A general world model for autonomous driving , author=. arXiv preprint arXiv:2311.13549 , year=

    arXiv

  39. [47]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Driving into the future: Multiview visual forecasting and planning with world model for autonomous driving , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  40. [48]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Motus: A unified latent action world model , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  41. [49]

    arXiv preprint arXiv:2504.02792 , year=

    Unified world models: Coupling video and action diffusion for pretraining on large robotic datasets , author=. arXiv preprint arXiv:2504.02792 , year=

    arXiv

  42. [50]

    Advances in neural information processing systems , volume=

    Videovla: Video generators can be generalizable robot manipulators , author=. Advances in neural information processing systems , volume=

  43. [51]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    World4drive: End-to-end autonomous driving via intention-aware physical latent world model , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  44. [52]

    arXiv preprint arXiv:2511.20156 , year=

    Map-World: Masked Action planning and Path-Integral World Model for Autonomous Driving , author=. arXiv preprint arXiv:2511.20156 , year=

  45. [53]

    arXiv preprint arXiv:2512.11872 , year=

    WAM-Diff: A Masked Diffusion VLA Framework with MoE and Online Reinforcement Learning for Autonomous Driving , author=. arXiv preprint arXiv:2512.11872 , year=

  46. [54]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    What makes training multi-modal classification networks hard? , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  47. [55]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Balanced multimodal learning via on-the-fly gradient modulation , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  48. [56]

    International conference on machine learning , pages=

    Modality competition: What makes joint training of multi-modal network fail in deep learning?(provably) , author=. International conference on machine learning , pages=. 2022 , organization=

    2022

  49. [57]

    International Conference on Machine Learning , pages=

    On uni-modal feature learning in supervised multi-modal learning , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.