Pith. sign in

REVIEW 3 major objections 6 minor 54 references

A car's own traffic world model can be turned into a structured multi-agent adversary that hardens its planner against rare interactive failures without external simulators.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-07-14 10:20 UTC pith:TMLLRZYL

load-bearing objection Endogenous role-conditioned world-model self-play is a real, usable advance for tokenized planners; the coverage-gap theory is honest but unmeasured, and the closed-loop gains are real but modest. the 3 major comments →

arxiv 2607.10630 v1 pith:TMLLRZYL submitted 2026-07-12 cs.RO cs.AI

World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planning

Tong Nie , Yuewen Mei , Junlin He , Yihong Tang , Jian Sun , Wei Ma This is my paper
classification cs.RO cs.AI
keywords adversarial world modelsmulti-agent self-playautoregressive motion planningrobust motion planningcounterfactual credit assignmentregret-aware CVaRnuPlanclosed-loop planning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Modern autoregressive driving planners learn well from ordinary traffic logs but break on rare interactive failures such as aggressive cut-ins and coordinated blocking. This paper argues that the planner's own predictive world model already lives in the same tokenized motion space, so it can be repurposed as an endogenous adversary. The authors cast robust planning as a constrained min-max game and solve it with a two-stage self-play pipeline: first the world model learns sparse, scene-adaptive attack coalitions by role conditioning and counterfactual credit; then the ego planner is fine-tuned as a regret-aware best response that emphasizes tail risk while staying close to its nominal behavior. On nuPlan and InterPlan, the resulting planner improves hard and long-tail closed-loop scores without broadly sacrificing ordinary driving, and the learned adversary transfers stress across other planners. The practical claim is that principled adversarial self-play can be done inside the planner's own generative interface rather than with external scenario generators or heavy simulators.

Core claim

Adversarially robust autoregressive motion planning can be cast as a constrained min-max game and solved tractably by converting the planner's predictive world model into a role-conditioned multi-agent adversary that discovers sparse scene-adaptive attack coalitions, then optimizing the ego planner as a regret-aware, trust-region best response to the frozen adversary's calibrated tail-risk distribution.

What carries the argument

Adversarial World Modeling (AWM): a decoupled Stage-A/B self-play solver in which role-conditioned counterfactual credit assignment learns sparse coalitions, and a regret-CVaR outer update with reference-anchored trust regions produces a robust planner against the frozen risk distribution.

Load-bearing premise

The sparse role-conditioned attack pipeline covers enough of the true worst-case interactive failures that freezing it and training a local best response is a faithful enough surrogate of the full min-max game.

What would settle it

If an independent closed-loop stress test finds safety-critical multi-agent failures that AWM never produces yet that still crash the fine-tuned planner (or if forcing denser, unrestricted coalitions yields large robustness gains that the calibrated sparse pipeline cannot match), the coverage claim of the decoupled solver would fail.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Autoregressive planners can be adversarially fine-tuned without external scenario generators or separate physical-simulator rollouts, because attacks stay inside the shared tokenized motion manifold.
  • Sparse scene-adaptive coalitions (often one or two agents) plus pair-admission calibration can supply useful training stress without collapsing into unrealistic, purely destructive traffic.
  • Regret-focused tail weighting plus nominal trust-region constraints can raise hard-case recovery while keeping ordinary closed-loop driving competitive.
  • A world model trained this way can later serve as a transferable simulator-side traffic host that degrades multiple other planners, not only the one used in self-play.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same endogenous-adversary pattern may apply to other tokenized sequential decision systems (robotics, multi-agent simulation) that already maintain a predictive world model.
  • Coverage-gap diagnostics (how much worst-case mass the proposal-top1-pair pipeline omits) could become a practical certification metric for whether Stage-A is ready to freeze.
  • If tokenized world-model fidelity remains the bottleneck, hybrid attacks that occasionally leave the discrete vocabulary may be needed for residual failure modes AWM cannot reach.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper formulates adversarially robust autoregressive motion planning as a constrained min-max game and proposes Adversarial World Modeling (AWM), a two-stage self-play fine-tuning method. Stage A converts the planner’s predictive world model into a role-conditioned adversary that discovers sparse, scene-adaptive attack coalitions via proposal/primary/pair selection and hybrid counterfactual credit (leave-one-out plus team role-switch), with a calibrator for pair admission. Stage B freezes this AWM and optimizes the ego planner as a regret-aware best response under closed-form two-point CVaR tail weights, reference-anchored trust-region retention, and dual safety constraints. Appendix C supplies local sensitivity, freezing-error, coverage-gap, Bayes admission, difference-reward, CVaR, and nominal-floor arguments aligned with the pipeline. On nuPlan and InterPlan, the resulting planner improves hard and long-tail closed-loop scores relative to Plan-R1 while largely preserving nominal performance, and the learned AWM transfers as a sim-agent stressor across multiple planners.

Significance. If the results hold, this is a useful contribution for closed-loop training of modern tokenized planners: it internalizes adversarial multi-agent interactions in the same autoregressive interface as the planner, avoiding external scenario generators and heavy simulator-only attackers. The combination of role-conditioned counterfactual credit, scene-adaptive coalitions, and regret-CVaR with explicit nominal retention is a coherent design, and the theory is unusually well matched to the implemented solver rather than decorative. Cross-planner sim-agent transfer (Table 2), training-paradigm comparison (Table 3), and mechanism ablations (Tables 9–15) strengthen the empirical case beyond a single scoreboard win. The work does not claim formal closed-loop safety certificates, but it offers a practical, theoretically scoped path to harden AR planners on interactive long-tail traffic.

major comments (3)
  1. [Appendix C.2, Prop. C.2–C.4] Appendix C.2, Prop. C.2–C.4 and Cor. C.4: the local Stackelberg justification for freezing Stage A is controlled by the structured-class coverage gap δ_cov and omitted support mass η_s, yet neither quantity (nor a proxy) is measured. Without evidence that the proposal→primary→pair→calibrator class with K_max=2 covers the planner-relevant low-utility support, the theory only bounds error conditional on an untested assumption, and the claim that the decoupled solver is a faithful surrogate of the ideal game remains incomplete. Please either (i) report a proxy for coverage (e.g., residual utility gap vs denser/search-based adversaries under matched realism gates, or mass of held-out failure modes recovered by AWM), or (ii) explicitly scope the theoretical claims to “robustness within the structured AWM manifold” in the main text and abstract.
  2. [Table 15, §4.2] Table 15 and §4.2: adapted CAT*/STRIVE*/KING* comparisons evaluate generators under the same AR interface, but they do not retrain the ego planner end-to-end under those adversaries with the same Stage-B objective and compute budget. The central claim that AWM’s self-play pipeline is preferable for robust planner learning therefore rests mainly on internal ablations (Tables 3–4, 9–12) and modest closed-loop gains (Table 1: e.g., Test14-hard NR +2.60, InterPlanLT +5.81). A controlled planner-training comparison against at least one strong external adversarial host (or a clear statement that such full-pipeline comparison is out of scope and why) is needed so readers can separate “better structured adversary for this architecture” from “better overall robust training recipe.”
  3. [Table 1, Table 5] Table 1 and Table 5: gains are concentrated on hard/interactive splits, while Test14-random NR is essentially flat (−0.01) and left_turn_yield on InterPlan-LongTail slightly degrades. The paper argues that hard-case improvement is not bought by broad nominal sacrifice, but the failure-mode analysis is thin. Please quantify when AWM-Planner underperforms the reference (scene types, attack branch, safety dual activity) and whether those cases indicate incomplete adversarial coverage, over-conservatism under retention, or dual-constraint side effects. This is load-bearing for the claim of competitive performance “in both nominal and highly interactive long-tail scenarios.”
minor comments (6)
  1. [Figure 1] Figure 1 is information-dense; a short caption walkthrough of Stage A vs Stage B data flow (what is frozen when) would help readers who skip the algorithm box.
  2. [§3, Algorithm 1] Notation switches between π_awm / π*_awm / π⋆_awm and As / A_s; unify star/asterisk notation and bolding of sets in the main text and Algorithm 1.
  3. [§2.2, Eq. (2)] Eq. (2) writes a constrained two-player zero-sum game with trust-region and attack-budget constraints; a one-sentence clarification that the trust region is only enforced in Stage B (outer problem) would avoid reading it as a joint constrained saddle solved simultaneously.
  4. [Table 2] Table 2 caption is long and easy to misread as ordinary planner closed-loop scores; consider a shorter main caption plus a footnote that R-Score is sim-agent score under NORMAL vs AWM hosts.
  5. [Appendix A] Related work is thorough; a brief explicit contrast with Adv-0 / CAT-style closed-loop adversarial training on whether they share the tokenized world-model interface would sharpen the novelty paragraph.
  6. [Appendix B.1.1] Typos/style: “interactivep scene state” (B.1.1); occasional double spaces; “A WM” line breaks in headings. Light copy-edit pass recommended.

Circularity Check

0 steps flagged

No significant circularity: theory is self-contained approximation analysis; empirics use external closed-loop benchmarks independent of the training objective.

full rationale

The paper’s load-bearing chain is (i) cast robust AR planning as a constrained min-max game, (ii) approximate it by a decoupled Stage-A/B solver with explicit local error terms (coverage gap, Stage-A/B suboptimality, trust-region radius), and (iii) evaluate the resulting planner on public nuPlan/InterPlan closed-loop scores against independent baselines. Appendix C derives approximation bounds (Prop. C.2–C.4), gradient alignment of hybrid counterfactual credit (Prop. C.6), regret as a control variate (Prop. C.9), closed-form two-point CVaR weights (Prop. C.11), and nominal/safety surrogates (Prop. C.12–C.13) from stated assumptions and standard score-function / CVaR identities; none of these equate a fitted parameter to the claimed prediction by construction. Empirical metrics (NR/R closed-loop scores, InterPlan templates, cross-planner sim-agent R-scores) are external simulator scores, not renamings of the Stage-B loss. Self-citations to the authors’ prior adversarial-scenario work appear in related work and as adapted baselines, but they are not uniqueness theorems or load-bearing premises of the derivation. Coverage gap η_s/δ_cov is unmeasured—an empirical completeness concern, not circularity. Score 0 is therefore appropriate.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 4 invented entities

The central claim rests on standard AR planning factorization and bounded finite-horizon utilities, plus several paper-specific modeling choices: sparse budgeted coalitions, role embeddings that stay on the motion manifold, freezing Stage A before Stage B, and hand-chosen risk/retention hyperparameters. Invented machinery (AWM roles, hybrid counterfactual rewards, pair calibrator, regret-CVaR weights) is justified by lemmas but validated mainly inside this pipeline.

free parameters (6)
  • K_max attack budget = 2
    Maximum active adversaries per scene; experiments fix K_max=2 and ablate fixed K=1/2 vs adaptive admission.
  • CVaR tail mass τ = 0.25
    Controls how aggressively Stage B upweights rare high-regret branches; default 0.25 with sensitivity in Table 14.
  • pair-admission / AWM threshold = 0.45
    Scene calibrator threshold deciding whether the second attacker is admitted during training/eval; default 0.45.
  • nominal retention weights (λ_kl, λ_margin, progress weight) = progress weight 10.0 (base)
    Trust-region and asymmetric margin penalties that trade robustness for nominal driving; progress weight default 10.0.
  • hybrid credit weights ω_loo, ω_team
    Mix leave-one-out and team counterfactual rewards for attacker policy gradients; chosen to preserve non-additive cooperation.
  • safety thresholds κ_m and dual logits
    Admissible violation rates and softmax-parameterized cost multipliers for collision/off-road constraints.
axioms (5)
  • domain assumption Planning-prediction factorization of joint token generation (Eq. 1) is an adequate interface for both ego optimization and environment dynamics.
    Section 2.1 treats this decoupled AR factorization as given, following Plan-R1-style architectures.
  • standard math Finite-horizon utility is bounded and locally Lipschitz in total variation of the planner token policy (Lemma C.1).
    Used throughout Appendix C to control trust-region and freezing errors.
  • domain assumption Role embeddings change behavioral distribution without leaving the shared discrete motion manifold or transition operator Γ.
    Section 3.2 and B.1; required so counterfactual role switches isolate credit rather than dynamics mismatch.
  • ad hoc to paper A sparse structured adversary class (proposal/primary/pair/calibrator) covers the adversarial support that matters for local robust value.
    Prop. C.2–C.4; the coverage gap δ_cov is the main theoretical residual of the decoupled solver.
  • ad hoc to paper Reference-relative regret under frozen AWM branches is the right risk random variable for tail optimization versus absolute utility.
    Section 3.3 and Prop. C.9; cancels scene-intrinsic difficulty under the stated decomposition.
invented entities (4)
  • Adversarial World Model (role-conditioned AWM) independent evidence
    purpose: Convert the benign predictive world model into a controllable multi-agent adversary on the same token interface.
    Core invented mechanism; independent_evidence is partial via cross-planner sim-agent transfer (Table 2), not external physics validation.
  • Hybrid counterfactual credit Δ_hyb (leave-one-out + team role-switch) no independent evidence
    purpose: Attribute scene-level attack utility to sparse coalitions without exploding joint action trees.
    Prop. C.6–C.7 cast it as a difference-reward estimator; evidence is ablation Table 9, not external theory independent of this paper.
  • Scene-adaptive pair calibrator q_η no independent evidence
    purpose: Decide whether a second attacker has positive expected marginal gain in the current scene.
    Prop. C.5 as Bayes gate; calibrated inside the training loop.
  • AWM-induced two-point regret-CVaR risk distribution no independent evidence
    purpose: Focus planner gradients on the worst of top-1 vs pair branches with closed-form weights.
    Prop. C.11; specific to the paper's branch construction.

pith-pipeline@v1.1.0-grok45 · 43656 in / 3806 out tokens · 62354 ms · 2026-07-14T10:20:45.839274+00:00 · methodology

0 comments
read the original abstract

Robust motion planning in dense traffic requires autonomous vehicles to interact in rare and safety-critical scenarios that are underrepresented in naturalistic driving data. Although adversarial training offers a feasible solution, existing methods often rely on external scenario generators, heuristic perturbations, or simulator-heavy rollouts, which makes them difficult to integrate with modern autoregressive planners. Here, we cast adversarially robust planner learning as a constrained min-max game and propose Adversarial World Modeling (AWM), a theoretically grounded multi-agent self-play fine-tuning framework. Since solving the exact game is intractable, AWM introduces a principled decoupled solver. In the inner minimization, the planner's predictive world model is converted into a role-conditioned adversary that learns sparse, scene-adaptive attack coalitions via counterfactual credit assignment. In the outer maximization, the ego planner optimizes a regret-aware robust best response against the frozen AWM, utilizing tail-risk weighting and reference-anchored trust regions to improve hard-case recovery while preserving nominal driving behavior. Experiments on the nuPlan and InterPlan benchmarks demonstrate that our method generates transferable adversarial interactions and yields a robust planner that achieves competitive closed-loop performance in both nominal and highly interactive long-tail scenarios. Theoretical analysis justifies the decoupled solver and the main optimization components.

Figures

Figures reproduced from arXiv: 2607.10630 by Jian Sun, Junlin He, Tong Nie, Wei Ma, Yihong Tang, Yuewen Mei.

Figure 1
Figure 1. Figure 1: Decoupled self-play fine-tuning with AWM. The full pipeline is described in Algorithm 1. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: (a) Cross-validation shows that robust fine-tuning improves the adversarial tail while [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Test14-hard qualitative cases. Orange denotes the original Plan-R1 planner and green [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: K=2 cooperative attack in sim-agent eval [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Sensitivity around the selected planner operating point. The default configuration balances [PITH_FULL_IMAGE:figures/full_fig_p036_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

54 extracted references · 9 linked inside Pith

  1. [1]

    A comprehensive survey of multia- gent reinforcement learning.IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 38(2):156–172, 2008

    Lucian Busoniu, Robert Babuska, and Bart De Schutter. A comprehensive survey of multia- gent reinforcement learning.IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 38(2):156–172, 2008

    2008

  2. [2]

    nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles.arXiv preprint arXiv:2106.11810, 2021

    Holger Caesar, Juraj Kabzan, Kok Seang Tan, Whye Kit Fong, Eric Wolff, Alex Lang, Luke Fletcher, Oscar Beijbom, and Sammy Omari. nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles.arXiv preprint arXiv:2106.11810, 2021

    Pith/arXiv arXiv 2021

  3. [3]

    Reinforcement learning with human feedback for realistic traffic simulation

    Yulong Cao, Boris Ivanovic, Chaowei Xiao, and Marco Pavone. Reinforcement learning with human feedback for realistic traffic simulation. In2024 IEEE international conference on robotics and automation (ICRA), pages 14428–14434. IEEE, 2024

    2024

  4. [4]

    Safe-sim: Safety-critical closed-loop traffic simulation with diffusion-controllable adversaries

    Wei-Jer Chang, Francesco Pittaluga, Masayoshi Tomizuka, Wei Zhan, and Manmohan Chan- draker. Safe-sim: Safety-critical closed-loop traffic simulation with diffusion-controllable adversaries. InEuropean conference on computer vision, pages 242–258. Springer, 2024

    2024

  5. [5]

    Rift: Group-relative rl fine-tuning for realistic and controllable traffic simulation.arXiv preprint arXiv:2505.03344, 2025

    Keyu Chen, Wenchao Sun, Hao Cheng, and Sifa Zheng. Rift: Group-relative rl fine-tuning for realistic and controllable traffic simulation.arXiv preprint arXiv:2505.03344, 2025

    arXiv 2025

  6. [6]

    Pluto: Pushing the limit of imitation learning- based planning for autonomous driving.arXiv preprint arXiv:2404.14327, 2024

    Jie Cheng, Yingbing Chen, and Qifeng Chen. Pluto: Pushing the limit of imitation learning- based planning for autonomous driving.arXiv preprint arXiv:2404.14327, 2024

    Pith/arXiv arXiv 2024

  7. [7]

    Rethinking imitation-based planners for autonomous driving

    Jie Cheng, Yingbing Chen, Xiaodong Mei, Bowen Yang, Bo Li, and Ming Liu. Rethinking imitation-based planners for autonomous driving. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 14123–14130. IEEE, 2024

    2024

  8. [8]

    Training gans with optimism.arXiv preprint arXiv:1711.00141, 2017

    Constantinos Daskalakis, Andrew Ilyas, Vasilis Syrgkanis, and Haoyang Zeng. Training gans with optimism.arXiv preprint arXiv:1711.00141, 2017. 10

    Pith/arXiv arXiv 2017

  9. [9]

    Parting with mis- conceptions about learning-based vehicle motion planning

    Daniel Dauner, Marcel Hallgarten, Andreas Geiger, and Kashyap Chitta. Parting with mis- conceptions about learning-based vehicle motion planning. InConference on Robot Learning, pages 1268–1281. PMLR, 2023

    2023

  10. [10]

    Artemis: Autoregressive end-to-end trajectory planning with mixture of experts for autonomous driving.IEEE Robotics and Automation Letters, 11(1): 226–233, 2025

    Renju Feng, Ning Xi, Duanfeng Chu, Rukang Wang, Zejian Deng, Anzheng Wang, Liping Lu, Jinxiang Wang, and Yanjun Huang. Artemis: Autoregressive end-to-end trajectory planning with mixture of experts for autonomous driving.IEEE Robotics and Automation Letters, 11(1): 226–233, 2025

    2025

  11. [11]

    Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment.Nature communications, 12(1):748, 2021

    Shuo Feng, Xintao Yan, Haowei Sun, Yiheng Feng, and Henry X Liu. Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment.Nature communications, 12(1):748, 2021

    2021

  12. [12]

    Dense reinforcement learning for safety validation of autonomous vehicles.Nature, 615 (7953):620–627, 2023

    Shuo Feng, Haowei Sun, Xintao Yan, Haojie Zhu, Zhengxia Zou, Shengyin Shen, and Henry X Liu. Dense reinforcement learning for safety validation of autonomous vehicles.Nature, 615 (7953):620–627, 2023

    2023

  13. [13]

    Breaking through safety performance stagnation in autonomous vehicles with dense learning.Nature Communications, 2026

    Shuo Feng, Haojie Zhu, Haowei Sun, Xintao Yan, Linxuan He, Jingxuan Yang, Guangzhen Su, Boqi Li, Shu Li, Ling Wang, et al. Breaking through safety performance stagnation in autonomous vehicles with dense learning.Nature Communications, 2026

    2026

  14. [14]

    Counterfactual multi-agent policy gradients

    Jakob Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson. Counterfactual multi-agent policy gradients. InProceedings of the AAAI conference on artificial intelligence, volume 32, 2018

    2018

  15. [15]

    Foundation models in autonomous driving: A survey on scenario generation and scenario analysis.IEEE Open Journal of Intelligent Transportation Systems, 2026

    Yuan Gao, Mattia Piccinini, Yuchen Zhang, Dingrui Wang, Korbinian Moller, Roberto Brusnicki, Baha Zarrouki, Alessio Gambi, Jan Frederik Totz, Kai Storms, et al. Foundation models in autonomous driving: A survey on scenario generation and scenario analysis.IEEE Open Journal of Intelligent Transportation Systems, 2026

    2026

  16. [16]

    Variance reduction techniques for gradient estimates in reinforcement learning.Journal of Machine Learning Research, 5(Nov): 1471–1530, 2004

    Evan Greensmith, Peter L Bartlett, and Jonathan Baxter. Variance reduction techniques for gradient estimates in reinforcement learning.Journal of Machine Learning Research, 5(Nov): 1471–1530, 2004

    2004

  17. [17]

    Can vehicle motion planning generalize to realistic long-tail scenarios? In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5388–5395

    Marcel Hallgarten, Julian Zapata, Martin Stoll, Katrin Renz, and Andreas Zell. Can vehicle motion planning generalize to realistic long-tail scenarios? In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5388–5395. IEEE, 2024

    2024

  18. [18]

    King: Generating safety-critical driving scenarios for robust imitation via kinematics gradients

    Niklas Hanselmann, Katrin Renz, Kashyap Chitta, Apratim Bhattacharyya, and Andreas Geiger. King: Generating safety-critical driving scenarios for robust imitation via kinematics gradients. InEuropean Conference on Computer Vision, pages 335–352. Springer, 2022

    2022

  19. [19]

    Solving motion planning tasks with a scalable generative model

    Yihan Hu, Siqi Chai, Zhening Yang, Jingyu Qian, Kun Li, Wenxin Shao, Haichao Zhang, Wei Xu, and Qiang Liu. Solving motion planning tasks with a scalable generative model. In European Conference on Computer Vision, pages 386–404. Springer, 2024

    2024

  20. [20]

    Gameformer: Game-theoretic modeling and learning of transformer-based interactive prediction and planning for autonomous driving

    Zhiyu Huang, Haochen Liu, and Chen Lv. Gameformer: Game-theoretic modeling and learning of transformer-based interactive prediction and planning for autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 3903–3913, 2023

    2023

  21. [21]

    Gen-drive: Enhancing diffusion generative driving policies with reward modeling and reinforcement learning fine-tuning

    Zhiyu Huang, Xinshuo Weng, Maximilian Igl, Yuxiao Chen, Yulong Cao, Boris Ivanovic, Marco Pavone, and Chen Lv. Gen-drive: Enhancing diffusion generative driving policies with reward modeling and reinforcement learning fine-tuning. In2025 IEEE International Conference on Robotics and Automation (ICRA), pages 3445–3451. IEEE, 2025

    2025

  22. [22]

    Approximately optimal approximate reinforcement learning

    Sham Kakade and John Langford. Approximately optimal approximate reinforcement learning. InProceedings of the nineteenth international conference on machine learning, pages 267–274, 2002

    2002

  23. [23]

    Beyond behavior cloning in autonomous driving: a survey of closed-loop training techniques.Authorea Preprints, 2025

    Peter Karkus, Maximilian Igl, Yuxiao Chen, Kashyap Chitta, Jef Packer, Bertrand Douillard, Ran Tian, Alexander Naumann, Guillermo Garcia-Cobo, Shuhan Tan, et al. Beyond behavior cloning in autonomous driving: a survey of closed-loop training techniques.Authorea Preprints, 2025. 11

    2025

  24. [24]

    Plannerrft: Reinforcing diffusion planners through closed-loop and sample-efficient fine-tuning.arXiv preprint arXiv:2601.12901, 2026

    Hongchen Li, Tianyu Li, Jiazhi Yang, Haochen Tian, Caojun Wang, Lei Shi, Mingyang Shang, Zengrong Lin, Gaoqiang Wu, Zhihui Hao, et al. Plannerrft: Reinforcing diffusion planners through closed-loop and sample-efficient fine-tuning.arXiv preprint arXiv:2601.12901, 2026

    arXiv 2026

  25. [25]

    Curse of rarity for autonomous vehicles.nature communications, 15(1):4808, 2024

    Henry X Liu and Shuo Feng. Curse of rarity for autonomous vehicles.nature communications, 15(1):4808, 2024

    2024

  26. [26]

    Adv-bmt: Bidirectional motion transformer for safety-critical traffic scenario generation.arXiv preprint arXiv:2506.09485, 2025

    Yuxin Liu, Zhenghao Peng, Xuanhao Cui, and Bolei Zhou. Adv-bmt: Bidirectional motion transformer for safety-critical traffic scenario generation.arXiv preprint arXiv:2506.09485, 2025

    arXiv 2025

  27. [27]

    Llm-attacker: Enhancing closed-loop adversarial scenario generation for autonomous driving with large language models.IEEE Transactions on Intelligent Transportation Systems, 2025

    Yuewen Mei, Tong Nie, Jian Sun, and Ye Tian. Llm-attacker: Enhancing closed-loop adversarial scenario generation for autonomous driving with large language models.IEEE Transactions on Intelligent Transportation Systems, 2025

    2025

  28. [28]

    The numerics of gans.Advances in neural information processing systems, 30, 2017

    Lars Mescheder, Sebastian Nowozin, and Andreas Geiger. The numerics of gans.Advances in neural information processing systems, 30, 2017

    2017

  29. [29]

    Steerable adversarial scenario generation through test-time preference alignment.arXiv preprint arXiv:2509.20102, 2025

    Tong Nie, Yuewen Mei, Yihong Tang, Junlin He, Jie Sun, Haotian Shi, Wei Ma, and Jian Sun. Steerable adversarial scenario generation through test-time preference alignment.arXiv preprint arXiv:2509.20102, 2025

    Pith/arXiv arXiv 2025

  30. [30]

    Adv-0: Closed-loop min-max adversarial training for long-tail robustness in autonomous driving

    Tong Nie, Yihong Tang, Junlin He, Yuewen Mei, Jie Sun, Lijun Sun, Wei Ma, and Jian Sun. Adv-0: Closed-loop min-max adversarial training for long-tail robustness in autonomous driving. arXiv preprint arXiv:2603.15221, 2026

    arXiv 2026

  31. [31]

    Advancing multi-agent traffic simulation via r1-style reinforcement fine-tuning.arXiv preprint arXiv:2509.23993, 2025

    Muleilan Pei, Shaoshuai Shi, and Shaojie Shen. Advancing multi-agent traffic simulation via r1-style reinforcement fine-tuning.arXiv preprint arXiv:2509.23993, 2025

    arXiv 2025

  32. [32]

    Improving agent behaviors with rl fine-tuning for autonomous driving

    Zhenghao Peng, Wenjie Luo, Yiren Lu, Tianyi Shen, Cole Gulino, Ari Seff, and Justin Fu. Improving agent behaviors with rl fine-tuning for autonomous driving. InEuropean Conference on Computer Vision, pages 165–181. Springer, 2024

    2024

  33. [33]

    Trajeglish: Traffic modeling as next-token prediction.arXiv preprint arXiv:2312.04535, 2023

    Jonah Philion, Xue Bin Peng, and Sanja Fidler. Trajeglish: Traffic modeling as next-token prediction.arXiv preprint arXiv:2312.04535, 2023

    Pith/arXiv arXiv 2023

  34. [34]

    Generating useful accident-prone driving scenarios via a learned traffic prior

    Davis Rempe, Jonah Philion, Leonidas J Guibas, Sanja Fidler, and Or Litany. Generating useful accident-prone driving scenarios via a learned traffic prior. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17305–17315, 2022

    2022

  35. [35]

    Optimization of conditional value-at-risk

    R Tyrrell Rockafellar, Stanislav Uryasev, et al. Optimization of conditional value-at-risk. Journal of risk, 2:21–42, 2000

    2000

  36. [36]

    Urban driver: Learning to drive from real-world demonstrations using policy gradients

    Oliver Scheel, Luca Bergamini, Maciej Wolczyk, Bła˙zej Osi´nski, and Peter Ondruska. Urban driver: Learning to drive from real-world demonstrations using policy gradients. InConference on Robot Learning, pages 718–728. PMLR, 2022

    2022

  37. [37]

    Trust region policy optimization

    John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. InInternational conference on machine learning, pages 1889–1897. PMLR, 2015

    2015

  38. [38]

    Motionlm: Multi-agent motion forecasting as language modeling

    Ari Seff, Brian Cera, Dian Chen, Mason Ng, Aurick Zhou, Nigamaa Nayakanti, Khaled S Refaat, Rami Al-Rfou, and Benjamin Sapp. Motionlm: Multi-agent motion forecasting as language modeling. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 8579–8590, 2023

    2023

  39. [39]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Pith/arXiv arXiv 2024

  40. [40]

    Seal: Towards safe autonomous driving via skill-enabled adversary learning for closed-loop scenario generation.IEEE Robotics and Automation Letters, 10(9):9320–9327, 2025

    Benjamin Stoler, Ingrid Navarro, Jonathan Francis, and Jean Oh. Seal: Towards safe autonomous driving via skill-enabled adversary learning for closed-loop scenario generation.IEEE Robotics and Automation Letters, 10(9):9320–9327, 2025. 12

    2025

  41. [41]

    Large trajectory models are scalable motion predictors and planners.arXiv preprint arXiv:2310.19620, 2023

    Qiao Sun, Shiduo Zhang, Danjiao Ma, Jingzhe Shi, Derun Li, Simian Luo, Yu Wang, Ningyi Xu, Guangzhi Cao, and Hang Zhao. Large trajectory models are scalable motion predictors and planners.arXiv preprint arXiv:2310.19620, 2023

    Pith/arXiv arXiv 2023

  42. [42]

    Flow matching-based autonomous driving planning with advanced interactive behavior modeling.arXiv preprint arXiv:2510.11083, 2025

    Tianyi Tan, Yinan Zheng, Ruiming Liang, Zexu Wang, Kexin Zheng, Jinliang Zheng, Jianxiong Li, Xianyuan Zhan, and Jingjing Liu. Flow matching-based autonomous driving planning with advanced interactive behavior modeling.arXiv preprint arXiv:2510.11083, 2025

    arXiv 2025

  43. [43]

    Plan-r1: Safe and feasible trajectory planning as language modeling.arXiv preprint arXiv:2505.17659, 2025

    Xiaolong Tang, Meina Kan, Shiguang Shan, and Xilin Chen. Plan-r1: Safe and feasible trajectory planning as language modeling.arXiv preprint arXiv:2505.17659, 2025

    Pith/arXiv arXiv 2025

  44. [44]

    Motion planning for autonomous driving: The state of the art and future perspectives.IEEE Transactions on Intelligent Vehicles, 8(6):3692–3711, 2023

    Siyu Teng, Xuemin Hu, Peng Deng, Bai Li, Yuchen Li, Yunfeng Ai, Dongsheng Yang, Lingxi Li, Zhe Xuanyuan, Fenghua Zhu, et al. Motion planning for autonomous driving: The state of the art and future perspectives.IEEE Transactions on Intelligent Vehicles, 8(6):3692–3711, 2023

    2023

  45. [45]

    Advsim: Generating safety-critical scenarios for self- driving vehicles

    Jingkang Wang, Ava Pun, James Tu, Sivabalan Manivasagam, Abbas Sadat, Sergio Casas, Mengye Ren, and Raquel Urtasun. Advsim: Generating safety-critical scenarios for self- driving vehicles. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9909–9918, 2021

    2021

  46. [46]

    Smart: Scalable multi-agent real-time motion generation via next-token prediction.Advances in Neural Information Processing Systems, 37:114048–114071, 2024

    Wei Wu, Xiaoxin Feng, Ziyan Gao, and Yuheng Kan. Smart: Scalable multi-agent real-time motion generation via next-token prediction.Advances in Neural Information Processing Systems, 37:114048–114071, 2024

    2024

  47. [47]

    Diffscene: Diffusion-based safety- critical scenario generation for autonomous vehicles

    Chejian Xu, Aleksandr Petiushko, Ding Zhao, and Bo Li. Diffscene: Diffusion-based safety- critical scenario generation for autonomous vehicles. InProceedings of the AAAI conference on artificial intelligence, volume 39, pages 8797–8805, 2025

    2025

  48. [48]

    Dap: A discrete-token autoregressive planner for autonomous driving.arXiv preprint arXiv:2511.13306, 2025

    Bowen Ye, Bin Zhang, and Hang Zhao. Dap: A discrete-token autoregressive planner for autonomous driving.arXiv preprint arXiv:2511.13306, 2025

    arXiv 2025

  49. [49]

    Carplanner: Consistent auto-regressive trajectory planning for large-scale reinforcement learning in autonomous driving

    Dongkun Zhang, Jiaming Liang, Ke Guo, Sha Lu, Qi Wang, Rong Xiong, Zhenwei Miao, and Yue Wang. Carplanner: Consistent auto-regressive trajectory planning for large-scale reinforcement learning in autonomous driving. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 17239–17248, 2025

    2025

  50. [50]

    Epona: Autoregressive diffusion world model for autonomous driving

    Kaiwen Zhang, Zhenyu Tang, Xiaotao Hu, Xingang Pan, Xiaoyang Guo, Yuan Liu, Jingwei Huang, Li Yuan, Qian Zhang, Xiao-Xiao Long, et al. Epona: Autoregressive diffusion world model for autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 27220–27230, 2025

    2025

  51. [51]

    Cat: Closed-loop adversarial training for safe end-to-end driving

    Linrui Zhang, Zhenghao Peng, Quanyi Li, and Bolei Zhou. Cat: Closed-loop adversarial training for safe end-to-end driving. InConference on Robot Learning, pages 2357–2372. PMLR, 2023

    2023

  52. [52]

    Closed-loop supervised fine-tuning of tokenized traffic models

    Zhejun Zhang, Peter Karkus, Maximilian Igl, Wenhao Ding, Yuxiao Chen, Boris Ivanovic, and Marco Pavone. Closed-loop supervised fine-tuning of tokenized traffic models. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 5422–5432, 2025

    2025

  53. [53]

    Diffusion-based planning for autonomous driving with flexible guidance.arXiv preprint arXiv:2501.15564, 2025

    Yinan Zheng, Ruiming Liang, Kexin Zheng, Jinliang Zheng, Liyuan Mao, Jianxiong Li, Weihao Gu, Rui Ai, Shengbo Eben Li, Xianyuan Zhan, et al. Diffusion-based planning for autonomous driving with flexible guidance.arXiv preprint arXiv:2501.15564, 2025

    Pith/arXiv arXiv 2025

  54. [54]

    Behaviorgpt: Smart agent simulation for autonomous driving with next-patch prediction.Advances in Neural Information Processing Systems, 37:79597–79617, 2024

    Zikang Zhou, Haibo Hu, Xinhong Chen, Jianping Wang, Nan Guan, Kui Wu, Yung-Hui Li, Yu- Kai Huang, and Chun J Xue. Behaviorgpt: Smart agent simulation for autonomous driving with next-patch prediction.Advances in Neural Information Processing Systems, 37:79597–79617, 2024. 13 Appendix Appendix Contents A Related Work 15 B Extended Architectural and Algorit...

    arXiv 2024

This paper was first reviewed by grok-4.5 on July 14, 2026.