VLM as Strategist: Adaptive Generation of Safety-critical Testing Scenarios via Guided Diffusion

Junyi Chen; Naiting Zhong; Xinzheng Wu; Yong Shen

REVIEW 4 major objections 5 minor 21 references

The paper claims a vision-language model, acting as a test strategist, can direct a guided diffusion model to turn ordinary driving logs into adversarial closed-loop scenarios that raise the at-fault collision rate of autonomous planners by

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 18:54 UTC pith:7CWZA4FB

load-bearing objection Reasonable VLM-guided diffusion framework for adversarial scenario generation, but the headline 4.2x result is not attributable to the VLM and the evaluation lacks baselines and statistical power. the 4 major comments →

arxiv 2512.02844 v1 pith:7CWZA4FB submitted 2025-12-02 cs.RO cs.LG

VLM as Strategist: Adaptive Generation of Safety-critical Testing Scenarios via Guided Diffusion

Xinzheng Wu , Junyi Chen , Naiting Zhong , Yong Shen This is my paper

classification cs.RO cs.LG

keywords autonomous driving testingsafety-critical scenariosvision language modelguided diffusionscenario generationclosed-loop simulationchain-of-thought reasoningtraffic simulation

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper is trying to establish that a vision-language model (VLM) can act as the strategist in an automated safety-testing pipeline for autonomous vehicles: it examines a recording of an ordinary driving scene, decides what kind of crash is plausible, picks which surrounding vehicle should cause trouble and when, and passes those decisions to a guided diffusion model that generates the vehicle's trajectories in real time within a closed-loop simulator. The core architectural claim is a three-layer split—VLM at the strategic layer choosing goals, predefined guidance templates at the tactical layer, and guided diffusion at the operational layer—where the guidance functions are gated by a time window and trigger distance so background vehicles respond to the actions of the vehicle under test. The headline quantitative claim is that the generated scenarios increase the average at-fault collision rate of four different planning algorithms by approximately 4.2 times relative to the original scenarios, while also increasing collision rate and lowering path completion. A sympathetic reader would care because safety-critical scenarios are rare in real traffic; if this pipeline works, fleets of ordinary driving logs could be converted into targeted stress tests automatically, without hand-writing scenario scripts.

Core claim

The central claim is that VLM-directed adaptive guided diffusion generates safety-critical testing scenarios in closed-loop simulation. Concretely, the paper constructs a three-layer hierarchy. At the strategic layer, a VLM receives a bird's-eye-view image and a structured textual summary of a scenario, performs chain-of-thought reasoning in three stages (scenario understanding, risk association and objective determination, guidance-function formulation), and outputs an adversarial vehicle ID, a behavioral intention, and a concrete accident type drawn from a crash-typology database. At the tactical layer, the VLM selects and combines predefined guidance-function templates—speed, goal-point,

What carries the argument

The load-bearing mechanism is the 'VLM as strategist' loop feeding an adaptive guided diffusion sampler. The central object is the guidance function: a weighted sum of smoothness, speed, goal-point, and interaction templates, each gated by a time window [t_s, t_e] and, for interaction, by a trigger distance d_trigger. These gates are what convert a static naturalistic log into a reactive closed-loop test, because the adversarial behavior switches on only when the vehicle under test is near. The VLM chooses which templates to combine and sets all parameters—including the adversarial vehicle ID, the target speed, the goal point, and the timing—after chain-of-thought reasoning over the bird's-e

Load-bearing premise

The load-bearing premise is that the VLM's choices—which vehicle to weaponize, which guidance template to use, and how to set its time window, trigger distance, and weights—are correct and necessary; if those choices are arbitrary or poorly calibrated for a new road geometry, the reported collision-rate increase does not follow.

What would settle it

Rerun the pipeline on a larger set of scenarios (e.g., 100+ logs) with the VLM replaced by a fixed policy—say, always selecting the nearest front vehicle, a single interaction template, and fixed timing—and compare at-fault collision rates. If the fixed policy matches or exceeds the VLM's 4.2× increase, the central claim that VLM reasoning is the driver of safety-critical generation fails. A secondary check: run the same VLM-chosen guidance with the guidance strength λ set to zero; if collisions persist, the diffusion model alone, not the VLM-directed guidance, is responsible.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

If the claim holds, safety testing can be automated from existing driving logs: the VLM decides what is risky, the diffusion model makes it happen, and no expert needs to write scenario scripts.
The adaptive gates (time window and trigger distance) mean the generated scenarios are interactive: background vehicle behavior changes as the vehicle under test responds, distinguishing the approach from static replay or one-shot parameter search.
Diversity is expected: different VLMs (or the same VLM on different scenarios) select different adversarial vehicles and maneuvers, yielding a range of plausible critical scenarios rather than a fixed library.
The reported 4.2× average increase in at-fault collision rate across optimization-based, learning-based, and hybrid planners suggests the scenarios expose planner-specific weaknesses, not just forced geometric collisions.
The paper shows VLM understanding accuracy depends on both visual (BEV) and textual inputs plus chain-of-thought prompting; removing any component degrades scenario understanding, especially fine-grained vehicle identification.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the paper never shows that the VLM's choices—rather than the diffusion guidance itself—cause the 4.2× increase; a comparison against random or fixed guidance-function selection on a larger scenario set would isolate the VLM's contribution.
Editorial inference: because the guidance templates and accident typology constrain what the VLM can choose, the diversity ceiling is set by the database; expanding the typology (e.g., defensive-driving or multi-vehicle cooperative attacks) is a natural testable extension the authors name as future work.
Editorial inference: the framework's dependence on a proprietary VLM means reproducibility is tied to that model's version and behavior; an open-weight VLM or a distilled local model would make the claimed result independently checkable.
Editorial inference: the authors report one anomaly (one planner's high-risk exposure slightly decreases), which suggests the generated scenarios change the failure mode rather than uniformly increasing risk; a finer probe of failure types would refine what 'safety-critical' means here.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Reasonable VLM-guided diffusion framework for adversarial scenario generation, but the headline 4.2x result is not attributable to the VLM and the evaluation lacks baselines and statistical power.

read the letter

The framework here is a reasonable integration of known pieces: a guided diffusion backbone, predefined guidance templates with time windows and trigger distances, and a VLM that selects and parameterizes them. That last piece — the VLM-as-strategist layer — is the actual novelty, and the adaptive guidance mechanism is a modest but real contribution. The co-simulation platform and the effort to run four different planners are also useful.

The problem is the evidence. The headline claim — a 4.2x increase in at-fault collision rate — is computed on a sample of 12 scenarios per AUT, filtered to those where the AUTs originally had low collision rates. There are no confidence intervals, and there is no comparison against any prior scenario generator. The paper explicitly says it does not select baselines, which is not a justification; it is a gap. Since the interaction guidance loss directly minimizes the distance between the adversarial BV and the VUT, a higher collision rate is expected from almost any adversarial optimizer — it is close to the objective. The VLM's specific contribution is never isolated. To support the 'strategist' claim, the authors would need at least one ablation where guidance functions are chosen by a random policy or a fixed heuristic (e.g., always pick the nearest vehicle, always use interaction guidance). Table 5 only validates VLM perception accuracy on 30 hand-annotated scenarios, not whether those choices actually cause the measured increase.

There is also a statistical fragility: with 12 scenarios per cell, one percentage point is one scenario. The 4.2x figure is not something I would rely on.

What the paper does well: the methodology is clearly described, the case studies are plausible, and the framework is flexible enough that any diffusion backbone could be swapped in. It is a genuine engineering contribution, and the authors are open about their design choices. The citation pattern looks appropriate.

Who is it for: researchers working on scenario generation for ADS testing who want to see a concrete way to wire a VLM into guided diffusion. It is a useful reference, even if the current evaluation is not strong enough to validate the central claim.

Recommendation: send it to peer review. A good referee can push for the missing baselines, a larger and unfiltered evaluation, and a proper VLM ablation. The idea deserves discussion, but it needs substantially stronger evidence before it should be accepted.

Referee Report

4 major / 5 minor

Summary. The paper proposes a three-layer framework for safety-critical testing scenario generation: a strategic layer in which a VLM selects generation objectives and adversarial vehicles from BEV and textual scenario inputs; a tactical layer that maps those objectives to predefined guidance-function templates with time windows, trigger distances, and weights; and an operational layer that performs adaptive guided diffusion to generate BV trajectories in closed-loop simulation. The diffusion model is trained on nuPlan segments, and a custom co-simulation platform controls the VUT via one of four AUTs while diffusion-generated BVs are updated at 1 Hz. Experiments compare original filtered nuPlan scenarios with generated scenarios on collision, risk, and task-related metrics, reporting an average 4.2× increase in at-fault collision rate. Case studies illustrate adaptivity across scenarios and AUTs, and a VLM understanding ablation is reported on 30 hand-annotated scenarios.

Significance. If the central empirical claim were established, the framework would be a useful contribution: it demonstrates a coherent way to combine a VLM's high-level scene understanding with guided diffusion for automated adversarial scenario generation, and the adaptive time-window/trigger-distance mechanism is a sensible mechanism for making guidance reactive to VUT behavior. The paper is also commendable for providing a concrete algorithm, describing the co-simulation platform, and including closed-loop experiments with multiple planners. However, the headline quantitative claim is not yet supported by the experimental design: the VLM's contribution is not isolated from the guided-diffusion optimizer, the comparison is only against deliberately filtered original scenarios, and the statistical evidence is weak (small sample, no confidence intervals, no baseline method). The framework's novelty and architectural clarity are not in question; what needs strengthening is the causal attribution and robustness of the 4.2× result.

major comments (4)

[§4.2.3, §4.3, Table 4] The headline claim that VLM-chosen guidance increases the average at-fault collision rate by 4.2× is not attributable to the VLM. Table 4 compares generated guided-diffusion scenarios with original filtered scenarios; there is no ablation in which guidance functions are produced by a random-policy baseline, a fixed template (e.g., always choose the nearest BV and always use interaction guidance), or a text-only LLM. The measured gap therefore conflates the guided-diffusion optimizer's effect with the VLM's strategic choices. Because the VLM's perception accuracy in Table 5 is validated only on labels, not on closed-loop outcomes, the paper does not establish that VLM reasoning causes the Table 4 result. I request an ablation comparing the full VLM-directed pipeline against at least one non-VLM guidance-selection condition, reporting the same Table 4 metrics for each AUT.
[§3.3.2, Eq. (18)] Part of the reported improvement is by construction. The interaction guidance function minimizes the distance between adversarial BVs and the VUT, and Eq. (19) combines this with a gradient-descent guided sampling step. Thus, an increased collision rate is precisely what the optimizer is asked to produce, and the comparison against non-adversarial original scenarios is expected to show an increase regardless of VLM understanding. The paper should separate the effect of the objective itself from the effect of VLM-selected objectives. A minimal control condition would run the same guided diffusion with interaction guidance on a randomly chosen BV and compare that to the VLM-selected BV and guidance template.
[§4.3, Table 4] The quantitative evidence is statistically fragile. The percentages in Table 4 are consistent with 12 scenarios per AUT (8.3% = 1/12, 66.7% = 8/12, etc.), but the number of scenarios is never stated, and no confidence intervals, error bars, or significance tests are provided. With 12 scenarios, a single additional collision changes the rate by 8.3 percentage points, so the per-metric differences and the 4.2× factor are not robustly separable from sampling noise. Moreover, the denominator bias is relevant: §4.2.3 intentionally filters for low-collision original scenarios, which inflates the multiplicative ratio. Please report exact scenario counts and per-cell collision counts, and provide bootstrap or Bayesian intervals for the reported rates and the ratio.
[§4, 'this paper does not select baselines for comparison'] The absence of any comparison to prior generation methods weakens the claim that the framework 'efficiently generates' realistic, diverse, and highly interactive safety-critical scenarios. The manuscript's argument that any SOTA diffusion model can be plugged into the framework is not a substitute for an empirical baseline; at minimum, the paper should compare against a prior guided-diffusion scenario generator (e.g., CTG, DiffScene, or an LLM-attacker-style baseline) on the same metrics, or explicitly restrict the paper's claim to an ablative self-comparison. Without such a comparison, the reader cannot judge whether the proposed VLM-directed mechanism improves on existing automated adversarial-scenario generation.

minor comments (5)

[Abstract / §4.3] The 4.2× figure is ambiguous: it could be the average of per-AUT at-fault collision rate ratios or the ratio of the averaged rates. The computation should be stated explicitly, with uncertainty.
[§4.2.3 / §4.7] The VLM used is named 'Claude-3-7-Sonnet', but no exact model version/date or sampling configuration is given. Given the sensitivity of VLM outputs to model version and prompting, the reproducibility of the VLM-directed component would be improved by reporting the exact API/model identifier and temperature settings.
[§4.5, Fig. 8] The quantitative comparison of BV speed metrics across AUTs in Fig. 8 is anecdotal; it reports one scenario, not aggregate statistics. Consider reporting means over the full scenario set if this adaptivity claim is to be supported.
[§3.4.2, Table 2] The accident knowledge database is described as derived from NHTSA pre-crash typology, but Table 2 appears to be a compact mapping table. It would help to state which of the 37 typology entries were used and whether the mapping was manually curated or validated.
[General] Throughout the paper, 'the generated scenarios increase the average at-fault collision rate' is sometimes phrased as if the AUTs' performance degrades; this is intended for testing, but the framing should consistently distinguish testing criticality from a safety guarantee about the generator.

Circularity Check

2 steps flagged

The 4.2x at-fault-collision headline is partly the optimizer's own objective plus an easy baseline, not an independent prediction; the VLM attribution is an evidence gap rather than a circular step.

specific steps

fitted input called prediction [Section 3.3.2, Eq. (18); evaluated in Section 4.3, Table 4]
"The interaction guidance function is designed to encourage specified agents to continuously challenge the VUT in testing scenarios... It calculates the minimum distance between all the specified agents and the VUT over the prediction horizon... By minimizing this value, it increases the probability of close-range interactions between agents and the VUT, as formalized below: ... s_interact = - min_t min_i ||p_i_t, p_VUT_t||_2 ..."

The headline metric (collision rate / at-fault collision rate) is a close-proximity/collision outcome, and Eq. (18) is exactly the objective that Algorithm 1 optimizes during guided diffusion. The reported 4.2x increase is therefore the value of the adversarial objective the generator was instructed to maximize, renamed as an empirical validation result. The 'prediction' of higher criticality is forced by the guidance function, not independently derived.
other [Section 4.2.3, Implementation details]
"Then, scenarios where AUTs exhibit low collision rates yet contain sufficient interactions are filtered as the original scenarios. This step aims to ensure that the filtered original scenarios did not pose significant challenges to the AUTs, thereby enabling clearer comparison with subsequently generated safety-critical scenarios (which are created by modifying these original scenarios) to validate the effectiveness of the proposed method."

The baseline against which the 4.2x improvement is measured is deliberately preselected to have low collision rates and low challenge. Since the generated scenarios are then adversarially optimized to challenge the VUT, a large increase in collision metrics is partly guaranteed by the selection rule. The magnitude of the reported improvement is thus partly an artifact of the baseline-selection procedure, not solely a measured property of the generation framework.

full rationale

The core diffusion/guided-diffusion pipeline is not circular: the denoiser is trained on nuPlan data with an L1 reconstruction loss (Eq. 9) and conventional auxiliary losses, and the guided-sampling ideas are adopted from external work (Zhong et al.; Huang et al.). I find no load-bearing self-citation chain or imported uniqueness theorem; the authors' earlier papers are cited only as background. However, the paper's headline quantitative claim is partially circular in two places. First, Eq. (18) defines interaction guidance as minimizing the BV-VUT distance, and Algorithm 1 applies that gradient during sampling; the later claim of a 4.2x increase in at-fault collisions is essentially scoring the method on the objective it was explicitly optimized to achieve. Second, the 'original scenarios' baseline is deliberately filtered to have low collision rates and low challenge, so the comparison is selected to make the generated scenarios look critical. These two by-construction elements account for the direction and much of the magnitude of the headline improvement. The VLM's role is not independently isolated: Table 5 is a perception-accuracy ablation, not an end-to-end ablation with random/manual guidance, so the VLM-specific attribution is unsupported; however, that is missing evidence rather than circularity and does not itself raise the circularity score. Overall: partial circularity (6).

Axiom & Free-Parameter Ledger

8 free parameters · 6 axioms · 0 invented entities

The central claim relies on a standard diffusion framework, a domain assumption about guided sampling, a constructed accident-knowledge prior, and VLM reliability. The free parameters are mostly VLM-chosen or unreported, which makes the quantitative 4.2× result underdetermined.

free parameters (8)

Guidance strength λ = not reported
Controls the magnitude of the guidance gradient in Eq. 14; no value is given anywhere in the paper.
Guidance function weights ω_j = VLM-chosen per scenario
Selected by the VLM in §3.4.3 and used in Eq. 19; no fixed values or validation.
Time windows [t_s,j, t_e,j] and trigger distance d_trigger = VLM-chosen per scenario
Core to the claimed adaptive behavior; configured by the VLM, not learned or systematically swept.
Target speed v_target and goal point p_goal = VLM-chosen per scenario
Required by the speed and goal guidance functions (Eqs. 16-17); values come from VLM reasoning.
Loss weights λ_diff, λ_traj, λ_prob, λ_driv = 1.0, 1.0, 0.5, 0.5
Training hyperparameters in Table 1, chosen by hand.
High-risk threshold T_safety = 1 s
Defines the high-risk exposure rate metric; arbitrary threshold in §4.2.2.
Original-scenario filtering criteria = unspecified
§4.2.3 filters for 'low collision rates yet sufficient interactions' but never quantifies either criterion.
Algorithm 1 hyperparameters N_guide, K_guide_start, f_rep = not reported
Inputs to the adaptive guidance algorithm; no values are reported in the experimental setup.

axioms (6)

standard math DDPM forward/reverse process with cosine noise schedule and the standard posterior formulas (Eqs. 1-7).
Unproved background result assumed from Ho et al. 2020 and Nichol & Dhariwal 2021.
domain assumption Guidance by gradient of a differentiable guidance function on μ_k (Eq. 14) induces the desired trajectory properties.
Adopted from Zhong et al. 2022 (CTG); the paper does not prove convergence or constraint satisfaction.
domain assumption The NHTSA crash typology and Table 2's behavior-to-accident mapping are a sufficient prior for safety-critical scenario generation.
The VLM queries this fixed database to set objectives; the paper itself notes in §5 that scenario types depend on this database and future work will add defensive driving scenarios, admitting incompleteness.
domain assumption The proprietary VLM (Claude-3-7-Sonnet) produces reliable scenario understanding and guidance parameter choices.
Only standalone perception accuracy on 30 scenarios is validated (Table 5); end-to-end reliability is not demonstrated.
domain assumption A diffusion model trained on nuPlan generates realistic, drivable BV trajectories under guidance.
No feasibility check, collision-avoidance check, or fidelity metric is reported for generated trajectories.
domain assumption Trajectory-driven co-simulation with BV replanning at 1 Hz is a faithful closed-loop evaluation environment.
The custom simulator in §4.1 is not compared against nuPlan-devkit or real-world closed-loop behavior.

pith-pipeline@v1.3.0-alltime-deepseek · 23009 in / 14029 out tokens · 125060 ms · 2026-08-03T18:54:08.428376+00:00 · methodology

0 comments

read the original abstract

The safe deployment of autonomous driving systems (ADSs) relies on comprehensive testing and evaluation. However, safety-critical scenarios that can effectively expose system vulnerabilities are extremely sparse in the real world. Existing scenario generation methods face challenges in efficiently constructing long-tail scenarios that ensure fidelity, criticality, and interactivity, while particularly lacking real-time dynamic response capabilities to the vehicle under test (VUT). To address these challenges, this paper proposes a safety-critical testing scenario generation framework that integrates the high-level semantic understanding capabilities of Vision Language Models (VLMs) with the fine-grained generation capabilities of adaptive guided diffusion models. The framework establishes a three-layer hierarchical architecture comprising a strategic layer for VLM-directed scenario generation objective determination, a tactical layer for guidance function formulation, and an operational layer for guided diffusion execution. We first establish a high-quality fundamental diffusion model that learns the data distribution of real driving scenarios. Next, we design an adaptive guided diffusion method that enables real-time, precise control of background vehicles (BVs) in closed-loop simulation. The VLM is then incorporated to autonomously generate scenario generation objectives and guidance functions through deep scenario understanding and risk reasoning, ultimately guiding the diffusion model to achieve VLM-directed scenario generation. Experimental results demonstrate that the proposed method can efficiently generate realistic, diverse, and highly interactive safety-critical testing scenarios. Furthermore, case studies validate the adaptability and VLM-directed generation performance of the proposed method.

Figures

Figures reproduced from arXiv: 2512.02844 by Junyi Chen, Naiting Zhong, Xinzheng Wu, Yong Shen.

**Figure 2.** Figure 2: Structure of the fundamental diffusion model. embeddings, a shared multilayer perceptron (MLP) network is utilized to extracts temporal dynamic features along the time dimension and outputs 𝒆̂𝑎𝑔𝑒𝑛𝑡. For map lane markings, each point is first encoded using an MLP, followed by max-pooling along the point sequence dimension to obtain 𝒆̂𝑚𝑎𝑝. For traffic signal information, an MLP is also applied to generate 𝒆̂… view at source ↗

**Figure 3.** Figure 3: An example of the VLM input information. Xinzheng Wu et al.: Preprint submitted to Elsevier Page 12 of 25 [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗

**Figure 4.** Figure 4: The prompts for each CoT reasoning step. Xinzheng Wu et al.: Preprint submitted to Elsevier Page 14 of 25 [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗

**Figure 5.** Figure 5: The co-simulation platform. 4.2. Experimental setup 4.2.1. Algorithms under test (AUTs) To comprehensively evaluate the performance of the generated safety-critical testing scenarios, this paper selects multiple representative autonomous driving D&P algorithms for testing. These algorithms cover diverse design concepts and technical approaches, aiming to verify the adaptability of our proposed method acros… view at source ↗

**Figure 6.** Figure 6: Examples of the generation process and results for safety-critical testing scenarios under different initial conditions. different initial conditions. For this and subsequent case studies, PDM-Hybrid is adopted as the AUT to control the SV unless otherwise specified. To provide a comprehensive demonstration of the entire generation process corresponding to the three-layer hierarchical framework in [PITH_F… view at source ↗

**Figure 7.** Figure 7: Examples of safety-critical scenarios generated for different AUTs under identical scenario conditions [PITH_FULL_IMAGE:figures/full_fig_p020_7.png] view at source ↗

**Figure 8.** Figure 8: Comparative analysis of adversarial vehicle speed metrics across different AUTs. Xinzheng Wu et al.: Preprint submitted to Elsevier Page 20 of 25 [PITH_FULL_IMAGE:figures/full_fig_p020_8.png] view at source ↗

**Figure 9.** Figure 9: Examples of generating safety-critical scenarios using different VLMs under identical initial conditions and with the same AUT. Xinzheng Wu et al.: Preprint submitted to Elsevier Page 21 of 25 [PITH_FULL_IMAGE:figures/full_fig_p021_9.png] view at source ↗

discussion (0)

Reference graph

Works this paper leans on

21 extracted references · 3 canonical work pages

[5]

IEEE Transactions on Intelligent Transportation Systems 24, 14393–14405

An adaptive multi-fidelity sampling framework for safety analysis of connected and automated vehicles. IEEE Transactions on Intelligent Transportation Systems 24, 14393–14405. doi:10.1109/TITS.2023.3296716. Hallgarten,M.,Stoll,M.,Zell,A.,2023. Frompredictiontoplanningwithgoalconditionedlanegraphtraversals,in:2023IEEE26thInternational Conference on Intelli...

arXiv 2023
[7]

Journal of Computer Science and Technology 39, 509–541

A survey of multimodal controllable diffusion models. Journal of Computer Science and Technology 39, 509–541. doi:10.1007/s11390-024-3814-0. Li, C., Sifakis, J., Wang, Q., Yan, R., Zhang, J.,

work page doi:10.1007/s11390-024-3814-0
[8]

Simulation-based validation for autonomous driving systems, in: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, Association for Computing Machinery, New York, NY, USA. pp. 842–853. doi:10.1145/3597926.3598100. Li,N.,Yao,Y.,Kolmanovsky,I.,Atkins,E.,Girard,A.R.,2022. Game-theoreticmodelingofmulti-vehicleinteract...

arXiv 2022
[9]

IEEETransactionsonIntelligentVehicles9,7138–7164

A survey on autonomous driving datasets: Statistics,annotationquality,andafutureoutlook. IEEETransactionsonIntelligentVehicles9,7138–7164. doi:10.1109/TIV.2024.3394735. Lu, J., Azam, S., Alcan, G., Kyrki, V., 2024a. Data-driven diffusion models for enhancing safety in autonomous vehicle traffic simulations. doi:10.48550/arXiv.2410.04809,arXiv:2410.04809. ...

arXiv 2024
[10]

Accessed: 2025-11-05

Chapter 8: Defensive driving.https://dmv.ny.gov/about-dmv/ chapter-8-defensive-driving. Accessed: 2025-11-05. Nichol, A.Q., Dhariwal, P.,

2025
[13]

Urban driver: Learning to drive from real-world demonstrations using policy gradients, in: Proceedings of the 5th Conference on Robot Learning, PMLR. pp. 718–728. Sun,J.,Zhang,H.,Zhou,H.,Yu,R.,Tian,Y.,2022. Scenario-basedtestautomationforhighlyautomatedvehicles:Areviewandpavingthewayfor systematic safety assurance. IEEE Transactions on Intelligent Transpo...

arXiv 2022
[15]

Automotive Innovation doi:10.1007/s42154-023-00279-4

Accelerated testing and evaluation of autonomous vehicles based on dual surrogates. Automotive Innovation doi:10.1007/s42154-023-00279-4. Wu, X., Chen, J., Wu, J., Zhang, L., Xia, T., Shen, Y.,

work page doi:10.1007/s42154-023-00279-4
[17]

Transportation Research Part C: Emerging Technologies 179, 105256

Adaptive safety performance testing for autonomous vehicles with adaptive importance sampling. Transportation Research Part C: Emerging Technologies 179, 105256. doi:10.1016/j.trc.2025.105256. Yang, X., Wen, L., Ma, Y., Mei, J., Li, X., Wei, T., Lei, W., Fu, D., Cai, P., Dou, M., Shi, B., He, L., Liu, Y., Qiao, Y., 2024a. Drivearena: A closed-loop generat...

arXiv 2025
[18]

doi:10.48550/arXiv.2503.05808,arXiv:2503.05808

Drivegen: Towards infinite diverse traffic scenarios with large models. doi:10.48550/arXiv.2503.05808,arXiv:2503.05808. Zhao, Y., Xiao, W., Mihalj, T., Hu, J., Eichberger, A.,

Pith/arXiv arXiv doi:10.48550/arxiv.2503.05808
[19]

Chat2scenario: Scenario extraction from dataset through utilization of large language model, in: 2024 IEEE Intelligent Vehicles Symposium (IV), pp. 559–566. doi:10.1109/IV55156.2024.10588843. Zhong, Z., Rempe, D., Chen, Y., Ivanovic, B., Cao, Y., Xu, D., Pavone, M., Ray, B.,

arXiv 2024
[20]

Language-guided traffic simulation via scene-level diffusion, in: Proceedings of The 7th Conference on Robot Learning, PMLR. pp. 144–177. Zhong,Z.,Rempe,D.,Xu,D.,Chen,Y.,Veer,S.,Che,T.,Ray,B.,Pavone,M.,2022. Guidedconditionaldiffusionforcontrollabletrafficsimulation. doi:10.48550/arXiv.2210.17366,arXiv:2210.17366. Zhou, Y., Lin, G., Tang, Y., Yang, K., Ji...

Pith/arXiv arXiv doi:10.48550/arxiv.2210.17366 2022
[21]

doi:10.1109/ICRA48891.2023.10160868

Flyover: A model-driven method to generate diverse highwayinterchangesforautonomousvehicletesting,in:2023IEEEInternationalConferenceonRoboticsandAutomation(ICRA),pp.11389– 11395. doi:10.1109/ICRA48891.2023.10160868. Zhu,Y.,Chen,J.,Liang,L.,Kuang,Z.,Jin,L.,Zhang,W.,2021. Fouriercontourembeddingforarbitrary-shapedtextdetection,in:2021IEEE/CVF Conference on ...

arXiv 2023
[748]

Gong, X., Feng, S., Pan, Y.,

doi:10.1038/s41467-021-21007-8. Gong, X., Feng, S., Pan, Y.,

work page doi:10.1038/s41467-021-21007-8
[2000]

Physical Review E 62, 1805–1824

Congested traffic states in empirical observations and microscopic simulations. Physical Review E 62, 1805–1824. doi:10.1103/PhysRevE.62.1805. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q.V., Zhou, D.,

doi:10.1103/physreve.62.1805
[2016]

IEEE Intelligent Transportation Systems Magazine 8, 68–78

If, when, and how to perform lane change maneuvers on highways. IEEE Intelligent Transportation Systems Magazine 8, 68–78. doi:10.1109/MITS.2016.2565718. Peng, M., Chen, K., Guo, X., Zhang, Q., Zhong, H., Zhu, M., Yang, H., 2025a. Diffusion models for intelligent transportation systems: A survey. doi:10.48550/arXiv.2409.15816,arXiv:2409.15816. Peng,M.,Xie...

arXiv 2016
[2020]

Denoising diffusion probabilistic models, in: Advances in Neural Information Processing Systems, Curran Associates, Inc.. pp. 6840–6851. Huang, Z., Karkus, P., Ivanovic, B., Chen, Y., Pavone, M., Lv, C., 2024a. Dtpp: Differentiable joint conditional prediction and cost evaluation for tree policy planning in autonomous driving, in: 2024 IEEE International ...

arXiv 2024
[2022]

doi:10.48550/arXiv.2106.11810,arXiv:2106.11810

Nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles. doi:10.48550/arXiv.2106.11810,arXiv:2106.11810. Chen,D.,Zhu,M.,Yang,H.,Wang,X.,Wang,Y.,2024. Data-driventrafficsimulation:Acomprehensivereview. IEEETransactionsonIntelligent Vehicles 9, 4730–4748. doi:10.1109/TIV.2024.3367919. Cui, C., Ma, Y., Cao, X., Ye, W., Zhou, Y., Liang, K., ...

Pith/arXiv arXiv doi:10.48550/arxiv.2106.11810 2024
[2023]

Nature 615, 620–627

Dense reinforcement learning for safety validation of autonomous vehicles. Nature 615, 620–627. doi:10.1038/s41586-023-05732-2. Feng, S., Yan, X., Sun, H., Feng, Y., Liu, H.X.,

doi:10.1038/s41586-023-05732-2
[2024]

A survey on multimodal large language models for autonomous driving, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 958–979. Xinzheng Wu et al.:Preprint submitted to ElsevierPage 23 of 25 Dauner,D.,Hallgarten,M.,Geiger,A.,Chitta,K.,2023. Partingwithmisconceptionsaboutlearning-basedvehiclemotionplanning,in:Proceed...

arXiv 2023
[2025]

Transportation Research Part C: Emerging Technologies 174, 105106

Intelligent testing environment generation for autonomous vehicles with implicit distributions of traffic behaviors. Transportation Research Part C: Emerging Technologies 174, 105106. doi:10.1016/j.trc.2025.105106. Scheel, O., Bergamini, L., Wolczyk, M., Osiński, B., Ondruska, P.,

arXiv 2025
[2026]

Accident Analysis & Prevention 224, 108280

Make full use of testing information: An integrated accelerated testing and evaluation method for autonomous driving systems. Accident Analysis & Prevention 224, 108280. doi:10.1016/j.aap.2025.108280. Wu,X.,Chen,J.,Ye,S.,Jiang,W.,Shen,Y.,2025. Anevolvingscenariogenerationmethodbasedondual-modaldrivermodeltrainedbymulti-agent reinforcement learning. doi:10...

arXiv 2025

[1] [5]

IEEE Transactions on Intelligent Transportation Systems 24, 14393–14405

An adaptive multi-fidelity sampling framework for safety analysis of connected and automated vehicles. IEEE Transactions on Intelligent Transportation Systems 24, 14393–14405. doi:10.1109/TITS.2023.3296716. Hallgarten,M.,Stoll,M.,Zell,A.,2023. Frompredictiontoplanningwithgoalconditionedlanegraphtraversals,in:2023IEEE26thInternational Conference on Intelli...

arXiv 2023

[2] [7]

Journal of Computer Science and Technology 39, 509–541

A survey of multimodal controllable diffusion models. Journal of Computer Science and Technology 39, 509–541. doi:10.1007/s11390-024-3814-0. Li, C., Sifakis, J., Wang, Q., Yan, R., Zhang, J.,

work page doi:10.1007/s11390-024-3814-0

[3] [8]

Simulation-based validation for autonomous driving systems, in: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, Association for Computing Machinery, New York, NY, USA. pp. 842–853. doi:10.1145/3597926.3598100. Li,N.,Yao,Y.,Kolmanovsky,I.,Atkins,E.,Girard,A.R.,2022. Game-theoreticmodelingofmulti-vehicleinteract...

arXiv 2022

[4] [9]

IEEETransactionsonIntelligentVehicles9,7138–7164

A survey on autonomous driving datasets: Statistics,annotationquality,andafutureoutlook. IEEETransactionsonIntelligentVehicles9,7138–7164. doi:10.1109/TIV.2024.3394735. Lu, J., Azam, S., Alcan, G., Kyrki, V., 2024a. Data-driven diffusion models for enhancing safety in autonomous vehicle traffic simulations. doi:10.48550/arXiv.2410.04809,arXiv:2410.04809. ...

arXiv 2024

[5] [10]

Accessed: 2025-11-05

Chapter 8: Defensive driving.https://dmv.ny.gov/about-dmv/ chapter-8-defensive-driving. Accessed: 2025-11-05. Nichol, A.Q., Dhariwal, P.,

2025

[6] [13]

Urban driver: Learning to drive from real-world demonstrations using policy gradients, in: Proceedings of the 5th Conference on Robot Learning, PMLR. pp. 718–728. Sun,J.,Zhang,H.,Zhou,H.,Yu,R.,Tian,Y.,2022. Scenario-basedtestautomationforhighlyautomatedvehicles:Areviewandpavingthewayfor systematic safety assurance. IEEE Transactions on Intelligent Transpo...

arXiv 2022

[7] [15]

Automotive Innovation doi:10.1007/s42154-023-00279-4

Accelerated testing and evaluation of autonomous vehicles based on dual surrogates. Automotive Innovation doi:10.1007/s42154-023-00279-4. Wu, X., Chen, J., Wu, J., Zhang, L., Xia, T., Shen, Y.,

work page doi:10.1007/s42154-023-00279-4

[8] [17]

Transportation Research Part C: Emerging Technologies 179, 105256

Adaptive safety performance testing for autonomous vehicles with adaptive importance sampling. Transportation Research Part C: Emerging Technologies 179, 105256. doi:10.1016/j.trc.2025.105256. Yang, X., Wen, L., Ma, Y., Mei, J., Li, X., Wei, T., Lei, W., Fu, D., Cai, P., Dou, M., Shi, B., He, L., Liu, Y., Qiao, Y., 2024a. Drivearena: A closed-loop generat...

arXiv 2025

[9] [18]

doi:10.48550/arXiv.2503.05808,arXiv:2503.05808

Drivegen: Towards infinite diverse traffic scenarios with large models. doi:10.48550/arXiv.2503.05808,arXiv:2503.05808. Zhao, Y., Xiao, W., Mihalj, T., Hu, J., Eichberger, A.,

Pith/arXiv arXiv doi:10.48550/arxiv.2503.05808

[10] [19]

Chat2scenario: Scenario extraction from dataset through utilization of large language model, in: 2024 IEEE Intelligent Vehicles Symposium (IV), pp. 559–566. doi:10.1109/IV55156.2024.10588843. Zhong, Z., Rempe, D., Chen, Y., Ivanovic, B., Cao, Y., Xu, D., Pavone, M., Ray, B.,

arXiv 2024

[11] [20]

Language-guided traffic simulation via scene-level diffusion, in: Proceedings of The 7th Conference on Robot Learning, PMLR. pp. 144–177. Zhong,Z.,Rempe,D.,Xu,D.,Chen,Y.,Veer,S.,Che,T.,Ray,B.,Pavone,M.,2022. Guidedconditionaldiffusionforcontrollabletrafficsimulation. doi:10.48550/arXiv.2210.17366,arXiv:2210.17366. Zhou, Y., Lin, G., Tang, Y., Yang, K., Ji...

Pith/arXiv arXiv doi:10.48550/arxiv.2210.17366 2022

[12] [21]

doi:10.1109/ICRA48891.2023.10160868

Flyover: A model-driven method to generate diverse highwayinterchangesforautonomousvehicletesting,in:2023IEEEInternationalConferenceonRoboticsandAutomation(ICRA),pp.11389– 11395. doi:10.1109/ICRA48891.2023.10160868. Zhu,Y.,Chen,J.,Liang,L.,Kuang,Z.,Jin,L.,Zhang,W.,2021. Fouriercontourembeddingforarbitrary-shapedtextdetection,in:2021IEEE/CVF Conference on ...

arXiv 2023

[13] [748]

Gong, X., Feng, S., Pan, Y.,

doi:10.1038/s41467-021-21007-8. Gong, X., Feng, S., Pan, Y.,

work page doi:10.1038/s41467-021-21007-8

[14] [2000]

Physical Review E 62, 1805–1824

Congested traffic states in empirical observations and microscopic simulations. Physical Review E 62, 1805–1824. doi:10.1103/PhysRevE.62.1805. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q.V., Zhou, D.,

doi:10.1103/physreve.62.1805

[15] [2016]

IEEE Intelligent Transportation Systems Magazine 8, 68–78

If, when, and how to perform lane change maneuvers on highways. IEEE Intelligent Transportation Systems Magazine 8, 68–78. doi:10.1109/MITS.2016.2565718. Peng, M., Chen, K., Guo, X., Zhang, Q., Zhong, H., Zhu, M., Yang, H., 2025a. Diffusion models for intelligent transportation systems: A survey. doi:10.48550/arXiv.2409.15816,arXiv:2409.15816. Peng,M.,Xie...

arXiv 2016

[16] [2020]

Denoising diffusion probabilistic models, in: Advances in Neural Information Processing Systems, Curran Associates, Inc.. pp. 6840–6851. Huang, Z., Karkus, P., Ivanovic, B., Chen, Y., Pavone, M., Lv, C., 2024a. Dtpp: Differentiable joint conditional prediction and cost evaluation for tree policy planning in autonomous driving, in: 2024 IEEE International ...

arXiv 2024

[17] [2022]

doi:10.48550/arXiv.2106.11810,arXiv:2106.11810

Nuplan: A closed-loop ml-based planning benchmark for autonomous vehicles. doi:10.48550/arXiv.2106.11810,arXiv:2106.11810. Chen,D.,Zhu,M.,Yang,H.,Wang,X.,Wang,Y.,2024. Data-driventrafficsimulation:Acomprehensivereview. IEEETransactionsonIntelligent Vehicles 9, 4730–4748. doi:10.1109/TIV.2024.3367919. Cui, C., Ma, Y., Cao, X., Ye, W., Zhou, Y., Liang, K., ...

Pith/arXiv arXiv doi:10.48550/arxiv.2106.11810 2024

[18] [2023]

Nature 615, 620–627

Dense reinforcement learning for safety validation of autonomous vehicles. Nature 615, 620–627. doi:10.1038/s41586-023-05732-2. Feng, S., Yan, X., Sun, H., Feng, Y., Liu, H.X.,

doi:10.1038/s41586-023-05732-2

[19] [2024]

A survey on multimodal large language models for autonomous driving, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 958–979. Xinzheng Wu et al.:Preprint submitted to ElsevierPage 23 of 25 Dauner,D.,Hallgarten,M.,Geiger,A.,Chitta,K.,2023. Partingwithmisconceptionsaboutlearning-basedvehiclemotionplanning,in:Proceed...

arXiv 2023

[20] [2025]

Transportation Research Part C: Emerging Technologies 174, 105106

Intelligent testing environment generation for autonomous vehicles with implicit distributions of traffic behaviors. Transportation Research Part C: Emerging Technologies 174, 105106. doi:10.1016/j.trc.2025.105106. Scheel, O., Bergamini, L., Wolczyk, M., Osiński, B., Ondruska, P.,

arXiv 2025

[21] [2026]

Accident Analysis & Prevention 224, 108280

Make full use of testing information: An integrated accelerated testing and evaluation method for autonomous driving systems. Accident Analysis & Prevention 224, 108280. doi:10.1016/j.aap.2025.108280. Wu,X.,Chen,J.,Ye,S.,Jiang,W.,Shen,Y.,2025. Anevolvingscenariogenerationmethodbasedondual-modaldrivermodeltrainedbymulti-agent reinforcement learning. doi:10...

arXiv 2025