Pith. sign in

REVIEW 3 major objections 4 minor 58 references

Generating Critical Scenarios for Testing Automated Driving Systems

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read AVASTRA claims that a reinforcement-learning agent which represents both the internal states of the ADS (localization, perception, control) and external environmental states generates 30–115% more collision scenarios than DeepCollision…

desk verdict AVASTRA is a solid extension of DeepCollision with a genuine internal-state contribution, but the headline collision advantage is undercut by unquantified exclusion rules and missing variance. read the letter →

arxiv 2412.02574 v1 pith:45WENRU4 submitted 2024-12-03 cs.RO cs.AIcs.SE

Trung-Hieu Nguyen , Truong-Giang Vuong , Hong-Nam Duong , Son Nguyen , Hieu Dinh Vo , Toshiaki Aoki , Thu-Trang Nguyen This is my paper · ORCID
classification cs.ROcs.AIcs.SE
keywords autonomousdrivingsystemscriticalscenariogenerationreinforcementlearningcollisionsimulation-basedtestingDoubleDeepQ-Networkinternalandexternalstatesheuristicconstraints
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Testing autonomous driving systems on real roads is expensive and risky, so the industry relies on simulators that can be configured into dangerous scenarios. This paper tries to establish that a reinforcement-learning agent generates more of those safety-critical scenarios when it is given a holistic view of the situation: both the internal state of the ADS (localization error, perception error, and control signals) and the external environment (weather, time of day, traffic flow, road condition). The authors build AVASTRA, which trains a Double Deep-Q Network to set weather, time, and the placement and behavior of NPC vehicles and pedestrians, subject to heuristic realism constraints. Across four road configurations in the LGSVL simulator with Apollo 7.0, they report 30–115% more collisions than the prior RL method DeepCollision and up to 275% more than random search. If these numbers hold, simulation-based testing can uncover more ADS failure modes before deployment.

What carries the argument

The machinery is a 19-variable state representation (seven external plus twelve internal variables) feeding a Double Deep-Q Network (a deep Q-learning variant with separate selection and evaluation networks to reduce overestimation) whose 45 actions configure the simulator. The external variables follow the JAMA safety-evaluation framework's perception disturbance factors; the internal variables turn ADS component health into numbers by measuring localization deviation, perception discrepancies for nearby objects, and actual control signals, with prediction and planning represented indirectly through their inputs and outputs. The action space is kept tractable and realistic by three heuristic constraints: chronological time progression, distance-dependent spawn positions for NPC vehicles and pedestrians (a weighted function of distance mode, vehicle size, and speed), and behavior restrictions for pedestrians. The reward is a collision probability $ProC$ in $[0,1]$, computed from longitudinal and lateral safety distances versus current distance, with $R_{col}$ for actual collisions. This combination is what the paper credits for learning to place the AV in dangerous but plausible situations.

What would settle it

Compare both methods on the same four roads with a single pre-registered collision-counting rule that reports raw collision events before any exclusion of subjective collisions, and also report the number of excluded events per method; if DeepCollision's raw or post-filter counts are not below AVASTRA's by the same margins, the claimed 30–115% advantage is a measurement artifact.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that configuring a driving simulator can be learned as a Markov decision process whose state is a 19-variable tuple: seven external state variables (weather condition, time of day, nearest traffic light, number of obstacles, distance to nearest obstacle, speed of nearest obstacle, road condition) and twelve internal state variables (localization distance and angle deviation, perception position/direction/size/velocity differences, and control values throttle, steerRate, steerTarget, accRate, brakePercentage, speed). The RL agent acts by choosing among 45 actions that change the time of day, set a weather phenomenon and intensity, or spawn an NPC vehicle or pedestrian with a specific behavior, distance, speed, and lane. AVASTRA trains a DDQN with prioritized experience replay, rewarding actions that raise a collision probability computed from longitudinal and lateral safety distances. In the evaluation, AVASTRA's #Collisions are 12.0, 18.9, 11.6, and 13.4 on the four roads, versus DeepCollision's 7.6, 12.6, 5.4, and 10.4 and Random Search's 3.2, 5.4, 5.2, and 7.0, which the paper reports as 30–115% and up to 275% improvements. The paper also claims the full state space beats internal-only or external-only variants, that all three action groups contribute, that heuristic constraints double the collision count and halve subjective collisions, and that training on all four roads transfers best to a fixed test road.

Load-bearing premise

The paper's headline improvement depends on the rules it uses to count collisions—excluding 'subjective' collisions that an ADS cannot avoid and counting only the first collision in a repeated sequence—and those rules must be applied identically to AVASTRA and both baselines for the 30–115% and 275% numbers to be a fair comparison.

Editorial extensions

If this is right

  • If AVASTRA's central claim is correct, ADS testing can move more of the search for failure-inducing scenarios from expensive, risky on-road testing into simulation.
  • The 30–115% margin over DeepCollision implies that adding internal ADS states (localization, perception, control) to the RL observation is a genuine lever, not just extra bookkeeping.
  • The heuristic constraints' effect—doubling collisions and halving subjective collisions—suggests that realism filters can improve both yield and relevance rather than merely suppress candidates.
  • The RQ4 result that comprehensive-road training beats same-road and cross-road training implies scenario generators should be trained on diverse maps even when the target road is fixed.
  • The mixed TTC results imply the method trades some time-to-collision speed for a higher volume of collisions; scenario-generation efficiency has at least two dimensions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported advantage may depend on the Section 5.2.3 collision-counting conventions; re-running DeepCollision with the same raw-event counting and the same subjective-collision exclusion pipeline could shrink or widen the 30–115% range.
  • Because the state tuple and collision-probability equations are simulator-agnostic, the same design could be dropped into CARLA or AirSim; the main unknowns are the action-space mapping and whether those simulators expose comparable internal ADS errors.
  • A testable extension is to keep the state and action spaces fixed but swap DDQN for an on-policy or actor-critic algorithm; if gains persist, the state design rather than the specific RL algorithm is doing the work.
  • The authors' own median example (Fig. 4) suggests adding lane-structure awareness, such as medians and road boundaries, to the state or action constraints could remove false-risk configurations and push the collision yield higher on roads with physical separation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes AVASTRA, a reinforcement-learning approach for generating critical driving scenarios that stress-test an autonomous driving system (ADS) in the LGSVL simulator with Apollo 7.0. AVASTRA represents the environment through both external factors (weather, time of day, traffic, road condition) and internal ADS states (localization error, perception error, control parameters), uses a 45-action space to configure those factors, and applies heuristic constraints to keep scenarios realistic. The authors compare AVASTRA against DeepCollision and Random Search on four road configurations, reporting that AVASTRA generates 30% to 115% more collisions than DeepCollision and up to 275% more than Random Search. They also conduct component, parameter, and sensitivity analyses. The core claim is that combining internal and external state awareness with realism constraints improves the discovery of safety-critical scenarios.

Significance. If the claimed advantage is real, AVASTRA is a useful contribution to simulation-based ADS testing: it is grounded in actual simulator collisions rather than a reward proxy, it uses a publicly available implementation, and its ablations give insight into the value of internal states, action groups, and heuristic constraints. The four-road evaluation across two maps and the concrete scenario examples are strengths. However, the headline result is not yet verifiable from the reported data because the collision-counting conventions are not quantified per method and no variance or significance information is provided. The tuning of distance constants and RL hyperparameters on the evaluation roads also makes the comparison vulnerable to overfitting. These issues are addressable with additional reporting and re-analysis rather than being fundamental flaws.

major comments (3)
  1. [Sec. 5.2.3 and Table 5] The #Collisions metric counts only first collisions and excludes 'subjective collisions,' but the paper reports neither the raw collision counts nor the number of excluded events per method per road, and it does not operationalize the 'beyond ADS control' or 'safety zone' judgments. This matters because Table 7 shows that AVASTRA's constraints reduce subjective collisions from 4.4 to 2.2, while DeepCollision, by the authors' own characterization, does not enforce such realism and may therefore produce a larger share of the excluded categories. Without per-method, per-road exclusion counts and a repeatable exclusion rule, the 30–115% improvement in Table 5 could be a measurement artifact rather than a genuine difference in scenario-generation ability.
  2. [Sec. 5.2.2 and Table 5] All headline results are reported as means over five seeds and 15 episodes with no error bars, confidence intervals, or significance tests. The Road 4 gap (10.4 vs 13.4 collisions) is especially likely to fall within seed-to-seed noise, so the claim that AVASTRA 'consistently outperforms' across all roads is not statistically supported. Please report per-seed distributions or standard deviations and, where appropriate, a paired significance test, or justify why such statistics are unnecessary.
  3. [Sec. 3.2, Eq. (7), and Sec. 6.3] The distance constants fd, fv, fs and the weights α, β, γ in Eq. (7) are described as 'empirically determined via various experiments,' and the RL hyperparameters OTP, Rcol, and θ_epsilon are then selected through the experiments in Figs. 7–9, without specifying which road(s) those parameter studies used. If the same four roads were used for both tuning and the final comparison, the reported advantage over the baselines may partly reflect tuning to those specific test roads. Please state the road(s) used in the parameter analyses and assess the sensitivity of the main comparison to the chosen values.
minor comments (4)
  1. [Fig. 3] The subfigure captions are out of order: (c) labels Road 4 and (d) labels Road 3. Please reorder the panels or captions so that the road numbering is consistent.
  2. [Sec. 6.5] There are several typos, including 'Apolo' for 'Apollo' and 'migrate' for 'mitigate.' In Sec. 6.1, 'versa vise' should be 'vice versa.'
  3. [Sec. 6.1] The discussion of TTC is cursory: the paper states that lower TTC is better, then reports AVASTRA with higher TTC than DeepCollision on Roads 1 and 3 without explaining why a method that generates more collisions can take longer to trigger them. A brief explanation would help readers interpret the TTC metric when collision counts differ.
  4. [Sec. 3.2] The paragraph on weather pattern constraints says the authors 'do not consider weather constraints in this paper' and defer them to future work. This is a clear statement of a limitation and should be acknowledged in the threats-to-validity section as well.

Circularity Check

1 steps flagged · score 4.0 of 10

The headline #Collisions advantage is partly inscribed in the metric: AVASTRA's realism constraints and the evaluation's exclusion of 'subjective collisions' encode the same judgment, creating a partial self-definitional circularity; no other load-bearing circularity is found.

  1. self definitional [Sec. 3.2 (Traffic Participant Constraints) and Sec. 5.2.3 (Metrics), with Table 7]
    "However, obstacles should not be introduced too abruptly or closely so that a collision becomes unavoidable regardless of the ADS’s capabilities. ... Excluding subjective collisions: These are unavoidable collisions caused by environmental configurations beyond the handling capability of the ADS. ... Sudden obstacles in the safety zone: If an object, like a pedestrian or vehicle, suddenly appears within the ADS safety zone, it will likely result in an unavoidable collision regardless of the ADS’s capabilities."

    AVASTRA's 'realistic' obstacle placement constraint is the same condition the metric excludes: both label sudden or too-close obstacles as unavoidable and beyond ADS control. The metric then discards the collision class that AVASTRA's constraints are designed to prevent, while DeepCollision does not enforce those constraints. Table 7 shows the constraints halve subjective collisions (4.4 to 2.2), but per-method excluded counts for the Table 5 comparison are never reported. The 30–115% advantage is therefore partly inscribed in the scoring rule rather than being a pure measure of finding ADS-controllable collisions.

full rationale

The paper's core result is an empirical RL comparison, not a formal derivation, and the reward function (Eq. 8) and #Collisions metric legitimately use the same collision/probability signal as the training objective; that alignment is a standard RL setup, not circularity. No self-citation chain or imported uniqueness theorem is load-bearing, and the safety-distance formulae are adopted from external prior work. The one genuine circularity is partial and localized: the exclusion of 'subjective collisions' in Sec. 5.2.3 mirrors the realism constraint that AVASTRA is explicitly engineered to satisfy, so the baseline comparison is partly self-confirming unless per-method exclusion counts are reported. Hyperparameter choices (OTP, Rcol, θϵ) are tuned on the same roads, which is an overfitting threat but not a by-construction equivalence, so I do not count it as a separate circular step. Overall score 4: the central comparison retains independent empirical content, but one scoring rule is aligned with the method's own design.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The method rests on several hand-set parameters for obstacle placement, a geometric collision-probability proxy for rewards, and subjective exclusion rules for the evaluation metric. These are modeling choices rather than independently established facts, and they should be audited before the reported gains are fully trusted.

free parameters (7)
  • alpha, beta, gamma weights in Eq. 7 = alpha=beta=gamma=1
    Weights for the obstacle-distance formula are set equal by hand, with no sensitivity analysis.
  • fd(dist_mode) base distances = fd(Far)=50m, fd(Near)=12m
    Chosen base distances for spawning obstacles in Near and Far modes; these values affect scenario challenge.
  • fv(size) distance adjustment = fv(Small)=0m, fv(Large)=3m
    Empirically assigned extra distance for large vehicles such as BoxTruck and SchoolBus.
  • fs(speed) distance adjustment = Not specified in paper, listed on website
    Empirically determined speed-dependent extra distance; central to obstacle placement realism.
  • OTP (observation-time period) = 6 seconds
    Selected as the best value in parameter analysis on the same evaluation setup.
  • Rcol (collision reward) = 7.5
    Chosen after sweeping 1.0, 2.5, 7.5, 10.0 and picking the value with the highest collision count.
  • theta_epsilon (exploration threshold) = 0.2
    Reported as the peak in the epsilon analysis; the value is tuned on the test setup.
assumptions (6)
  • domain assumption AVs can react almost instantaneously, so reaction time is disregarded in safety-distance calculation.
    Invoked in Sec. 4.2.1 to simplify Eq. 9; may underestimate required safety distance and inflate collision probability.
  • domain assumption The Berkeley safety-distance model with deceleration -6 m/s2 and Rmin=5m applies to the LGSVL/Apollo environment.
    Eqs. 9-10 use these default settings without validating them for the specific simulator and ADS.
  • ad hoc to paper The collision-probability formula ProC (Eqs. 12-16) is a valid monotone proxy for actual collision risk in the reward function.
    ProC is derived from geometric distance ratios and not empirically calibrated as a probability.
  • ad hoc to paper Obstacles whose trajectories intersect the ego vehicle, placed at distances given by Eq. 7, produce realistic and fair test scenarios.
    The trajectory-intersection heuristic and distance formula are specific to this paper and their constants are hand-tuned.
  • domain assumption The JAMA safety evaluation framework provides a sufficient basis for selecting external state variables.
    Used in Sec. 3.1.1 to justify weather, time, traffic, and road conditions as the external state space.
  • domain assumption 'Subjective collisions' (sudden obstacles in the safety zone, side/rear-end collisions) can be reliably identified and excluded without biasing method comparison.
    Sec. 5.2.3 relies on manual judgment to classify which collisions are unavoidable and should not be counted.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Generating Critical Scenarios for Testing Automated Driving Systems." pith.science (2026). https://pith.science/paper/45WENRU4

@misc{pith2026241202574,
  author       = {Pith},
  title        = {Pith review of: Generating Critical Scenarios for Testing Automated Driving Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/45WENRU4}},
  note         = {Machine review of arXiv:2412.02574}
}
read the original abstract

Autonomous vehicles (AVs) have demonstrated significant potential in revolutionizing transportation, yet ensuring their safety and reliability remains a critical challenge, especially when exposed to dynamic and unpredictable environments. Real-world testing of an Autonomous Driving System (ADS) is both expensive and risky, making simulation-based testing a preferred approach. In this paper, we propose AVASTRA, a Reinforcement Learning (RL)-based approach to generate realistic critical scenarios for testing ADSs in simulation environments. To capture the complexity of driving scenarios, AVASTRA comprehensively represents the environment by both the internal states of an ADS under-test (e.g., the status of the ADS's core components, speed, or acceleration) and the external states of the surrounding factors in the simulation environment (e.g., weather, traffic flow, or road condition). AVASTRA trains the RL agent to effectively configure the simulation environment that places the AV in dangerous situations and potentially leads it to collisions. We introduce a diverse set of actions that allows the RL agent to systematically configure both environmental conditions and traffic participants. Additionally, based on established safety requirements, we enforce heuristic constraints to ensure the realism and relevance of the generated test scenarios. AVASTRA is evaluated on two popular simulation maps with four different road configurations. Our results show AVASTRA's ability to outperform the state-of-the-art approach by generating 30% to 115% more collision scenarios. Compared to the baseline based on Random Search, AVASTRA achieves up to 275% better performance. These results highlight the effectiveness of AVASTRA in enhancing the safety testing of AVs through realistic comprehensive critical scenario generation.

Figures

Figures reproduced from arXiv: 2412.02574 by the authors.

Figure 1
Figure 1. AVASTRA: Approach Overview [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Double Deep-Q Network Architecture in AV [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Representation of experimental roads in AV [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Ineffective scenario generated by AVASTRA on Road2 Regarding TTC, AVASTRA performs better by generating collision scenarios faster in certain roads, but there are instances where its performance lags behind DeepCollision. This suggests that while AVASTRA excels at gene…
Figure 5
Figure 5. Figure 5: Critical scenario generated by AVASTRA on Road 1 AVASTRA effectively utilizes both the internal states of the AV and the external states of the surrounding factors to create a dangerous situation. First, to challenge the AV’s operation when it enters the junction, AVAS…
Figure 6
Figure 6. Figure 6: State analysis on Road 1 and Road 3 (#Collisions: Left axis; TTC: Right axis) TABLE 6: Performance of generating critical scenarios using difference configurations of action-space Action space #Collisions TTC NPC Vehicles + Weather&Time 14.4 24.4 Weather&Time + Pedestr…
Figure 7
Figure 7. Figure 7: Impact of OT P values on AVASTRA’s performance (#Collisions: Left axis; TTC: Right axis) mance. Additionally, incorporating constraints during configuring the environment enhances the relevance and the realism of the generated scenarios. 6.3 Answer RQ3: Parameter Analy…
Figure 9
Figure 9. Figure 9: Impact of θϵ on AVASTRA’s performance (#Collisions: Left axis; TTC: Right axis) a significantly greater reward for selecting effective actions. This helps the RL agents better distinguish the actions that actually lead to collisions, thereby enabling a more effective g…
Figure 10
Figure 10. Figure 10: Impact of training settings on AVASTRA’s perfor￾mance (#Collisions: Left axis; TTC: Right axis) The Comprehensive-Road setting offers the optimal balance, combining the generalization benefits of diverse training with the relevance of specific exposure. A comprehensiv…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 52 canonical work pages

  1. [1]

    Autonomous vehicle market size worldwide 2021-2030,

    Statista, “Autonomous vehicle market size worldwide 2021-2030,” https://www.statista.com/statistics/1224515/ av-market-size-worldwide-forecast, 2024, [Online; accessed 28-July-2024]

    work page arXiv 2021

  2. [2]

    Google sibling Waymo launches fully autonomous ride-hailing service,

    T. Guardian, “Google sibling Waymo launches fully autonomous ride-hailing service,” https:// www.theguardian.com/technology/2017/nov/07/ google-waymo-announces-fully-autonomous-ride-hailing-service-uber-alphabet/, 2024, [Online; accessed 28-July-2024]

    work page 2017

  3. [3]

    60,000 Drivers Now Have Tesla Full Self Driving (FSD) — What It Is & How To Get It (Part 1),

    CleanTechnica, “60,000 Drivers Now Have Tesla Full Self Driving (FSD) — What It Is & How To Get It (Part 1),” https://cleantechnica.com/2022/03/05/ 60000-drivers-now-have-tesla-full-self-driving-fsd-what-it-is-faq-part-1/, 2024, [Online; accessed 28-July-2024]

    work page 2022

  4. [4]

    Lgsvl simulator: A high fidelity simulator for autonomous driving,

    G. Rong, B. H. Shin, H. Tabatabaee, Q. Lu, S. Lemke, M. Možeiko, E. Boise, G. Uhm, M. Gerow, S. Mehta et al., “Lgsvl simulator: A high fidelity simulator for autonomous driving,” in2020 IEEE 23rd International conference on intelligent transportation systems (ITSC) . IEEE, 2020, pp. 1–6

    work page 2020

  5. [5]

    Carla: An open urban driving simulator,

    A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V . Koltun, “Carla: An open urban driving simulator,” in Conference on robot learning. PMLR, 2017, pp. 1–16

    2017

  6. [6]

    Mosat: finding safety violations of autonomous driving systems using multi-objective genetic algorithm,

    H. Tian, Y. Jiang, G. Wu, J. Yan, J. Wei, W. Chen, S. Li, and D. Ye, “Mosat: finding safety violations of autonomous driving systems using multi-objective genetic algorithm,” in Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering , 2022, pp. 94–106

    work page 2022

  7. [7]

    Av-fuzzer: Finding safety violations in autonomous driving systems,

    G. Li, Y. Li, S. Jha, T. Tsai, M. Sullivan, S. K. S. Hari, Z. Kalbarczyk, and R. Iyer, “Av-fuzzer: Finding safety violations in autonomous driving systems,” in 2020 IEEE 31st international symposium on software reliability engineering (ISSRE). IEEE, 2020, pp. 25–36

    work page 2020

  8. [8]

    Learning configurations of operating environment of au- tonomous vehicles to maximize their collisions,

    C. Lu, Y. Shi, H. Zhang, M. Zhang, T. Wang, T. Yue, and S. Ali, “Learning configurations of operating environment of au- tonomous vehicles to maximize their collisions,” IEEE Transactions on Software Engineering, vol. 49, no. 1, pp. 384–402, 2022

    work page 2022

Show all 58 references
  1. [9]

    Diversity- guided search exploration for self-driving cars test genera- tion through frenet space encoding,

    T. Blattner, C. Birchler, T. Kehrer, and S. Panichella, “Diversity- guided search exploration for self-driving cars test genera- tion through frenet space encoding,” in Proceedings of the 17th ACM/IEEE International Workshop on Search-Based and Fuzz Testing , 2024, pp. 9–12

    2024

  2. [10]

    Auto- mated Driving Safety Evaluation Framework Ver 3.0,

    I. Japan Automobile Manufacturers Association, “Auto- mated Driving Safety Evaluation Framework Ver 3.0,” https://www.jama.or.jp/english/reports/docs/Automated_ Driving_Safety_Evaluation_Framework_Ver3.0.pdf, 2022, [Online; accessed 14-Nov-2024]

    2022

  3. [11]

    Generating critical scenarios for testing automated driving systems

    T.-H. Nguyen, T.-G. Vuong, H.-N. Duong, S. Nguyen, D. H. Vo, T. Aoki, and T.-T. Nguyen, “Generating critical scenarios for testing automated driving systems.” [Online]. Available: https://github.com/iSE-UET-VNU/AVASTRA

  4. [12]

    Generating avoidable collision scenarios for testing autonomous driving sys- tems,

    A. Calò, P . Arcaini, S. Ali, F. Hauer, and F. Ishikawa, “Generating avoidable collision scenarios for testing autonomous driving sys- tems,” in 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST). IEEE, 2020, pp. 375–386

    2020

  5. [13]

    Finding critical scenarios for automated driving systems: A sys- tematic literature review,

    X. Zhang, J. Tao, K. Tan, M. Törngren, J. M. G. Sánchez, M. R. Ramli, X. Tao, M. Gyllenhammar, F. Wotawa, N. Mohan et al. , “Finding critical scenarios for automated driving systems: A sys- tematic literature review,” arXiv preprint arXiv:2110.08664, 2021

    arXiv

  6. [14]

    Defining and substantiating the terms scene, situation, and sce- nario for automated driving,

    S. Ulbrich, T. Menzel, A. Reschka, F. Schuldt, and M. Maurer, “Defining and substantiating the terms scene, situation, and sce- nario for automated driving,” in 2015 IEEE 18th international conference on intelligent transportation systems. IEEE, 2015, pp. 982– 988

    2015

  7. [15]

    A first look at the integration of machine learning models in complex autonomous driving systems: a case study on apollo,

    Z. Peng, J. Yang, T.-H. Chen, and L. Ma, “A first look at the integration of machine learning models in complex autonomous driving systems: a case study on apollo,” in Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Fo...

    2020

  8. [16]

    Robust and precise vehicle localization based on multi-sensor fusion in diverse city scenes,

    G. Wan, X. Yang, R. Cai, H. Li, Y. Zhou, H. Wang, and S. Song, “Robust and precise vehicle localization based on multi-sensor fusion in diverse city scenes,” in 2018 IEEE international conference on robotics and automation (ICRA). IEEE, 2018, pp. 4670–4677

    2018

  9. [17]

    Airsim: High-fidelity visual and physical simulation for autonomous vehicles,

    S. Shah, D. Dey, C. Lovett, and A. Kapoor, “Airsim: High-fidelity visual and physical simulation for autonomous vehicles,” in Field and Service Robotics: Results of the 11th International Conference . Springer, 2018, pp. 621–635

    2018

  10. [18]

    A theoretical analysis of deep q-learning,

    J. Fan, Z. Wang, Y. Xie, and Z. Yang, “A theoretical analysis of deep q-learning,” Proceedings of Machine Learning Research vol , vol. 120, pp. 1–4, 2020

    2020

  11. [19]

    Weighted double q- learning,

    Z. Zhang, Z. Pan, and M. J. Kochenderfer, “Weighted double q- learning,” in Proceedings of the 26th International Joint Conference on Artificial Intelligence, 2017, pp. 3455–3461

    2017

  12. [20]

    Human-level control through deep reinforcement learning,

    V . Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al., “Human-level control through deep reinforcement learning,” nature, vol. 518, no. 7540, pp. 529–533, 2015

    2015

  13. [21]

    Longitudinal collision avoidance and lateral stability adaptive control system based on mpc of autonomous vehicles,

    S. Cheng, L. Li, H.-Q. Guo, Z.-G. Chen, and P . Song, “Longitudinal collision avoidance and lateral stability adaptive control system based on mpc of autonomous vehicles,” IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 6, pp. 2376–2385, 2019

    2019

  14. [22]

    A survey on rear end collision avoidance system for automobiles,

    K. Manjunath and N. Jaisankar, “A survey on rear end collision avoidance system for automobiles,”Int. J. Eng. Technol, vol. 5, no. 2, pp. 1368–1372, 2013

    2013

  15. [23]

    Vehicle forward collision warning algorithm based on road friction,

    T. Chen, K. Liu, Z. Wang, G. Deng, and B. Chen, “Vehicle forward collision warning algorithm based on road friction,” Transportation research part D: transport and environment, vol. 66, pp. 49–57, 2019

    2019

  16. [24]

    Heterogeneous traffic mixing regular and connected vehicles: Modeling and stabilization,

    D.-F. Xie, X.-M. Zhao, and Z. He, “Heterogeneous traffic mixing regular and connected vehicles: Modeling and stabilization,” IEEE Transactions on Intelligent Transportation Systems, vol. 20, no. 6, pp. 2060–2071, 2018

    2018

  17. [25]

    Automotive lidar performance verification in fog and rain,

    M. Kutila, P . Pyykönen, H. Holzhüter, M. Colomb, and P . Duthon, “Automotive lidar performance verification in fog and rain,” in 2018 21st International Conference on Intelligent Transportation Sys- tems (ITSC). IEEE, 2018, pp. 1695–1701

    2018

  18. [26]

    Autonomous vehicle ultrasonic sensor vulnerability and impact assessment,

    B. S. Lim, S. L. Keoh, and V . L. Thing, “Autonomous vehicle ultrasonic sensor vulnerability and impact assessment,” in 2018 IEEE 4th World Forum on Internet of Things (WF-IoT) . IEEE, 2018, pp. 231–236

    2018

  19. [27]

    I can see the light: Attacks on autonomous vehicles using invisible lights,

    W. Wang, Y. Yao, X. Liu, X. Li, P . Hao, and T. Zhu, “I can see the light: Attacks on autonomous vehicles using invisible lights,” in Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, 2021, pp. 1930–1944

    2021

  20. [28]

    Adaptive square attack: Fooling autonomous cars with adversarial traffic signs,

    Y. Li, X. Xu, J. Xiao, S. Li, and H. T. Shen, “Adaptive square attack: Fooling autonomous cars with adversarial traffic signs,” IEEE Internet of Things Journal, vol. 8, no. 8, pp. 6337–6347, 2020

    2020

  21. [29]

    Targeting patterns of driving characteristics in testing autonomous driving systems,

    P . Arcaini, X.-Y. Zhang, and F. Ishikawa, “Targeting patterns of driving characteristics in testing autonomous driving systems,” in 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST). IEEE, 2021, pp. 295–305

    2021

  22. [30]

    An incremental ap- proach for understanding collision avoidance of an industrial path planner,

    X.-Y. Zhang, P . Arcaini, and F. Ishikawa, “An incremental ap- proach for understanding collision avoidance of an industrial path planner,” IEEE Transactions on Dependable and Secure Computing , vol. 20, no. 4, pp. 2713–2730, 2022

    2022

  23. [31]

    Parameter coverage for testing of autonomous driving systems 18 under uncertainty,

    T. Laurent, S. Klikovits, P . Arcaini, F. Ishikawa, and A. Ventresque, “Parameter coverage for testing of autonomous driving systems 18 under uncertainty,” ACM Transactions on Software Engineering and Methodology, vol. 32, no. 3, pp. 1–31, 2023

    2023

  24. [32]

    Automatically testing self- driving cars with search-based procedural content generation,

    A. Gambi, M. Mueller, and G. Fraser, “Automatically testing self- driving cars with search-based procedural content generation,” in Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, 2019, pp. 318–328

    2019

  25. [33]

    Model-based exploration of the frontier of behaviours for deep learning system testing,

    V . Riccio and P . Tonella, “Model-based exploration of the frontier of behaviours for deep learning system testing,” in Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2020, pp. 876–888

    2020

  26. [34]

    Adaptive stress testing with reward augmentation for au- tonomous vehicle validatio,

    A. Corso, P . Du, K. Driggs-Campbell, and M. J. Kochender- fer, “Adaptive stress testing with reward augmentation for au- tonomous vehicle validatio,” in 2019 IEEE Intelligent Transportation Systems Conference (ITSC). IEEE, 2019, pp. 163–168

    2019

  27. [35]

    Reality bites: Assessing the realism of driving scenarios with large language models,

    J. Wu, C. Lu, A. Arrieta, T. Yue, and S. Ali, “Reality bites: Assessing the realism of driving scenarios with large language models,” in Proceedings of the 2024 IEEE/ACM First International Conference on AI Foundation Models and Software Engineering , 2024, pp. 40–51

    2024

  28. [36]

    Diavio: Llm- empowered diagnosis of safety violations in ads simulation test- ing,

    Y. Lu, Y. Tian, Y. Bi, B. Chen, and X. Peng, “Diavio: Llm- empowered diagnosis of safety violations in ads simulation test- ing,” in Proceedings of the 33rd ACM SIGSOFT International Sympo- sium on Software Testing and Analysis, 2024, pp. 376–388

    2024

  29. [37]

    A survey on safety-critical driving scenario generation—a methodological perspective,

    W. Ding, C. Xu, M. Arief, H. Lin, B. Li, and D. Zhao, “A survey on safety-critical driving scenario generation—a methodological perspective,” IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 7, pp. 6971–6988, 2023

    2023

  30. [38]

    A driver-vehicle model for ads scenario-based testing,

    R. Queiroz, D. Sharma, R. Caldas, K. Czarnecki, S. García, T. Berger, and P . Pelliccione, “A driver-vehicle model for ads scenario-based testing,” IEEE Transactions on Intelligent Transporta- tion Systems, 2024

    2024

  31. [39]

    How does simulation-based testing for self-driving cars match human perception?

    C. Birchler, T. K. Mohammed, P . Rani, T. Nechita, T. Kehrer, and S. Panichella, “How does simulation-based testing for self-driving cars match human perception?” Proceedings of the ACM on Software Engineering, vol. 1, no. FSE, pp. 929–950, 2024

    2024

  32. [40]

    Ac3r: automatically recon- structing car crashes from police reports,

    T. Huynh, A. Gambi, and G. Fraser, “Ac3r: automatically recon- structing car crashes from police reports,” in 2019 IEEE/ACM 41st International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). IEEE, 2019, pp. 31–34

    2019

  33. [41]

    Sctrans: Constructing a large public scenario dataset for simu- lation testing of autonomous driving systems,

    J. Dai, B. Gao, M. Luo, Z. Huang, Z. Li, Y. Zhang, and M. Yang, “Sctrans: Constructing a large public scenario dataset for simu- lation testing of autonomous driving systems,” in Proceedings of the 46th IEEE/ACM International Conference on Software Engineering, 2024, pp. 1–13

    2024

  34. [42]

    Sovar: Build generalizable scenarios from accident reports for autonomous driving testing,

    A. Guo, Y. Zhou, H. Tian, C. Fang, Y. Sun, W. Sun, X. Gao, A. T. Luu, Y. Liu, and Z. Chen, “Sovar: Build generalizable scenarios from accident reports for autonomous driving testing,” in Proceed- ings of the 39th IEEE/ACM International Conference on Automated Software Engineer...

    2024

  35. [43]

    Search-based selection and prioritization of test scenarios for autonomous driving systems,

    C. Lu, H. Zhang, T. Yue, and S. Ali, “Search-based selection and prioritization of test scenarios for autonomous driving systems,” in International Symposium on Search Based Software Engineering . Springer, 2021, pp. 41–55

    2021

  36. [44]

    Model based generation of driving scenarios,

    T. Hempen, S. Biank, W. Huber, and C. Diedrich, “Model based generation of driving scenarios,” in Intelligent Transport Systems– From Research and Development to the Market Uptake: First Interna- tional Conference, INTSYS 2017, Hyvinkää, Finland, November 29-30, 2017, Proceedi...

    2017

  37. [45]

    A survey on automated driving system testing: Landscapes and trends,

    S. Tang, Z. Zhang, Y. Zhang, J. Zhou, Y. Guo, S. Liu, S. Guo, Y.-F. Li, L. Ma, Y. Xue et al., “A survey on automated driving system testing: Landscapes and trends,” ACM Transactions on Software Engineering and Methodology, vol. 32, no. 5, pp. 1–62, 2023

    2023

  38. [46]

    Testing of autonomous driving systems: where are we and where should we go?

    G. Lou, Y. Deng, X. Zheng, M. Zhang, and T. Zhang, “Testing of autonomous driving systems: where are we and where should we go?” in Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2022, pp. 31–43

    2022

  39. [47]

    Testing autonomous cars for feature interaction failures using many-objective search,

    R. B. Abdessalem, A. Panichella, S. Nejati, L. C. Briand, and T. Stifter, “Testing autonomous cars for feature interaction failures using many-objective search,” in Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, 2018, pp. 143–154

    2018

  40. [48]

    Dirty road can attack: Security of deep learning based automated lane centering under {Physical-World} attack,

    T. Sato, J. Shen, N. Wang, Y. Jia, X. Lin, and Q. A. Chen, “Dirty road can attack: Security of deep learning based automated lane centering under {Physical-World} attack,” in 30th USENIX security symposium (USENIX Security 21), 2021, pp. 3309–3326

    2021

  41. [49]

    Scenarionet: Open-source platform for large-scale traffic scenario simulation and modeling,

    Q. Li, Z. M. Peng, L. Feng, Z. Liu, C. Duan, W. Mo, and B. Zhou, “Scenarionet: Open-source platform for large-scale traffic scenario simulation and modeling,” Advances in neural information process- ing systems, vol. 36, 2024

    2024

  42. [50]

    Assess- ing the safety and reliability of autonomous vehicles from road testing,

    X. Zhao, V . Robu, D. Flynn, K. Salako, and L. Strigini, “Assess- ing the safety and reliability of autonomous vehicles from road testing,” in 2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 2019, pp. 13–23

    2019

  43. [51]

    Hardware-in-the- loop simulation platform for autonomous vehicle aeb prototyping and validation,

    Y. Gao, Z. Xu, X. Zhao, G. Wang, and Q. Yuan, “Hardware-in-the- loop simulation platform for autonomous vehicle aeb prototyping and validation,” in 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2020, pp. 1–6

    2020

  44. [52]

    A real-time co-simulation framework for virtual test and validation on a high dynamic vehicle test bed,

    H. Li, D. Nalic, V . Makkapati, A. Eichberger, X. Fang, and T. Tet- tamanti, “A real-time co-simulation framework for virtual test and validation on a high dynamic vehicle test bed,” in 2021 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2021, pp. 1132–1137

    2021

  45. [53]

    Critical scenario identification concept: the role of the scenario-in-the-loop approach in future automotive testing,

    Z. Szalay, “Critical scenario identification concept: the role of the scenario-in-the-loop approach in future automotive testing,” IEEE Access, 2023

    2023

  46. [54]

    Black-box adversarial attacks in autonomous vehicle technology,

    K. N. Kumar, C. Vishnu, R. Mitra, and C. K. Mohan, “Black-box adversarial attacks in autonomous vehicle technology,” in 2020 IEEE Applied Imagery Pattern Recognition Workshop (AIPR) . IEEE, 2020, pp. 1–7

    2020

  47. [55]

    Robust physical-world at- tacks on deep learning visual classification,

    K. Eykholt, I. Evtimov, E. Fernandes, B. Li, A. Rahmati, C. Xiao, A. Prakash, T. Kohno, and D. Song, “Robust physical-world at- tacks on deep learning visual classification,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 1625–1634

    2018

  48. [56]

    Physical adversarial exam- ples for object detectors,

    D. Song, K. Eykholt, I. Evtimov, E. Fernandes, B. Li, A. Rahmati, F. Tramer, A. Prakash, and T. Kohno, “Physical adversarial exam- ples for object detectors,” in 12th USENIX workshop on offensive technologies (WOOT 18), 2018

    2018

  49. [57]

    Chatgpt- based scenario engineer: A new framework on scenario generation for trajectory prediction,

    X. Li, E. Liu, T. Shen, J. Huang, and F.-Y. Wang, “Chatgpt- based scenario engineer: A new framework on scenario generation for trajectory prediction,” IEEE Transactions on Intelligent Vehicles , 2024

    2024

  50. [58]

    Automotive lidar sensor development scenarios for harsh weather conditions,

    M. Kutila, P . Pyykönen, W. Ritter, O. Sawade, and B. Schäufele, “Automotive lidar sensor development scenarios for harsh weather conditions,” in 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2016, pp. 265–270

    2016

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.