REVIEW 11 cited by
Multimodal Large Language Model Driven Scenario Testing for Autonomous Vehicles
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
The generation of corner cases has become increasingly crucial for efficiently testing autonomous vehicles prior to road deployment. However, existing methods struggle to accommodate diverse testing requirements and often lack the ability to generalize to unseen situations, thereby reducing the convenience and usability of the generated scenarios. A method that facilitates easily controllable scenario generation for efficient autonomous vehicles (AV) testing with realistic and challenging situations is greatly needed. To address this, we proposed OmniTester: a multimodal Large Language Model (LLM) based framework that fully leverages the extensive world knowledge and reasoning capabilities of LLMs. OmniTester is designed to generate realistic and diverse scenarios within a simulation environment, offering a robust solution for testing and evaluating AVs. In addition to prompt engineering, we employ tools from Simulation of Urban Mobility to simplify the complexity of codes generated by LLMs. Furthermore, we incorporate Retrieval-Augmented Generation and a self-improvement mechanism to enhance the LLM's understanding of scenarios, thereby increasing its ability to produce more realistic scenes. In the experiments, we demonstrated the controllability and realism of our approaches in generating three types of challenging and complex scenarios. Additionally, we showcased its effectiveness in reconstructing new scenarios described in crash report, driven by the generalization capability of LLMs.
Forward citations
Cited by 11 Pith papers
-
Chat2Scenic: An Iterative RAG-Based Framework for Scenario Generation in Autonomous Driving
Chat2Scenic generates executable Scenic driving-scenario scripts from regulation-style text with 76.4% compilation success, using iterative component-wise generation with retrieval-augmented prompting.
-
Adversarial Generation and Collaborative Evolution of Safety-Critical Scenarios for Autonomous Vehicles
ScenGE generates more collision-prone autonomous driving test scenarios by combining LLM-suggested adversarial events with optimized background traffic, beating prior generators on CARLA benchmarks.
-
CrashAgent: Crash Scenario Generation via Multi-modal Reasoning
A multi-agent vision-language framework converts NHTSA crash reports into simulation-ready road layouts and collision scenarios, with modest accuracy gains over direct VLM baselines.
-
Exploring Critical Testing Scenarios for Decision-Making Policies: An LLM Approach
An LLM-driven generate-test-feedback loop with a multi-scale mutation strategy finds more and more diverse failure scenarios for black-box decision-making policies than MDPFuzz and random testing.
-
Generative AI for Testing of Autonomous Driving Systems: A Survey
A systematic survey that organizes 91 studies of generative AI for autonomous driving testing into six scenario-based tasks and catalogs 27 limitations.
-
LLM-based Property-based Test Generation for Guardrailing Cyber-Physical Systems
An LLM-based tool, ChekProp, extracts safety properties from CPS code and documentation and generates property-based tests that can serve as design-time checks and proposed runtime guardrails.
-
From Words to Collisions: LLM-Guided Evaluation and Adversarial Generation of Safety-Critical Driving Scenarios
An LLM with ego-centric prompts detects collisions and generates adversarial driving scenarios more reliably than Cartesian prompts, though validation of generation is limited.
-
Generating Out-Of-Distribution Scenarios Using Language Models
The paper generates rare driving scenarios via an LLM-built tree, simulates them in CARLA, but its OOD-ness metric is interpreted in a way that contradicts its equation.
-
A Survey on the Applications of Generative Artificial Intelligence in Automated Driving Systems Test Scenario Generation Methods
A literature survey of scenario-generation methods for ADS testing that adds an unvalidated AII/RAS/OCS metric suite and ODD-difficulty schema, undermined by inconsistent calculations in the worked examples.
-
AutoODD: Agentic Audits via Bayesian Red Teaming in Black-Box Models
AutoODD combines an LLM agent with per-axis Gaussian Process uncertainty to automatically discover failure modes of black-box models, demonstrated on missing-digit MNIST and aircraft detect-and-avoid.
-
Seeking to Collide: Online Safety-Critical Scenario Generation for Autonomous Driving with Retrieval Augmented Large Language Models
An LLM framework with a growing memory of intent-to-planner pairs generates adversarial driving scenarios that produce a 75% collision rate on 81 Waymo scenes, exceeding fixed-library baselines.
Discussion (0). Continue with ORCID to comment.