REVIEW 7 cited by
Multimodal Large Language Model Driven Scenario Testing for Autonomous Vehicles
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The generation of corner cases has become increasingly crucial for efficiently testing autonomous vehicles prior to road deployment. However, existing methods struggle to accommodate diverse testing requirements and often lack the ability to generalize to unseen situations, thereby reducing the convenience and usability of the generated scenarios. A method that facilitates easily controllable scenario generation for efficient autonomous vehicles (AV) testing with realistic and challenging situations is greatly needed. To address this, we proposed OmniTester: a multimodal Large Language Model (LLM) based framework that fully leverages the extensive world knowledge and reasoning capabilities of LLMs. OmniTester is designed to generate realistic and diverse scenarios within a simulation environment, offering a robust solution for testing and evaluating AVs. In addition to prompt engineering, we employ tools from Simulation of Urban Mobility to simplify the complexity of codes generated by LLMs. Furthermore, we incorporate Retrieval-Augmented Generation and a self-improvement mechanism to enhance the LLM's understanding of scenarios, thereby increasing its ability to produce more realistic scenes. In the experiments, we demonstrated the controllability and realism of our approaches in generating three types of challenging and complex scenarios. Additionally, we showcased its effectiveness in reconstructing new scenarios described in crash report, driven by the generalization capability of LLMs.
Forward citations
Cited by 7 Pith papers
-
Chat2Scenic: An Iterative RAG-Based Framework for Scenario Generation in Autonomous Driving
Chat2Scenic generates executable Scenic driving-scenario scripts from regulation-style text with 76.4% compilation success, using iterative component-wise generation with retrieval-augmented prompting.
-
Adversarial Generation and Collaborative Evolution of Safety-Critical Scenarios for Autonomous Vehicles
ScenGE generates more collision-prone autonomous driving test scenarios by combining LLM-suggested adversarial events with optimized background traffic, beating prior generators on CARLA benchmarks.
-
Operating Within the Operational Design Domain: Zero-Shot Perception with Vision-Language Models
Vision-language models achieve usable zero-shot ODD perception in driving scenes when guided by definition-anchored chain-of-thought prompting with persona decomposition.
-
Generative AI for Testing of Autonomous Driving Systems: A Survey
A systematic survey that organizes 91 studies of generative AI for autonomous driving testing into six scenario-based tasks and catalogs 27 limitations.
-
Operating Within the Operational Design Domain: Zero-Shot Perception with Vision-Language Models
Vision-language models can serve as zero-shot ODD sensors for autonomous driving when using definition-anchored chain-of-thought prompting with persona decomposition.
-
A Survey on the Applications of Generative Artificial Intelligence in Automated Driving Systems Test Scenario Generation Methods
A literature survey of scenario-generation methods for ADS testing that adds an unvalidated AII/RAS/OCS metric suite and ODD-difficulty schema, undermined by inconsistent calculations in the worked examples.
-
AutoODD: Agentic Audits via Bayesian Red Teaming in Black-Box Models
AutoODD combines an LLM agent with per-axis Gaussian Process uncertainty to automatically discover failure modes of black-box models, demonstrated on missing-digit MNIST and aircraft detect-and-avoid.
Discussion (0). Sign in to comment.