Building reliable sim driving agents by scaling self-play
read the original abstract
Simulation agents are essential for designing and testing systems that interact with humans, such as autonomous vehicles (AVs). These agents serve various purposes, from benchmarking AV performance to stress-testing system limits, but all applications share one key requirement: reliability. To enable sound experimentation, a simulation agent must behave as intended. It should minimize actions that may lead to undesired outcomes, such as collisions, which can distort the signal-to-noise ratio in analyses. As a foundation for reliable sim agents, we propose scaling self-play to thousands of scenarios on the Waymo Open Motion Dataset under semi-realistic limits on human perception and control. Training from scratch on a single GPU, our agents solve almost the full training set within a day. They generalize to unseen test scenes, achieving a 99.8% goal completion rate with less than 0.8% combined collision and off-road incidents across 10,000 held-out scenarios. Beyond in-distribution generalization, our agents show partial robustness to out-of-distribution scenes and can be fine-tuned in minutes to reach near-perfect performance in such cases. We open-source the pre-trained agents and integrate them with a batched multi-agent simulator. Demonstrations of agent behaviors can be viewed at https://sites.google.com/view/reliable-sim-agents, and we open-source our agents at https://github.com/Emerge-Lab/gpudrive.
This paper has not been read by Pith yet.
Forward citations
Cited by 7 Pith papers
-
Scaling Self-Play for End-to-End Driving
Self-play DAgger training in a batched pixel renderer produces end-to-end driving policies that reach competitive performance on HUGSIM and NAVSIM-v2 after real-world adaptation and improve with more self-play compute.
-
Beyond Self-Play: Hierarchical Reasoning for Continuous Motion in Closed-Loop Traffic Simulation
A hierarchical Stackelberg MARL plus continuous-motion architecture with hybrid co-training produces smoother and safer closed-loop traffic behavior than standard self-play methods.
-
What Probing Reveals about Autonomous Driving: Linking Internal Prediction Errors to Ego Planning
Autonomous driving policies with strong closed-loop performance frequently lack timely internal predictions of surrounding vehicles during near-collision events, and causal correction of prediction errors leads to imp...
-
EvoFlock: evolved inverse design of multi-agent motion
Genetic algorithm optimizes parameters of multi-agent flocking models to match user-defined objectives, with alignment emerging from spacing maintenance.
-
Human-like autonomy emerges from self-play and a pinch of human data
Self-play RL regularized with 30 minutes of human data produces driving policies that coordinate with humans, training in 15 hours on one GPU with 2500x less data than imitation learning.
-
CoPark: Learning Reactive Parking via Self-Play
CoPark uses multi-agent self-play RL with a residual policy and threat-modulated asymmetric prior release to achieve 70-85% success and 3-6% collision rates in reactive parking benchmarks.
-
Toward Efficient and Robust Behavior Models for Multi-Agent Driving Simulation
An instance-centric representation with local frames, relative positional encodings, and adaptive reward transformation in adversarial IRL yields scalable, accurate, and robust behavior models for multi-agent driving ...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.