REVIEW 15 cited by
CaRL: Learning Scalable Planning Policies with Simple Rewards
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
CaRL: Learning Scalable Planning Policies with Simple Rewards
read the original abstract
We investigate reinforcement learning (RL) for privileged planning in autonomous driving. State-of-the-art approaches for this task are rule-based, but these methods do not scale to the long tail. RL, on the other hand, is scalable and does not suffer from compounding errors like imitation learning. Contemporary RL approaches for driving use complex shaped rewards that sum multiple individual rewards, \eg~progress, position, or orientation rewards. We show that PPO fails to optimize a popular version of these rewards when the mini-batch size is increased, which limits the scalability of these approaches. Instead, we propose a new reward design based primarily on optimizing a single intuitive reward term: route completion. Infractions are penalized by terminating the episode or multiplicatively reducing route completion. We find that PPO scales well with higher mini-batch sizes when trained with our simple reward, even improving performance. Training with large mini-batch sizes enables efficient scaling via distributed data parallelism. We scale PPO to 300M samples in CARLA and 500M samples in nuPlan with a single 8-GPU node. The resulting model achieves 64 DS on the CARLA longest6 v2 benchmark, outperforming other RL methods with more complex rewards by a large margin. Requiring only minimal adaptations from its use in CARLA, the same method is the best learning-based approach on nuPlan. It scores 91.3 in non-reactive and 90.6 in reactive traffic on the Val14 benchmark while being an order of magnitude faster than prior work.
Forward citations
Cited by 15 Pith papers
-
Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Driving
BehaviorBench reveals that self-play RL policies for autonomous driving overfit to their training traffic agents and do not generalize to other behaviors, motivating a hybrid rule-based plus learned planner.
-
Fail2Drive: Benchmarking Closed-Loop Driving Generalization
Fail2Drive is the first paired-route benchmark for closed-loop generalization in CARLA, showing an average 22.8% success-rate drop on shifted scenarios and revealing failure modes such as ignoring visible LiDAR objects.
-
On Data Thinning for Model Validation in Small Area Estimation
Data thinning splits area-level observations to enable out-of-sample validation of Fay-Herriot models, with recommendations for thinning parameters that balance bias and variance for stable model comparison.
-
Scaling Self-Play for End-to-End Driving
Self-play DAgger training in a batched pixel renderer produces end-to-end driving policies that reach competitive performance on HUGSIM and NAVSIM-v2 after real-world adaptation and improve with more self-play compute.
-
MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving
MAPLE performs closed-loop multi-agent training of VLA driving models entirely in latent space using supervised fine-tuning followed by RL with safety, progress, and diversity rewards, reaching SOTA on Bench2Drive.
-
MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving
MAPLE proposes latent multi-agent rollouts with supervised fine-tuning followed by reinforcement learning using safety, progress, interaction, and diversity rewards to enable scalable closed-loop training for end-to-e...
-
On Data Thinning for Model Validation in Small Area Estimation
Thinned-data MSE for small-area models is unbiased for a risk that systematically differs from full-data risk; under Fay-Herriot the gap is closed-form in the model's shrinkage, and the thinning fraction faces a sharp...
-
Learning Dexterous Grasping from Sparse Taxonomy Guidance
GRIT learns dexterous grasping from sparse taxonomy guidance, achieving 87.9% success and better generalization to novel objects via a two-stage prediction-plus-policy approach.
-
Learning Dexterous Grasping from Sparse Taxonomy Guidance
Dense spatial modulation of multiple 3D LUTs plus an uncertainty-weighted reconstruction loss improves mixed under/over-exposure correction over global-modulation baselines on four benchmarks.
-
Goal-Oriented Reactive Simulation for Closed-Loop Trajectory Prediction
Closed-loop on-policy training with a reactive goal-oriented scene decoder cuts collision rates by up to 79.5% in dense traffic compared to standard open-loop baselines.
-
Zero-Human Demonstration End-to-end Autonomous Driving with Trajectory Scorer
A reward-only offline RL method for trajectory planning in end-to-end autonomous driving achieves state-of-the-art on Navhard and competitive closed-loop HUGSIM performance without imitation learning.
-
CLEAR: Closed-Loop Reinforcement Learning at Scale for End-to-End Autonomous Driving
Residual waypoint RL around a frozen VLA prior, scaled via heterogeneous CARLA/H100 infrastructure, raises closed-loop driving score and success rate on longest6 v2 and Bench2Drive.
-
DriveSafer: End-to-End Autonomous Driving with Safety Guidance
DriveSafer reduces catastrophic failures (PDMS=0) by 48% and drivable-area compliance failures by over 65% versus DiffusionDrive on the NAVSIM benchmark by combining training-time safety constraints with inference-tim...
-
End-to-End Crop Row Navigation via LiDAR-Based Deep Reinforcement Learning
Raw 3D LiDAR, compressed into flattened voxel maps, trains a reinforcement learning policy that reliably follows straight crop rows in simulation and degrades on curvier rows.
-
PlanRL: A Trajectory Planning Architecture for Reinforcement Learning-based Driving Experts
PlanRL uses RL to generate parameters for polynomial trajectories in a Frenet frame with kinematic checks, achieving 5-11% better driving scores on CARLA benchmarks than control-based RL.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.