Pith. sign in

REVIEW 47 cited by

Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1610.03295 v1 pith:NCLQY5DO submitted 2016-10-11 cs.AI cs.LGstat.ML

Shai Shalev-Shwartz , Shaked Shammah , Amnon Shashua This is my paper · ORCID
classification cs.AIcs.LGstat.ML
keywords drivingotherautonomouslearningmulti-agentpolicyapplybehavior
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Autonomous driving is a multi-agent setting where the host vehicle must apply sophisticated negotiation skills with other road users when overtaking, giving way, merging, taking left and right turns and while pushing ahead in unstructured urban roadways. Since there are many possible scenarios, manually tackling all possible cases will likely yield a too simplistic policy. Moreover, one must balance between unexpected behavior of other drivers/pedestrians and at the same time not to be too defensive so that normal traffic flow is maintained. In this paper we apply deep reinforcement learning to the problem of forming long term driving strategies. We note that there are two major challenges that make autonomous driving different from other robotic tasks. First, is the necessity for ensuring functional safety - something that machine learning has difficulty with given that performance is optimized at the level of an expectation over many instances. Second, the Markov Decision Process model often used in robotics is problematic in our case because of unpredictable behavior of other agents in this multi-agent scenario. We make three contributions in our work. First, we show how policy gradient iterations can be used without Markovian assumptions. Second, we decompose the problem into a composition of a Policy for Desires (which is to be learned) and trajectory planning with hard constraints (which is not learned). The goal of Desires is to enable comfort of driving, while hard constraints guarantees the safety of driving. Third, we introduce a hierarchical temporal abstraction we call an "Option Graph" with a gating mechanism that significantly reduces the effective horizon and thereby reducing the variance of the gradient estimation even further.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 47 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Seeing Before Colliding: Anticipatory Safe RL with Frozen Vision-Language Models

    cs.LG 2026-06 unverdicted novelty 7.0 of 10

    VLM-Safe-RL adds frozen VLM signals as anticipatory costs to the CMDP Lagrangian update via dual-path CLIP, VLM-Lagrange, and confidence gating, outperforming baselines on Safety-Gymnasium FormulaOne while showing par...

  2. Concentration of General Stochastic Approximation Under Heavy-Tailed Markovian Noise

    math.PR 2026-05 unverdicted novelty 7.0 of 10

    Establishes maximal concentration bounds for stochastic approximation under heavy-tailed Markovian noise, with tails ranging from sub-Gaussian to heavier than Weibull depending on step sizes and contractivity properti...

  3. NePPO: Near-Potential Policy Optimization for General-Sum Multi-Agent Reinforcement Learning

    cs.LG 2026-03 unverdicted novelty 7.0 of 10

    NePPO learns a player-independent potential function via a novel objective whose minimization yields an approximate Nash equilibrium for general-sum multi-agent games.

  4. Reinforcement Learning Control of Quantum Error Correction

    quant-ph 2025-11 conditional novelty 7.0 of 10

    A reinforcement-learning controller that treats quantum error-detection events as rewards stabilizes a superconducting surface/color code under injected drift, cuts logical error rates ~20% after expert calibration, a...

  5. Solving Zero-Sum Convex Markov Games

    cs.GT 2025-06 conditional novelty 7.0 of 10

    Independent policy-gradient algorithms provably compute approximate Nash equilibria in two-player zero-sum convex Markov games.

  6. Policy Optimization for Continuous-time Linear-Quadratic Graphon Mean Field Games

    math.OC 2025-06 accept novelty 7.0 of 10

    A bilevel policy optimization algorithm for continuous-time linear-quadratic graphon mean field games converges linearly to best-response policies and globally to the Nash equilibrium.

  7. Single-Loop Federated Actor-Critic across Heterogeneous Environments

    cs.LG 2024-12 reject novelty 7.0 of 10

    SFAC is a single-loop federated actor-critic method with convergence guarantees for agents in heterogeneous environments, achieving linear speedup when environment heterogeneity is zero.

  8. Provable Partially Observable Reinforcement Learning with Privileged Information

    cs.LG 2024-12 conditional novelty 7.0 of 10

    The paper gives the first provable polynomial-sample and quasi-polynomial-time guarantees for expert distillation and belief-weighted asymmetric actor-critic in POMDPs with privileged state information.

  9. Robust General Utility for Reinforcement Learning

    cs.LG 2026-08 conditional novelty 6.0 of 10

    The paper introduces robust general-utility RL, a minimax formulation over utility uncertainty sets, and proves convergence rates for projected gradient descent-ascent and prox-extragradient algorithms.

  10. Adaptive Partitioning and Learning for Stochastic Control of Diffusion Processes

    cs.LG 2025-12 conditional novelty 6.0 of 10

    APL-Diffusion achieves a regret bound in K whose exponent is governed by a new zooming dimension tailored to unbounded diffusions, recovering Sinclair et al.'s rate as the initial-state moment p goes to infinity.

  11. Distributed primal-dual algorithm for constrained multi-agent reinforcement learning under coupled policies

    cs.MA 2025-11 conditional novelty 6.0 of 10

    Coupled-policy constrained MARL is solved by a distributed primal-dual method that exchanges only local estimates and converges to an ε-first-order stationary point with geometrically decaying truncation error.

  12. Corruption-Tolerant Asynchronous Q-Learning with Near-Optimal Rates

    cs.LG 2025-09 unverdicted novelty 6.0 of 10

    A novel robust asynchronous Q-learning algorithm achieves finite-time convergence rates that match clean-data bounds up to an additive term proportional to the corruption fraction, with a matching information-theoreti...

  13. SENIOR: Efficient Query Selection and Preference-Guided Exploration in Preference-based Reinforcement Learning

    cs.RO 2025-06 unverdicted novelty 6.0 of 10

    SENIOR improves feedback efficiency and policy learning speed in PbRL by combining motion-distinction query selection via kernel density estimation with preference-guided intrinsic rewards, showing gains on simulated ...

  14. Realistic Urban Traffic Generator using Decentralized Federated Learning for the SUMO simulator

    cs.LG 2025-06 conditional novelty 6.0 of 10

    DesRUTGe combines PPO-based reinforcement learning with decentralized federated learning to generate 24-hour traffic profiles that match real detector counts better than SUMO's RouteSampler and a centralized RUTGe baseline.

  15. Finite-Sample Convergence Bounds for Trust Region Policy Optimization in Mean-Field Games

    stat.ML 2025-05 conditional novelty 6.0 of 10

    Exact and sample-based trust-region policy optimization provably converge to approximate Nash equilibria in finite mean-field games with Õ(1/ε^6) sample complexity.

  16. Fox in the Henhouse: Supply-Chain Backdoor Attacks Against Reinforcement Learning

    cs.LG 2025-05 conditional novelty 6.0 of 10

    A malicious pre-trained opponent can, through legitimate in-game actions, embed a trigger-activated backdoor into a victim reinforcement learning agent.

  17. Robust Multi-agent Communication Based on Decentralization-Oriented Adversarial Training

    cs.MA 2025-04 conditional novelty 6.0 of 10

    DMAC is an adversarial training method that masks critical communication channels to force multi-agent communication policies into a more balanced, decentralized structure, improving robustness and task performance.

  18. Discrete GCBF Proximal Policy Optimization for Multi-agent Safe Optimal Control

    cs.RO 2025-02 conditional novelty 6.0 of 10

    DGPPO learns a discrete-time graph control barrier function and a distributed policy together, achieving high safety and task performance in multi-agent simulations with unknown dynamics, partial observability, and in...

  19. Optimistic {\epsilon}-Greedy Exploration for Cooperative Multi-Agent Reinforcement Learning

    cs.MA 2025-02 unverdicted novelty 6.0 of 10

    Optimistic ε-Greedy Exploration adds decoupled optimistic networks that converge in probability to maximum returns and samples from them with probability ε to increase optimal joint-action frequency in CTDE MARL.

  20. TACTIC: Task-Agnostic Contrastive pre-Training for Inter-Agent Communication

    cs.MA 2025-01 conditional novelty 6.0 of 10

    TACTIC uses offline contrastive pretraining, aligning integrated local observations and messages with each agent's egocentric state, to improve multi-agent coordination across varied sight ranges on SMACv2.

  21. Mitigating Relative Over-Generalization in Multi-Agent Reinforcement Learning

    cs.LG 2024-11 conditional novelty 6.0 of 10

    MMQ mitigates relative over-generalization in decentralized MARL by taking the maximum over sampled next states predicted from quantile bounds.

  22. Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Control

    cs.LG 2019-09 conditional novelty 6.0 of 10

    VBC regularizes message variance during training and gates communication on agent confidence and message variance, cutting communication overhead 2-10x on StarCraft II while improving winning rates.

  23. Is Inter-Seed Cross-Play Enough? Evaluating the Robustness of Zero-Shot Coordination Algorithms to Implementation Details

    cs.AI 2026-08 conditional novelty 5.0 of 10

    For Other-Play in Yokai, agents trained with different implementation details coordinate across implementations about as well as across seeds, supporting inter-seed cross-play as a proxy for cross-implementation evaluation.

  24. Strategy-Following Multi-Agent Deep Reinforcement Learning Considering Control Strategies Provided to Other Agents

    cs.MA 2026-07 conditional novelty 5.0 of 10

    Feeding each agent a summed map of the instructions given to the other agents makes uninstructed agents cover overlooked areas and improves object collection by roughly 8% over the baseline in the tested game.

  25. Multi-agent DRL-based Lane Change Decision Model for Cooperative Platooning in Mixed Traffic

    cs.AI 2026-01 conditional novelty 5.0 of 10

    A CNN-QMIX lane-change controller lifts simulated cooperative-platoon formation over MOBIL and greedy baselines and keeps working as the number of connected agents varies.

  26. Scalable Quantum Reinforcement Learning on NISQ Devices with Dynamic-Circuit Qubit Reuse and Grover Optimization

    quant-ph 2025-09 unverdicted novelty 5.0 of 10

    A dynamic-circuit framework for multi-step quantum Markov decision processes reduces physical qubit count from O(T) to O(1) while preserving trajectory fidelity and applying Grover amplification for high-return paths.

  27. Focusing Influence Mechanism for Multi-Agent Reinforcement Learning

    cs.LG 2025-06 unverdicted novelty 5.0 of 10

    The Focusing Influence Mechanism (FIM) uses an entropy-based criterion and eligibility traces to help multiple agents in reinforcement learning focus and maintain their influence on under-explored parts of the state s...

  28. Provably safe and human-like car-following behaviors: Part 1. Analysis of phases and dynamics in standard models

    eess.SY 2025-05 conditional novelty 5.0 of 10

    Using phase-plane analysis, the paper shows that Newell, IDM, and Gipps car-following models each violate at least one stated safety or human-like principle in stationary-lead problems.

  29. CCL: Collaborative Curriculum Learning for Sparse-Reward Multi-Agent Reinforcement Learning via Co-evolutionary Task Evolution

    cs.AI 2025-05 reject novelty 5.0 of 10

    A curriculum learning method that co-evolves training tasks with agents improves cooperative multi-agent reinforcement learning under sparse rewards, according to experiments on MPE and Hide-and-Seek.

  30. Reducing Variance Caused by Communication in Decentralized Multi-agent Deep Reinforcement Learning

    cs.LG 2025-02 conditional novelty 5.0 of 10

    The paper proves that communication in decentralized critics adds variance to policy gradients and introduces baseline plus KL techniques that reduce this variance and improve learning.

  31. Speaking the Language of Teamwork: LLM-Guided Credit Assignment in Multi-Agent Reinforcement Learning

    cs.MA 2025-02 conditional novelty 5.0 of 10

    LLM-generated, agent-specific potential-based rewards accelerate sparse-reward MARL training in grid world and pistonball benchmarks.

  32. GAWM: Global-Aware World Model for Multi-Agent Reinforcement Learning

    cs.MA 2025-01 conditional novelty 5.0 of 10

    GAWM adds cross-agent observation fusion and reward smoothing to a world model for MARL, beating four baselines on eight SMAC maps.

  33. Cooperative Patrol Routing: Optimizing Urban Crime Surveillance through Multi-Agent Reinforcement Learning

    cs.AI 2025-01 reject novelty 5.0 of 10

    A VDPPO-based multi-agent model plans police patrol routes that cover most of the highest-crime nodes in two of three Malaga districts, with weaker results in the third.

  34. Approximate State Abstraction for Markov Games

    cs.GT 2024-12 conditional novelty 5.0 of 10

    A proof that if states are merged when their optimal Q-values are close, the Nash equilibrium of the abstracted zero-sum Markov game has a duality gap bounded by O(ε/(1−γ)^3).

  35. Investigating Relational State Abstraction in Collaborative MARL

    cs.AI 2024-12 conditional novelty 5.0 of 10

    MARC, a relational critic using spatial relations without distances, improves sample efficiency and asymptotic performance in collaborative MARL tasks over several baselines, though not on all continuous tasks.

  36. RMIO: A Model-Based MARL Framework for Scenarios with Observation Loss in Some Agents

    cs.MA 2024-11 conditional novelty 5.0 of 10

    RMIO is a model-based MARL framework that reconstructs lost agent observations with a world model, corrects them using other agents' observations, and reports higher performance than prior methods in SMAC and MaMuJoCo.

  37. Decentralized Multi-Player Q-Learning in Episodic Markov Decision Processes with Information Asymmetry

    cs.LG 2026-08 reject novelty 4.0 of 10

    Decentralized players using pre-agreed deterministic tie-breaking can match centralized Q-learning regret when either actions or rewards are shared, but the fully asymmetric setting rests on an exploration argument th...

  38. CoRe: Combined Rewards with Vision-Language Model Feedback for Preference-Aligned Reinforcement Learning

    cs.RO 2026-07 unverdicted novelty 4.0 of 10

    CoRe combines VLM-designed formal rewards with VLM-labeled residual rewards to produce preference-aligned policies on robotic manipulation tasks.

  39. Adversarial Agent Behavior Learning in Autonomous Driving Using Deep Reinforcement Learning

    cs.CV 2025-08 reject novelty 4.0 of 10

    An adversarial car trained with a collision-based reward reliably decreases the reward of a PPO-trained ego vehicle in Highway-Env, and a robust PPO policy trained against it recovers performance.

  40. Siamese Neural Network for Label-Efficient Critical Phenomena Prediction in 3D Percolation Models

    cond-mat.dis-nn 2025-07 conditional novelty 4.0 of 10

    A Siamese network trained only on non-critical percolation configurations predicts 3D site and bond percolation thresholds, but its learned representation is essentially the normalized largest-cluster size.

  41. A Systematic Study of Multi-Agent Deep Reinforcement Learning for Safe and Robust Autonomous Highway Ramp Entry

    cs.RO 2024-11 conditional novelty 4.0 of 10

    Multi-agent deep RL controllers trained by self-play achieve near-optimal collision avoidance in simulated highway merging with three or more vehicles.

  42. MARFT: Multi-Agent Reinforcement Fine-Tuning

    cs.MA 2025-04 conditional novelty 3.0 of 10

    MARFT applies multi-agent PPO-style fine-tuning to LLM agent teams, improving math and coding performance over vanilla multi-agent baselines in small-scale experiments.

  43. Position: Emergent Machina Sapiens Urge Rethinking Multi-Agent Paradigms

    cs.MA 2025-02 conditional novelty 3.0 of 10

    The paper advocates replacing static multi-agent coordination with dynamic, self-organizing norms for independently deployed AI agents, but provides no experimental evidence.

  44. Quantum framework for Reinforcement Learning: Integrating Markov decision process, quantum arithmetic, and trajectory search

    quant-ph 2024-12 unverdicted novelty 3.0 of 10

    The paper claims a fully quantum MDP model for RL with quantum state transitions, return calculation, and trajectory search that achieves quantum enhancement.

  45. Design Space of Behaviour Planning for Autonomous Driving

    cs.RO 2019-08 unverdicted novelty 3.0 of 10

    The paper decomposes the behaviour planning design space into environment representation, planner architecture, and decision logic, and describes qualitative trade-offs.

  46. Generative Models and Connected and Automated Vehicles: A Survey in Exploring the Intersection of Transportation and AI

    cs.LG 2024-03 unverdicted novelty 2.0 of 10

    A survey reviewing the integration of generative models with connected and automated vehicles to enhance predictive modeling, simulation accuracy, and decision-making.

  47. A Review of Cooperative Multi-Agent Deep Reinforcement Learning

    cs.LG 2019-08 conditional novelty 1.0 of 10

    A review that categorizes cooperative multi-agent deep RL into independent learners, observable critics, value factorization, consensus, and communication, with errors in the taxonomy and references.

Pith tools