REVIEW 2 major objections 32 references
Jointly training self-driving cars and pedestrians with multi-agent reinforcement learning yields more realistic crossing scenarios and lower collision rates than fixed policies.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.3
2026-06-30 18:38 UTC pith:RV3UIBPV
load-bearing objection MAPPO co-training with hidden per-pedestrian traits cuts simulated collisions 30% versus single-agent RL but offers no external check that the generated behaviors match real crossing data. the 2 major comments →
Multi-Agent Reinforcement Learning for Safe Autonomous Driving Under Pedestrian Behavioral Uncertainty
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Co-training an SDC and twelve pedestrians with MAPPO, where each pedestrian has a hidden trait sampled at the start of an episode that sets its jaywalking probability while an RL policy governs go or wait decisions and Dijkstra handles paths, allows the SDC to reach 78 percent of goals at a 14 percent collision rate in 500-episode tests versus 35 percent goals and 33 percent collisions for the best rule-based baseline, with co-training reducing collisions 30 percent compared with single-agent RL.
What carries the argument
Multi-Agent Proximal Policy Optimization (MAPPO) co-training of the self-driving car and pedestrians, with hidden per-pedestrian traits controlling jaywalking probability and RL policies controlling go/wait decisions.
Load-bearing premise
The sampled per-pedestrian trait together with scripted pathfinding and RL go or wait decisions is assumed to capture the true heterogeneity and uncertainty of real pedestrian crossing behavior.
What would settle it
Collecting real-world video data on pedestrian crossings and checking whether the observed rates of jaywalking, collision outcomes, and speed adjustments match the distributions produced by the co-trained simulation.
If this is right
- The co-trained SDC shows a 2.65 m/s speed increase near jaywalkers at close range, revealing that encounters remain unanticipated.
- Jaywalking accounts for 62 percent of collisions while forming only 13 percent of crossing events.
- Co-training improves goal success from 35 percent to 78 percent over rule-based pedestrian models.
- Single-agent RL suffers 30 percent more collisions than the multi-agent version.
Where Pith is reading between the lines
- Similar co-training could be applied to other uncertain agents such as cyclists or other vehicles to improve overall traffic safety models.
- Validation against real pedestrian trajectory datasets would be needed to confirm the simulated uncertainty distributions match observed behavior.
- The speed differential metric could serve as a general tool for measuring anticipation in reinforcement learning driving agents.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes jointly training an SDC and 12 pedestrians via MAPPO, with pedestrians using scripted Dijkstra pathfinding plus RL-controlled go/wait decisions modulated by a hidden per-pedestrian trait that sets jaywalking probability. The central claim is that this produces more realistic interaction scenarios than fixed-policy baselines; 500-episode evaluations report the co-trained SDC reaching 78% goals with 14% collisions (vs. 35%/33% for the best rule-based baseline), a 2.65 m/s speed differential near jaywalkers at 0-3 m, jaywalking comprising 13% of events but 62% of collisions, and a 30% collision reduction relative to single-agent RL.
Significance. If the quantitative improvements hold under proper statistical reporting and the behavioral model can be shown to align with real pedestrian data, the work would offer a practical route to increase the heterogeneity and uncertainty captured in simulation-based SDC testing. The explicit speed-differential metric and breakdown of collisions by crossing type are concrete strengths that allow direct measurement of anticipation gaps within the simulator.
major comments (2)
- [Abstract] Abstract/Evaluation: the reported performance numbers (78% goal rate, 14% collision rate, 2.65 m/s differential, 30% reduction) are given without training hyperparameters, statistical error bars, significance tests, or explicit description of how the metrics were aggregated across the 500 episodes, so the reliability of the claimed improvements cannot be assessed.
- [Abstract] Abstract: the assertion that co-training produces 'more realistic' scenarios rests solely on internal rollout metrics; no comparison is provided to empirical distributions of crossing times, jaywalking rates, or trait-conditioned behaviors from real-world pedestrian studies, leaving the realism improvement unverified externally.
Simulated Author's Rebuttal
We thank the referee for their constructive feedback. We agree that the abstract requires additional methodological detail and will revise accordingly. We also clarify the scope of our realism claims, which are framed relative to baselines rather than absolute external validation. Point-by-point responses follow.
read point-by-point responses
-
Referee: [Abstract] Abstract/Evaluation: the reported performance numbers (78% goal rate, 14% collision rate, 2.65 m/s differential, 30% reduction) are given without training hyperparameters, statistical error bars, significance tests, or explicit description of how the metrics were aggregated across the 500 episodes, so the reliability of the claimed improvements cannot be assessed.
Authors: We agree that the abstract lacks sufficient detail on evaluation methodology. In the revised manuscript we will add key training hyperparameters (MAPPO learning rate, clip parameter, number of epochs, and batch size), report all metrics as means with standard deviations across five independent random seeds, include paired t-test p-values for baseline comparisons, and explicitly state that the 500 episodes use held-out pedestrian trait samples with aggregation performed by averaging per-episode outcomes. revision: yes
-
Referee: [Abstract] Abstract: the assertion that co-training produces 'more realistic' scenarios rests solely on internal rollout metrics; no comparison is provided to empirical distributions of crossing times, jaywalking rates, or trait-conditioned behaviors from real-world pedestrian studies, leaving the realism improvement unverified externally.
Authors: The manuscript's claim is that co-training yields more realistic scenarios than fixed-policy baselines, as shown by the learned speed-differential gap and collision breakdown. We do not provide direct comparisons to real-world empirical distributions, as aligning the trait model to specific pedestrian datasets lies outside the present scope. We will revise the abstract to emphasize the relative comparison to rule-based baselines and to note that external validation against real pedestrian studies is an important avenue for future work. revision: partial
Circularity Check
No significant circularity; metrics are direct simulation outputs
full rationale
The paper reports performance metrics (goal rates, collision rates, speed differentials) as direct results of MAPPO rollouts in a custom simulator. No equations, fitted parameters, or self-citations are shown to reduce the central claims (e.g., realism of co-trained pedestrians) back to the inputs by construction. The model assumptions about latent traits and Dijkstra pathfinding are stated explicitly but do not create definitional loops or rename known results. This is a standard simulation study whose outputs are independent of any internal derivation chain.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption MAPPO training produces stable policies that reflect the intended pedestrian heterogeneity
invented entities (1)
-
per-pedestrian hidden trait controlling jaywalking probability
no independent evidence
read the original abstract
Simulation-based testing of self-driving cars (SDCs) typically relies on scripted pedestrian models that do not capture the heterogeneity and uncertainty of real crossing behavior, limiting the realism of safety assessments, especially for jaywalking, which is governed by latent personality traits the vehicle cannot observe. We hypothesize that jointly training pedestrians and the SDC with multi-agent reinforcement learning (MARL) yields more realistic interaction scenarios than training against fixed pedestrian policies, and that the behavior gap between predictable and unpredictable crossings can be measured directly from trajectories. We co-train an SDC and 12 pedestrians using Multi-Agent Proximal Policy Optimization (MAPPO): pedestrian locomotion follows scripted Dijkstra pathfinding while an RL policy controls high-level go/wait decisions, and jaywalking probability depends on a per-pedestrian trait sampled at episode start and hidden from the SDC. In 500-episode evaluations, the co-trained SDC reached 78% of goals with a 14% collision rate, versus 35%/33% for the best rule-based baseline. A speed differential metric shows the SDC traveled 2.65 m/s faster near jaywalkers than near crosswalk users at close range (0-3 m), indicating jaywalking encounters were not anticipated. Jaywalking was 13% of crossing events but 62% of collisions, and co-training reduced collisions by 30% relative to single-agent RL as pedestrians learned to wait when the SDC approached at speed.
Figures
Reference graph
Works this paper leans on
-
[1]
Simulation of pedestrian interaction with autonomous vehicles via social force model,
M. M. Rashid, M. Seyedi, and S. Jung, “Simulation of pedestrian interaction with autonomous vehicles via social force model,” Simulation Modelling Practice and Theory, vol. 132, p. 102901, Apr
-
[2]
Available: https://www.sciencedirect.com/science/ article/pii/S1569190X24000157
[Online]. Available: https://www.sciencedirect.com/science/ article/pii/S1569190X24000157
-
[3]
How Does Simulation-Based Testing for Self-Driving Cars Match Human Perception?
C. Birchler, T. K. Mohammed, P. Rani, T. Nechita, T. Kehrer, and S. Panichella, “How Does Simulation-Based Testing for Self-Driving Cars Match Human Perception?”Replication Package - ”How does Simulation-based Testing for Self-driving Cars match Human Perception?”, vol. 1, no. FSE, pp. 42:929–42:950, Jul. 2024. [Online]. Available: https://dl.acm.org/doi/...
-
[4]
The surprising effectiveness of PPO in cooperative multi- agent games,
C. Yu, A. Velu, E. Vinitsky, J. Gao, Y . Wang, A. Bayen, and Y . Wu, “The surprising effectiveness of PPO in cooperative multi- agent games,” inProceedings of the 36th International Conference on Neural Information Processing Systems, ser. NIPS ’22. Red Hook, NY , USA: Curran Associates Inc., Nov. 2022, pp. 24 611–24 624
work page 2022
-
[5]
Z. Zhang, H. Li, T. Chen, N. N. Sze, W. Yang, Y . Zhang, and G. Ren, “Decision-making of autonomous vehicles in interactions with jaywalkers: A risk-aware deep reinforcement learning approach,” Accident Analysis & Prevention, vol. 210, p. 107843, Feb. 2025. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S0001457524003889
work page 2025
-
[6]
Social Force Model for Pedestrian Dynamics
D. Helbing and P. Molnar, “Social Force Model for Pedestrian Dynamics,”Physical Review E, vol. 51, no. 5, pp. 4282– 4286, May 1995, arXiv:cond-mat/9805244. [Online]. Available: http://arxiv.org/abs/cond-mat/9805244
work page internal anchor Pith review Pith/arXiv arXiv 1995
-
[7]
Will automated vehicles encourage more jaywalking? Results from a stated preference survey,
X. Dong, E. Guerra, and R. A. Daziano, “Will automated vehicles encourage more jaywalking? Results from a stated preference survey,” Transportation Research Part F: Traffic Psychology and Behaviour, vol. 103, pp. 217–229, May 2024. [Online]. Available: https: //www.sciencedirect.com/science/article/pii/S1369847824000858
work page 2024
-
[8]
Multi-Agent Reinforcement Learning-based Cooperative Autonomous Driving in Smart Intersections,
T. Yu, K. Wang, Z. Li, T. Yu, and K. Sakaguchi, “Multi-Agent Reinforcement Learning-based Cooperative Autonomous Driving in Smart Intersections,” May 2025, arXiv:2505.04231 [cs]. [Online]. Available: http://arxiv.org/abs/2505.04231
-
[9]
arXiv preprint arXiv:2408.09675 , year=
R. Zhang, J. Hou, F. Walter, S. Gu, J. Guan, F. R ¨ohrbein, Y . Du, P. Cai, G. Chen, and A. Knoll, “Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey,” Aug. 2024, arXiv:2408.09675 [cs]. [Online]. Available: http://arxiv.org/abs/2408.09675
-
[10]
Multi-agent actor-critic for mixed cooperative-competitive environments,
R. Lowe, Y . Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch, “Multi-agent actor-critic for mixed cooperative-competitive environments,” inProceedings of the 31st International Conference on Neural Information Processing Systems, ser. NIPS’17. Red Hook, NY , USA: Curran Associates Inc., Dec. 2017, pp. 6382–6393. [Online]. Available: https://dl.acm.org/d...
-
[11]
Monotonic value function factorisation for deep multi-agent reinforcement learning,
T. Rashid, M. Samvelyan, C. S. De Witt, G. Farquhar, J. Foerster, and S. Whiteson, “Monotonic value function factorisation for deep multi-agent reinforcement learning,”J. Mach. Learn. Res., vol. 21, no. 1, pp. 178:7234–178:7284, Jan. 2020. [Online]. Available: https://dl.acm.org/doi/10.5555/3455716.3455894
-
[12]
Counterfactual multi-agent policy gradients,
J. N. Foerster, G. Farquhar, T. Afouras, N. Nardelli, and S. Whiteson, “Counterfactual multi-agent policy gradients,” inProceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence, ser. AA...
-
[13]
C. S. d. Witt, T. Gupta, D. Makoviichuk, V . Makoviychuk, P. H. S. Torr, M. Sun, and S. Whiteson, “Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?” Nov. 2020, arXiv:2011.09533 [cs]. [Online]. Available: http://arxiv.org/abs/2011. 09533
-
[14]
Proximal Policy Optimization Algorithms
J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal Policy Optimization Algorithms,” Aug. 2017, arXiv:1707.06347 [cs]. [Online]. Available: http://arxiv.org/abs/ 1707.06347
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[15]
High-Dimensional Continuous Control Using Generalized Advantage Estimation
J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel, “High- Dimensional Continuous Control Using Generalized Advantage Estimation,” Oct. 2018, arXiv:1506.02438 [cs]. [Online]. Available: http://arxiv.org/abs/1506.02438
work page internal anchor Pith review Pith/arXiv arXiv 2018
-
[16]
F. A. Oliehoek and C. Amato,A Concise Introduction to Decentralized POMDPs, ser. SpringerBriefs in Intelligent Systems. Cham: Springer International Publishing, 2016. [Online]. Available: http://link.springer.com/10.1007/978-3-319-28929-8
-
[17]
Fuzzy Logic- Based Model That Incorporates Personality Traits for Heterogeneous Pedestrians,
Z. Xue, Q. Dong, X. Fan, Q. Jin, H. Jian, and J. Liu, “Fuzzy Logic- Based Model That Incorporates Personality Traits for Heterogeneous Pedestrians,”Symmetry, vol. 9, no. 10, p. 239, Oct. 2017. [Online]. Available: https://www.mdpi.com/2073-8994/9/10/239
work page 2017
-
[18]
Y . Wang, A. R. Srinivasan, Y . M. Lee, and G. Markkula, “Modeling Pedestrian Crossing Behavior: A Reinforcement Learning Approach with Sensory Motor Constraints,” Sep. 2024, arXiv:2409.14522 [cs]. [Online]. Available: http://arxiv.org/abs/2409.14522
-
[19]
Social LSTM: Human Trajectory Prediction in Crowded Spaces,
A. Alahi, K. Goel, V . Ramanathan, A. Robicquet, L. Fei- Fei, and S. Savarese, “Social LSTM: Human Trajectory Prediction in Crowded Spaces,” 2016, pp. 961–971. [Online]. Available: https://openaccess.thecvf.com/content cvpr 2016/ html/Alahi Social LSTM Human CVPR 2016 paper.html
work page 2016
-
[20]
Available: https://arxiv.org/abs/2001.03093
T. Salzmann, B. Ivanovic, P. Chakravarty, and M. Pavone, “Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Data,” Jan. 2021, arXiv:2001.03093 [cs]. [Online]. Available: http://arxiv.org/abs/2001.03093
-
[21]
Impact of jaywalking on pedestrian interaction behavior: A multiagent Markov Game- based analysis,
E. A. Khuzam, G. Lanzaro, and T. Sayed, “Impact of jaywalking on pedestrian interaction behavior: A multiagent Markov Game- based analysis,”Accident Analysis & Prevention, vol. 220, p. 108141, Sep. 2025. [Online]. Available: https://www.sciencedirect. com/science/article/pii/S0001457525002271
work page 2025
-
[22]
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
A. Kendall and Y . Gal, “What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?” Oct. 2017, arXiv:1703.04977 [cs]. [Online]. Available: http://arxiv.org/abs/1703.04977
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[23]
C.-J. Hoel, K. Wolff, and L. Laine, “Tactical Decision-Making in Autonomous Driving by Reinforcement Learning with Uncertainty Estimation,” Apr. 2020, arXiv:2004.10439 [cs]. [Online]. Available: http://arxiv.org/abs/2004.10439
-
[24]
K. Wang, C. Shen, X. Li, and J. Lu, “Uncertainty Quantification for Safe and Reliable Autonomous Vehicles: A Review of Methods and Applications,”IEEE Transactions on Intelligent Transportation Systems, vol. 26, no. 3, pp. 2880–2896, Mar. 2025. [Online]. Available: https://ieeexplore.ieee.org/document/10879299
-
[25]
CARLA: An Open Urban Driving Simulator
A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V . Koltun, “CARLA: An Open Urban Driving Simulator,” Nov. 2017, arXiv:1711.03938 [cs]. [Online]. Available: http://arxiv.org/abs/1711. 03938
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[26]
P. A. Lopez, M. Behrisch, L. Bieker-Walz, J. Erdmann, Y .-P. Fl¨otter¨od, R. Hilbrich, L. L ¨ucken, J. Rummel, P. Wagner, and E. WieBner, “Microscopic Traffic Simulation using SUMO,” in2018 21st International Conference on Intelligent Transportation Systems (ITSC). Maui, HI, USA: IEEE Press, Nov. 2018, pp. 2575–2582. [Online]. Available: https://doi.org/1...
-
[27]
JAX: composable transformations of Python+NumPy programs,
J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, Y . Katariya, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne, and Q. Zhang, “JAX: composable transformations of Python+NumPy programs,” 2018. [Online]. Available: http://github.com/jax-ml/jax
work page 2018
-
[28]
JaxMARL: Multi-Agent RL Environments and Algorithms in JAX,
A. Rutherford, B. Ellis, M. Gallici, J. Cook, A. Lupu, G. Ingvarsson, T. Willi, R. Hammond, A. Khan, C. S. d. Witt, A. Souly, S. Bandyopadhyay, M. Samvelyan, M. Jiang, R. T. Lange, S. Whiteson, B. Lacerda, N. Hawes, T. Rocktaschel, C. Lu, and J. N. Foerster, “JaxMARL: Multi-Agent RL Environments and Algorithms in JAX,” Nov. 2024, arXiv:2311.10090 [cs]. [O...
work page internal anchor Pith review arXiv 2024
-
[29]
CleanRL: High-quality Single- file Implementations of Deep Reinforcement Learning Algorithms,
S. Huang, R. F. J. Dossa, C. Ye, J. Braga, D. Chakraborty, K. Mehta, and J. G. M. Ara ´ujo, “CleanRL: High-quality Single- file Implementations of Deep Reinforcement Learning Algorithms,” Journal of Machine Learning Research, vol. 23, no. 274, pp. 1–18,
-
[30]
Available: http://jmlr.org/papers/v23/21-1342.html
[Online]. Available: http://jmlr.org/papers/v23/21-1342.html
-
[31]
P. Polack, F. Altch ´e, B. d’Andr´ea Novel, and A. de La Fortelle, “The kinematic bicycle model: A consistent model for planning feasible tra- jectories for autonomous vehicles?” in2017 IEEE Intelligent Vehicles Symposium (IV), Jun. 2017, pp. 812–818
work page 2017
-
[32]
P. Aryan and S. Panichella, “Physics-informed machine learning for precision unmanned aerial vehicle control: Adaptive transformers with safety guarantees,”Engineering Applications of Artificial Intelligence, vol. 172, p. 114379, 2026
work page 2026
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.