Pith. sign in

REVIEW 4 major objections 4 minor 36 references

Exposing the Copycat Problem of Imitation-based Planner: A Novel Closed-Loop Simulator, Causal Benchmark and Joint IL-RL Baseline

T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read IL-RL doubles goal completion on new causal driving benchmark

desk verdict Useful causal benchmark undermined by an abstract safety claim the paper's own tables contradict. read the letter →

arxiv 2504.14709 v1 pith:5XIZIU7O submitted 2025-04-20 cs.CV cs.AI

Hui Zhou , Shaoshuai Shi , Hongsheng Li This is my paper · ORCID
classification cs.CVcs.AI
keywords copycatproblemimitationlearningreinforcementclosed-loopsimulatorcausalbenchmarkautonomousdrivingWaymoOpenDatasetMTR-SAC
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that imitation-learned driving planners often cheat by extrapolating from the ego vehicle's own recent state rather than understanding the scene, a failure it calls the copycat problem. To expose and measure that failure, the authors build a lightweight closed-loop simulator and a causal benchmark, Causality9k, on top of the Waymo Open Dataset: each scenario keeps the same historical context but assigns new destinations generated by a depth-first search over lane-level actions (lane change, turn left, turn right, go straight). They then propose MTR-SAC, a simple joint imitation-reinforcement learning baseline that feeds MTR's transformer features together with the agent's position, velocity, and goal into an offline Soft Actor-Critic policy. In closed-loop simulation with MPC control and IDM-driven traffic, the imitation-only baseline completes 30.8% of the causal goals while MTR-SAC completes 49.6%, and the 'stuck' rate drops from 44.7% to 1.3%. The paper's central thesis is that endpoint-varying evaluation reveals whether a planner responds to goals at all, and that a joint IL-RL objective makes the learned policy far more adaptable than pure imitation.

What carries the argument

The load-bearing object is Causality9k, a causal benchmark created by re-goaling Waymo scenarios. Algorithm 1 enumerates alternative paths with a depth-first search over lane-change, left-turn, right-turn, and straight actions, using a cost table (LC:5, TL:1, TR:1, Go:1) with a maximum total cost of 10, discards paths longer than the original distance to the goal, samples an endpoint at that distance on each surviving path, and applies non-maximum suppression at 2.5 meters so the goals are distinct. The second carrier is MTR-SAC, which fuses the imitation model's learned transformer features with a low-dimensional state (location, velocity, goal) as the observation for an offline Soft Actor-Critic policy trained from a replay buffer of simulated rollouts. Together they turn the 'planner fails in new situations' observation into a measurable gap: the difference in completion between a policy that responds to the goal and one that copies its history.

What would settle it

Run a reward-weight Pareto sweep for MTR-SAC on Causality9k and check whether any setting reaches both a completion rate at or above 49.6% and collision and off-road rates at or below the imitation baseline's 15.6% and 8.9%. If no point on that frontier dominates the imitation-only policy, the paper's claim that joint IL-RL improves safety alongside adaptability is not supported.

Watch

Extended reading notes

Core claim

The paper's central claim is that imitation-only planners such as MTR exhibit the copycat problem: they output trajectories by extrapolating from the ego vehicle's historical state instead of learning the mapping from the full scene and a specified destination. To demonstrate this, the authors construct Causality9k, which re-goals Waymo scenarios at the same travel distance using a DFS over lane-level actions with a cost table (lane change costs 5, turns and straight cost 1, total budget 10) and non-maximum suppression at 2.5 meters. In the resulting closed-loop evaluations, the MTR baseline that selects the predicted trajectory closest to the goal completes only 30.8% of the scenarios, while the proposed MTR-SAC, which concatenates MTR's pre-head features with the simulator state and trains an offline SAC policy with a five-term reward, completes 49.6% and reduces the fraction of vehicles that become stuck from 44.7% to 1.3%. The authors conclude that combining imitation and reinforcement learning in a single model overcomes the copycat limitation, and they present the simulator and causal benchmark as tools for assessing whether ML planners genuinely understand driving goals.

Load-bearing premise

The DFS-generated alternative goals are realistic, reachable, and similar in difficulty to the original goal, so that performance differences on Causality9k measure the planner's response to different endpoints rather than artifacts of impossible or trivially easy destinations.

Editorial extensions

If this is right

  • Planners evaluated only on their original datasets overstate their generalization, because in those datasets goals and histories are correlated; Causality9k removes that confound.
  • The completion gap between MTR-Close* (30.8%) and MTR-SAC (49.6%) indicates that pure IL policies stall when the endpoint is novel, and an RL objective that rewards progress directly counters that.
  • The drop in the stuck rate from 44.7% to 1.3% suggests that the RL component learns to keep moving toward a goal rather than reproducing a memorized trajectory.
  • A closed-loop simulator with an MPC controller is necessary for fair evaluation: MTR's open-loop performance (47.4% completion) does not predict its closed-loop behavior in the paper's comparison.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next experiment is a full Pareto sweep over the five reward terms; the paper only varies three weights, and mapping the frontier would show whether the completion gain can be obtained without moving collision and off-road rates above the imitation baseline.
  • The DFS goal generator is specific to Waymo's lane graph; porting the same re-goaling idea to OpenDRIVE-based datasets such as nuPlan would test whether the copycat effect is a dataset artifact or a general IL failure.
  • An input-masking diagnostic could isolate the mechanism: on Causality9k, drop the ego vehicle's historical trajectory from the encoder input and measure the completion drop; a large drop indicates the planner was relying on ego extrapolation.
  • MTR-SAC's feature concatenation is the simplest point in the IL-RL fusion design space; comparing it with residual-action or KL-regularized combinations on the same benchmark would attribute the gain to the RL objective versus the feature representation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper addresses the "copycat problem" in imitation-learning-based planners by contributing three artifacts: (1) a lightweight closed-loop simulator for the Waymo Open Dataset that supports both IL and RL training; (2) a "causality benchmark" (Causality9k) in which alternative goal endpoints are generated for the same historical context via a DFS-based procedure; and (3) a joint IL-RL planner, MTR-SAC, that concatenates MTR's transformer features with the simulator state and trains a SAC policy. The authors claim in the abstract and introduction that MTR-SAC "significantly improves the safety and adaptability" over imitation-only baselines. The experiments compare MTR-Close* (IL-only), StateSAC (RL-only), and MTR-SAC on Test4k and Causality9k, and also ablate reward weights. The paper reports large gains in completion rate and large reductions in "stuck" rate for MTR-SAC, but also substantially higher collision and offroad rates.

Significance. If the causality benchmark were properly validated and the joint IL-RL method truly improved safety, this would be a worthwhile contribution: the community lacks lightweight PyTorch-based closed-loop simulators for Waymo data, and a causal evaluation protocol for the copycat problem is a good idea. The paper is also transparent about some limitations and points to a reproducible release. However, the central empirical claim as stated is not supported by the paper's own tables (collision and offroad rates worsen under MTR-SAC), and the benchmark's validity is not established because the goal-generation parameters are ad hoc and there is no reachability/difficulty analysis. The absence of error bars, test-set reward-weight tuning, and lack of comparison to existing IL-RL hybrids further weaken the evidence. The benchmark and simulator may be useful seeds for future work, but the paper in its current form does not substantiate its headline claims.

major comments (4)
  1. [IV-D, Tables III and IV] The abstract and Section I claim that MTR-SAC "significantly improves the safety and adaptability" of learned policies compared to imitation-only methods. On the paper's own safety metrics, the opposite holds. On Causality9k (Table IV), MTR-Close* (IL-only) has collision 0.156 and offroad 0.089, while MTR-SAC has collision 0.223 and offroad 0.268; on Test4k (Table III), collision rises from 0.122 to 0.273 and offroad from 0.041 to 0.142. These are exactly the events that the reward function in Section III-C penalizes as safety terms. The improvement is confined to completion rate and the "stuck" category, so the safety claim is contradicted by the paper's own measurements. This is a load-bearing problem for the central contribution.
  2. [IV-D-4, Table V, with IV-C] The reward weights are tuned on Causality9k, the same benchmark used for the final comparison, and the configuration selected for Table IV (1,1,10) is one of the rows of the tuning table. This is test-set tuning, and no held-out validation is reported. As a result, the completion advantage of MTR-SAC over IL-only in Table IV could reflect selection on the test set rather than a generalizable property of the method. The paper should report results on a separate validation split and give multiple-seed statistics with error bars for all metrics.
  3. [III-B, Algorithm 1] The validity of the causality benchmark rests on the assumption that the DFS-generated goals are realistic, reachable, and comparable in difficulty to the original goals. The parameters in Algorithm 1 (CostTable {LC:5, TL:1, TR:1, Go:1}, MaxCostThres=10, NmsThres=2.5 m) are given without justification, and the paper provides no analysis of the distribution of generated goal distances, no reachability check under the bicycle model, and no measure of task difficulty across goals. Without such validation, completion or collision differences on Causality9k do not cleanly isolate the planner's response to endpoints; they could be artifacts of goal generation. This is crucial because the benchmark is the basis for the paper's causal claims.
  4. [IV-D-3 and II-A] The paper credits prior IL-RL hybrid methods such as Lu et al. [22], Huang et al. [23], and Booher et al. [24], but it does not compare MTR-SAC against any of them. The only RL baseline, StateSAC, has very poor safety metrics (collision 0.328-0.339 and offroad 0.560-0.563 on both benchmarks), which makes MTR-SAC look favorable by contrast. To support the statement that the joint IL-RL framework is a strong baseline, the paper needs at least one comparison to an existing hybrid method under the same simulator and evaluation protocol.
minor comments (4)
  1. [III-C-2] The smoothness reward term is written as "-0.5 * logit or(∆accel > 1.5, ∆turning > 0.1)"; it is unclear whether "logit" is a typo for "logical" (i.e., a logical OR returning a binary flag) or a logistic function. Please clarify the notation and the intended functional form.
  2. [IV-D, Table II caption] The caption contains a typo: "COMPARSION" should be "COMPARISON". Similarly, the tables use inconsistent column header spacing, e.g., "Compl." vs "UnCompl."; standardizing would improve readability.
  3. [VI, Limitations] The limitations section states that certain aspects are overlooked "such as the physical constraints of vehicles," but it does not explicitly acknowledge that the reported MTR-SAC results show large increases in collision and offroad rates relative to IL-only. This is directly relevant to the safety claim and should be stated plainly.
  4. [Abstract and I] The abstract promises "The code for this work will be released soon," but no repository link or artifact description is provided. Given that the simulator is a major contribution, the paper should either include a link or state the intended release platform.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: the paper is an empirical systems/benchmark contribution, and its reward/evaluation overlap is a normal training loop rather than a construction that assumes its conclusion.

full rationale

This paper does not contain a formal derivation chain whose conclusion is equivalent to its inputs. The three contributions are an engineering artifact (simulator), a data-construction procedure (causal benchmark), and an empirical training method (MTR-SAC). None is obtained by defining an input in terms of an output or by renaming a fitted quantity as a prediction. The DFS goal generator uses a hand-set cost table, a max-cost threshold, equal-distance selection, and NMS; the goals are not derived from the evaluation metrics. MTR is used as a fixed pretrained backbone and is explicitly tied to an externally benchmarked open-source model, so the self-citations to MTR are not load-bearing in a circular sense and are not invoked to forbid alternative designs. The RL reward includes progress and completion terms that align with the paper's completion evaluation metric, but this is a standard training-to-evaluation loop, not a statistical fit disguised as a prediction, and the paper does not claim to derive the completion numbers analytically from the reward function. The abstract's 'safety' claim is contradicted by the paper's own Tables III and IV, where MTR-SAC increases collision and offroad rates relative to imitation-only baselines; however, that is an evidence mismatch or correctness concern, not circularity. The Limitations section openly concedes that the fusion strategy is still an open question and that physical constraints are overlooked, which further indicates the paper is not presenting a closed, self-justifying derivation. No circular step can be exhibited with the required quote-and-reduction standard, so the honest finding is 'no significant circularity.'

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central results rest on hand-chosen benchmark generation parameters (DFS costs, NMS threshold), reward weights that were selected on the evaluation set, and domain assumptions about Waymo scenarios, simulator dynamics, and metric validity. No new entities are invented.

free parameters (5)
  • DFS cost table = LC:5, TL:1, TR:1, Go:1
    Hand-chosen costs in Algorithm 1 to prune unrealistic lane-change sequences during goal generation.
  • MaxCostThres = 10
    Hand-set cutoff in Algorithm 1 that limits how many lane changes / turns a generated path can include; affects the diversity and reachability of goals.
  • NmsThres = 2.5 meters
    Hand-set NMS distance threshold removing nearby endpoints from the generated goal set.
  • Reward weights = (1,1,10) for offroad, collision, progress
    The headline MTR-SAC result in Table IV uses the (1,1,10) weight set, which was selected after ablating on Causality9k (Table V); this is a free parameter fitted to the evaluation set.
  • Completion threshold = 2 meters
    A scenario counts as complete when the SDC reaches within 2 m of the goal; this threshold is chosen without sensitivity analysis.
assumptions (4)
  • domain assumption The Waymo Open Dataset scenarios filtered to contain an ego index are a representative sample of real autonomous driving conditions.
    Section IV-A uses roughly 50k filtered training scenes; the paper does not test for selection bias introduced by removing scenarios without an ego index.
  • domain assumption DFS-generated goals from the lane graph are valid, reachable endpoints that do not themselves bias the completion metric.
    Section III-B Algorithm 1 assumes the cost table and equidistant goal selection produce comparable-difficulty goals; no reachability check is performed.
  • domain assumption The kinematic bicycle model and IDM are adequate proxies for real vehicle dynamics and reactive traffic during closed-loop evaluation.
    Section IV-C uses bicycle dynamics and IDM NPCs; the paper acknowledges 'physical constraints of vehicles' are still overlooked in the Limitations.
  • domain assumption Bounding box overlap and road-edge distance capture true collision and off-road safety.
    Section III-A defines collision as bounding-box intersection and offroad as crossing the road edge; no validation against real-world incidents is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exposing the Copycat Problem of Imitation-based Planner: A Novel Closed-Loop Simulator, Causal Benchmark and Joint IL-RL Baseline." pith.science (2026). https://pith.science/paper/5XIZIU7O

@misc{pith2026250414709,
  author       = {Pith},
  title        = {Pith review of: Exposing the Copycat Problem of Imitation-based Planner: A Novel Closed-Loop Simulator, Causal Benchmark and Joint IL-RL Baseline},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5XIZIU7O}},
  note         = {Machine review of arXiv:2504.14709}
}
read the original abstract

Machine learning (ML)-based planners have recently gained significant attention. They offer advantages over traditional optimization-based planning algorithms. These advantages include fewer manually selected parameters and faster development. Within ML-based planning, imitation learning (IL) is a common algorithm. It primarily learns driving policies directly from supervised trajectory data. While IL has demonstrated strong performance on many open-loop benchmarks, it remains challenging to determine if the learned policy truly understands fundamental driving principles, rather than simply extrapolating from the ego-vehicle's initial state. Several studies have identified this limitation and proposed algorithms to address it. However, these methods often use original datasets for evaluation. In these datasets, future trajectories are heavily dependent on initial conditions. Furthermore, IL often overfits to the most common scenarios. It struggles to generalize to rare or unseen situations. To address these challenges, this work proposes: 1) a novel closed-loop simulator supporting both imitation and reinforcement learning, 2) a causal benchmark derived from the Waymo Open Dataset to rigorously assess the impact of the copycat problem, and 3) a novel framework integrating imitation learning and reinforcement learning to overcome the limitations of purely imitative approaches. The code for this work will be released soon.

Figures

Figures reproduced from arXiv: 2504.14709 by the authors.

Figure 1
Figure 1. Illustration of the copycat problem. (a)-(b) At a certain intersection, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The framework is divided into three parts. The first part is the closed-loop simulator, which initializes the environment, executes actions, and outputs [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. One typical example in our causality benchmark illustrates going straight, turning right, and turning left. The red point is the goal. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Success and failure cases in four scenarios: U-Turn, Left-Turn, Right-Turn, and Go-Straight. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 19 canonical work pages

  1. [22]

    Imitation is not enough: Ro- bustifying imitation with reinforcement learning for challenging driving scenarios,

    Y . Lu, J. Fu, G. Tucker, X. Pan, E. Bronstein, R. Roelofs, B. Sapp, B. White, A. Faust, S. Whiteson et al. , “Imitation is not enough: Ro- bustifying imitation with reinforcement learning for challenging driving scenarios,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2023, pp. 7553–7560

    2023

  2. [23]

    Efficient deep reinforcement learning with imitative expert priors for autonomous driving,

    Z. Huang, J. Wu, and C. Lv, “Efficient deep reinforcement learning with imitative expert priors for autonomous driving,” IEEE Transactions on Neural Networks and Learning Systems , vol. 34, no. 10, pp. 7391–7403, 2022

    work page 2022

  3. [24]

    Cimrl: Combining imitation and reinforcement learning for safe autonomous driving,

    J. Booher, K. Rohanimanesh, J. Xu, V . Isenbaev, A. Balakrishna, I. Gupta, W. Liu, and A. Petiushko, “Cimrl: Combining imitation and reinforcement learning for safe autonomous driving,” arXiv preprint arXiv:2406.08878, 2024

    arXiv 2024

  4. [1]

    nuplan: A closed-loop ml- based planning benchmark for autonomous vehicles,

    H. Caesar, J. Kabzan, K. S. Tan, W. K. Fong, E. Wolff, A. Lang, L. Fletcher, O. Beijbom, and S. Omari, “nuplan: A closed-loop ml- based planning benchmark for autonomous vehicles,” arXiv preprint arXiv:2106.11810, 2021

    arXiv 2021

  5. [2]

    Parting with misconceptions about learning-based vehicle motion planning,

    D. Dauner, M. Hallgarten, A. Geiger, and K. Chitta, “Parting with misconceptions about learning-based vehicle motion planning,” in Con- ference on Robot Learning (CoRL) , 2023

    work page 2023

  6. [3]

    Planning- oriented autonomous driving,

    Y . Hu, J. Yang, L. Chen, K. Li, C. Sima, X. Zhu, S. Chai, S. Du, T. Lin, W. Wang, L. Lu, X. Jia, Q. Liu, J. Dai, Y . Qiao, and H. Li, “Planning- oriented autonomous driving,” in Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR) , June 2023, pp. 17 853–17 862

    work page 2023

  7. [4]

    End-to-end autonomous driving: Challenges and frontiers,

    L. Chen, P. Wu, K. Chitta, B. Jaeger, A. Geiger, and H. Li, “End-to-end autonomous driving: Challenges and frontiers,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024

    2024

  8. [5]

    Efficient sampling-based motion planning for on-road autonomous driving,

    L. Ma, J. Xue, K. Kawabata, J. Zhu, C. Ma, and N. Zheng, “Efficient sampling-based motion planning for on-road autonomous driving,” IEEE Transactions on Intelligent Transportation Systems , vol. 16, no. 4, pp. 1961–1976, 2015

    1961

Show all 36 references
  1. [6]

    Shortcut learning in deep neural networks,

    R. Geirhos, J.-H. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, M. Bethge, and F. A. Wichmann, “Shortcut learning in deep neural networks,” Nature Machine Intelligence , vol. 2, no. 11, pp. 665–673, 2020

    2020

  2. [7]

    Is ego status all you need for open-loop end-to-end autonomous driving?

    Z. Li, Z. Yu, S. Lan, J. Li, J. Kautz, T. Lu, and J. M. Alvarez, “Is ego status all you need for open-loop end-to-end autonomous driving?” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 14 864–14 873

    2024

  3. [8]

    Fighting copycat agents in behavioral cloning from observation histories,

    C. Wen, J. Lin, T. Darrell, D. Jayaraman, and Y . Gao, “Fighting copycat agents in behavioral cloning from observation histories,” Advances in Neural Information Processing Systems , vol. 33, pp. 2564–2575, 2020

    2020

  4. [9]

    Resolving copycat problems in visual imitation learning via residual action prediction,

    C.-C. Chuang, D. Yang, C. Wen, and Y . Gao, “Resolving copycat problems in visual imitation learning via residual action prediction,” in European Conference on Computer Vision . Springer, 2022, pp. 392– 409

    2022

  5. [10]

    Waymo open dataset,

    Waymo, “Waymo open dataset,” available at: https://waymo.com/open/ data/

  6. [11]

    Motion transformer with global intention localization and local movement refinement,

    S. Shi, L. Jiang, D. Dai, and B. Schiele, “Motion transformer with global intention localization and local movement refinement,” Advances in Neural Information Processing Systems , vol. 35, pp. 6531–6543, 2022

    2022

  7. [12]

    Mtr++: Multi-agent motion prediction with symmetric scene modeling and guided intention query- ing,

    S. Shi, L. Jiang, D. Dai, and B. Sch iele, “Mtr++: Multi-agent motion prediction with symmetric scene modeling and guided intention query- ing,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024

    2024

  8. [13]

    End to end learning for self-driving cars

    M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller, J. Zhang et al. , “End to end learning for self-driving cars.”

  9. [14]

    End-to-end driving via conditional imitation learning,

    F. Codevilla, M. M ¨uller, A. L ´opez, V . Koltun, and A. Dosovitskiy, “End-to-end driving via conditional imitation learning,” in 2018 IEEE international conference on robotics and automation (ICRA) . IEEE, 2018, pp. 4693–4700

    2018

  10. [15]

    Vad: Vectorized scene representation for efficient autonomous driving,

    B. Jiang, S. Chen, Q. Xu, B. Liao, J. Chen, H. Zhou, Q. Zhang, W. Liu, C. Huang, and X. Wang, “Vad: Vectorized scene representation for efficient autonomous driving,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 8340–8350

    2023

  11. [16]

    Controlling an autonomous vehicle with deep reinforcement learning,

    A. Folkers, M. Rick, and C. B ¨uskens, “Controlling an autonomous vehicle with deep reinforcement learning,” in 2019 IEEE Intelligent V ehicles Symposium (IV). IEEE, 2019, pp. 2025–2031. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 8 Fig. 4. Success and failure ca...

    2019

  12. [17]

    Prox- imal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,” arXiv preprint arXiv:1707.06347 , 2017

    2017 arXiv

  13. [18]

    Deep reinforcement learning for autonomous driving,

    S. Wang, D. Jia, and X. Weng, “Deep reinforcement learning for autonomous driving,” arXiv preprint arXiv:1811.11329 , 2018

    2018 arXiv

  14. [19]

    Highway decision-making and motion planning for autonomous driving via soft actor-critic,

    X. Tang, B. Huang, T. Liu, and X. Lin, “Highway decision-making and motion planning for autonomous driving via soft actor-critic,” IEEE Transactions on V ehicular Technology, vol. 71, no. 5, pp. 4706–4717, 2022

    2022

  15. [20]

    Soft actor-critic algorithms and applications,

    T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, V . Ku- mar, H. Zhu, A. Gupta, P. Abbeel et al. , “Soft actor-critic algorithms and applications,” arXiv preprint arXiv:1812.05905 , 2018

    2018 arXiv

  16. [21]

    Congested traffic states in empirical observations and microscopic simulations,

    M. Treiber, A. Hennecke, and D. Helbing, “Congested traffic states in empirical observations and microscopic simulations,” Physical review E, vol. 62, no. 2, p. 1805, 2000

    2000

  17. [25]

    Chauffeurnet: Learning to drive by imitating the best and synthesizing the worst,

    M. Bansal, A. Krizhevsky, and A. Ogale, “Chauffeurnet: Learning to drive by imitating the best and synthesizing the worst,” arXiv preprint arXiv:1812.03079, 2018

    2018 arXiv

  18. [26]

    Regularized behavior cloning for blocking the leakage of past action information,

    S. Seo, H. Hwang, H. Yang, and K.-E. Kim, “Regularized behavior cloning for blocking the leakage of past action information,” Advances in Neural Information Processing Systems , vol. 36, 2024

    2024

  19. [27]

    Opendrive 2010 and beyond–status and future of the de facto standard for the description of road networks,

    M. Dupuis, M. Strobl, and H. Grezlikowski, “Opendrive 2010 and beyond–status and future of the de facto standard for the description of road networks,” in Proc. of the Driving Simulation Conference Europe , 2010, pp. 231–242

    2010

  20. [28]

    Waymax: An accelerated, data-driven simulator for large-scale autonomous driving research,

    C. Gulino, J. Fu, W. Luo, G. Tucker, E. Bronstein, Y . Lu, J. Harb, X. Pan, Y . Wang, X. Chenet al., “Waymax: An accelerated, data-driven simulator for large-scale autonomous driving research,” Advances in Neural Information Processing Systems , vol. 36, 2024

    2024

  21. [29]

    Smarts: An open-source scalable multi-agent rl training school for autonomous driving,

    M. Zhou, J. Luo, J. Villella, Y . Yang, D. Rusu, J. Miao, W. Zhang, M. Alban, I. Fadakar, Z. Chen et al. , “Smarts: An open-source scalable multi-agent rl training school for autonomous driving,” in Conference on robot learning . PMLR, 2021, pp. 264–285

    2021

  22. [30]

    Metadrive: Composing diverse driving scenarios for generalizable reinforcement learning,

    Q. Li, Z. Peng, L. Feng, Q. Zhang, Z. Xue, and B. Zhou, “Metadrive: Composing diverse driving scenarios for generalizable reinforcement learning,” IEEE transactions on pattern analysis and machine intelli- gence, vol. 45, no. 3, pp. 3461–3475, 2022

    2022

  23. [31]

    Intersim: Interactive traffic simulation via explicit relation modeling,

    Q. Sun, X. Huang, B. C. Williams, and H. Zhao, “Intersim: Interactive traffic simulation via explicit relation modeling,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2022, pp. 11 416–11 423

    2022

  24. [32]

    Bits: Bi-level imitation for traffic simulation,

    D. Xu, Y . Chen, B. Ivanovic, and M. Pavone, “Bits: Bi-level imitation for traffic simulation,” in 2023 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2023, pp. 2929–2936

    2023

  25. [33]

    Tactics2d: A multi- agent reinforcement learning environment for driving decision-making,

    Y . Li, S. Zhang, M. Jiang, X. Chen, and M. Yang, “Tactics2d: A multi- agent reinforcement learning environment for driving decision-making,” arXiv preprint arXiv:2311.11058 , 2023

    2023 arXiv

  26. [34]

    The kinematic bicycle model: A consistent model for planning feasible trajectories for autonomous vehicles?

    P. Polack, F. Altch ´e, B. d’Andr ´ea Novel, and A. de La Fortelle, “The kinematic bicycle model: A consistent model for planning feasible trajectories for autonomous vehicles?” in 2017 IEEE intelligent vehicles symposium (IV) . IEEE, 2017, pp. 812–818

    2017

  27. [35]

    The waymo open sim agents challenge,

    N. Montali, J. Lambert, P. Mougin, A. Kuefler, N. Rhinehart, M. Li, C. Gulino, T. Emrich, Z. Yang, S. Whitesonet al., “The waymo open sim agents challenge,” Advances in Neural Information Processing Systems , vol. 36, 2024

    2024

  28. [36]

    Causalagents: A robustness benchmark for motion forecasting,

    L. Sun, R. Roelofs, B. Caine, K. S. Refaat, B. Sapp, S. Ettinger, and W. Chai, “Causalagents: A robustness benchmark for motion forecasting,” in 2024 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2024, pp. 6820–6827

    2024

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.