Pith. sign in

REVIEW 3 major objections 5 minor 38 references

Bootstrapping Reinforcement Learning with Sub-optimal Policies for Autonomous Driving

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A sub-optimal rule-based lane-change controller, used as a KL soft constraint and as reward-augmented replay data, lets a Soft Actor-Critic agent learn a backward-gap overtaking maneuver that standard RL baselines never discover.

desk verdict A sensible demonstration-guided RL combination for a specific highway trap, but the baseline comparison needs scrutiny before the claimed advantage holds. read the letter →

arxiv 2509.04712 v1 pith:7NWYSE2H submitted 2025-09-04 cs.RO cs.AIcs.LGcs.SYeess.SY

Zhihao Zhang , Chengyang Peng , Ekim Yurtsever , Keith A. Redmill This is my paper · ORCID
classification cs.ROcs.AIcs.LGcs.SYeess.SY
keywords reinforcementlearningautonomousdrivingSoftActor-CriticsuboptimaldemonstrationsKLdivergenceconstraintofflinereplaybufferhighwayovertakingexploration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a cheap, rule-based driving controller that is only feasible—not expert—can bootstrap a reinforcement-learning agent through an overtaking maneuver that ordinary RL never finds on its own. The target scenario is a highway "trap" where two slow vehicles force the ego car to brake unless it exploits a backward gap to overtake. The authors integrate the rule-based controller into Soft Actor-Critic twice: as a KL-divergence soft constraint that biases early exploration, and as demonstration transitions stored in the replay buffer with a reward bonus. In their simulations, the combination reaches 100% escape success with no collisions, while plain SAC, CQL, and GAIL all remain stuck at 0% success. The practical claim is that cheap suboptimal heuristics can replace expensive expert demonstrations for guiding RL in long-horizon driving tasks.

What carries the argument

The load-bearing mechanism is the two-part integration of a heuristic controller into SAC. First, a stochastic version of the deterministic rule-based controller is built by putting probability 0.9 on its chosen action and spreading the rest uniformly; its Kullback-Leibler divergence from the learned policy is added as a penalty weighted by the entropy coefficient, keeping early policy behavior near the heuristic and relaxing as training proceeds. Second, 200 episodes of demonstration transitions are stored in the replay buffer, and training samples them at an initial ratio of 0.6 that anneals to 0, with a constant +2 reward bonus on demonstrated actions. The rule-based controller itself sel

What would settle it

Train plain SAC for five times more episodes with the same entropy schedule and reward function; if it begins escaping the trap at a non-zero rate, the claim that the demonstration prior is necessary for overcoming the exploration barrier is falsified. Alternatively, add a small hand-shaped reward for approaching the backward gap and test whether it alone reproduces the proposed method's 100% success, which would show the specific KL-plus-replay mechanism is not the decisive ingredient.

Watch

Extended reading notes

Core claim

The paper claims that an RL driving agent can be bootstrapped to an optimal overtaking policy by a deliberately suboptimal, rule-based lane-change controller used in two ways: as a KL-divergence soft constraint that nudges the SAC policy toward the heuristic early in training, and as a source of reward-augmented demonstration transitions placed in the replay buffer with an annealed sampling ratio. In a multilane highway "trap" where two slow vehicles block the ego vehicle, the suboptimal controller itself always escapes by overtaking through the backward gap, while standard SAC, CQL, and GAIL all converge to conservative car-following and never escape. With both guidance mechanisms active, t

Load-bearing premise

The comparison rests on the premise that vanilla SAC, CQL, and GAIL fail only because the backward-gap overtaking maneuver is hard to discover through exploration, and that the simulation and reward details taken from the authors' prior work are faithfully reproduced; if either premise fails, the reported superiority could be an artifact of setup rather than a genuine exploration barrier.

Editorial extensions

If this is right

  • A controller that is merely feasible—not optimal or expert-level—can serve as a bootstrap prior for RL in long-horizon driving tasks with delayed rewards.
  • Reward augmentation on demonstration transitions beats a Q-value margin loss when integrating suboptimal demonstrations into SAC.
  • The combination of a temporary KL soft constraint and an annealed offline replay buffer yields higher success, reward, speed, and distance than either mechanism alone.
  • The same recipe should transfer to other driving scenarios where a simple, safe heuristic can exhibit less-obvious but feasible behaviors.
  • Because the suboptimal controller itself escapes the trap every time, the demonstration dataset is reliably informative, even though the controller's raw reward is modest.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The entropy analysis suggests guided exploration can be reframed as reallocating an exploration budget toward promising actions rather than simply increasing randomness, a principle that could extend to any sparse-reward MDP with a cheap feasible policy, not just driving.
  • The ablation's 5% collision rate for the offline-buffer-only variant hints that diverse offline data can destabilize safety-critical policies unless a soft constraint is present; the paper does not test this interaction beyond the reported ablation.
  • A testable extension would vary the quality of the demonstration policy—for example, deliberately delayed lane changes—to map how much suboptimality the KL-plus-replay combination can tolerate before the agent stops improving.
  • The paper's success criterion counts episodes where the ego overtakes the first trap vehicle even if it later crashes; readers should check whether safety metrics, rather than raw success rate, are what drive the reported superiority.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes to bootstrap Soft Actor-Critic (SAC) for an autonomous highway overtaking task by incorporating a rule-based, suboptimal lane-change controller. The controller is used in two ways: as a KL-divergence soft constraint during early training and as a source of offline demonstration transitions added to the replay buffer with a constant reward bonus. Experiments in the highway-env simulator report that the proposed combination achieves 100% success in escaping a 'trap' of slow vehicles, outperforming SAC, CQL, and GAIL, which all obtain 0% success. An ablation study indicates that the best integration strategy is reward augmentation, and that combining the soft constraint with the offline replay buffer yields the highest reward and zero collisions.

Significance. If the empirical claims are reliable, the paper offers a practical recipe for using a simple, non-expert controller to guide RL exploration in a challenging driving scenario where standard RL agents get stuck in conservative behavior. The idea is plausible and the paper is clearly written. However, the current evidence is not fully convincing: the success metric is defined in a way that counts post-overtake crashes as successes, the baseline results show a suspicious pattern consistent with implementation collapse, and the ablation does not establish the necessity of the proposed combination. These issues are load-bearing for the central claim that the suboptimal policy provides a meaningful bootstrap.

major comments (3)
  1. [Sec. III-C4, Eq. (9) and Sec. V-B, Tables IV-V] The success metric counts episodes as successful even if the ego vehicle crashes or departs the road after overtaking the trap vehicle. This means the reported '100±0' success rates in Tables IV and V can include episodes with critical failures. Since the paper also separately reports collision rate, the reader can partially recover the truth, but the headline claim is inflated. The success rate should be redefined to require collision-free completion, or at least the success and collision metrics should be combined into a single safety-aware success criterion.
  2. [Sec. V-B, Table V and Sec. III-B/III-C3] CQL and GAIL are trained on a demonstration dataset that the paper states is 100% successful, yet both achieve 0% success in testing with almost identical constant-speed behavior (CQL: 10.00 m/s, 1500 m; GAIL: 9.99 m/s, 1488 m). This pattern is more consistent with a degenerate policy or baseline misconfiguration than with an exploration failure. The paper defers all simulation parameters, reward details, and baseline configurations to prior work [16], and reports no seed counts or code release. Without evidence that the baselines are properly tuned, the comparison in Table V does not support the claim that the proposed method overcomes an exploration barrier. The authors should provide complete hyperparameters, per-seed results, learning curves, and ideally open-source code.
  3. [Sec. V-A, Table IV] The ablation shows that SAC with the reward-augmented offline replay buffer alone already reaches 100% success, while adding the KL soft constraint changes the collision rate from 5±29.69% to 0±0%. The difference is not significance-tested, and the large standard deviation suggests it could stem from a single episode in one seed. The paper's conclusion that the combination of soft constraints and offline replay buffer 'better balances the exploration and exploitation trade-off' is not statistically supported. More seeds and a significance test (or at least per-seed reporting) are needed to substantiate the claimed advantage of the combination.
minor comments (5)
  1. [Sec. V-A title] The section header 'Abalation Study' should read 'Ablation Study.'
  2. [Sec. IV-A, Eq. (15)] The KL soft-constraint weight is set to ξ = 200·α, where α is the entropy coefficient that changes during training. The paper says ξ is 'relaxed' over time, but the exact schedule is not defined. Please specify how α evolves and how ξ is annealed in practice.
  3. [Sec. V-B, last paragraph] The text states that 'GAIL and CQL utilize demonstration data effectively, achieving high success rate in replicating demonstrated behaviors,' but Table V reports 0% success for both methods. This phrasing is misleading and should be revised to refer to imitation accuracy or trajectory-level behavior rather than task success.
  4. [Sec. III-D, Algorithm 1] The algorithm description is incomplete: it outputs a target speed and lane index but does not specify the lower-level controller that executes the lane change. The reader is referred to [16] for this detail, which is acceptable only if the referenced work is publicly available; please make the dependency explicit in the text.
  5. [General] Figures 4-6 show training curves without error bars or confidence intervals, while tables report mean±std. Adding shaded regions for variance would help assess the stability of the results, especially given the small apparent differences in the ablation.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the main result is an empirical comparison, not a derivation that reduces to its own inputs.

full rationale

The paper's central claim is that combining a suboptimal rule-based controller with SAC via a KL soft constraint and a reward-augmented offline replay buffer improves exploration and convergence in a highway overtaking task. This is an empirical claim, and the evaluation does not appear to be forced by construction. The demonstration policy (Algorithm 1) is designed to escape the trap, and the demonstration dataset is 100% successful by design, but that is an input to training, not a predicted outcome. The learned policy ("ours") achieves accumulated reward 44.92 ± 12.62 versus the suboptimal controller's 24.70 ± 0.44 and the offline-replay-buffer-only variant's 36.68 ± 9.66 (Table IV/V), so the agent is not merely reproducing the demonstrator. The reward augmentation (Eq. 20) adds a constant bonus to demonstrated actions, but this shapes the Q-function without pinning the final policy to the demonstrator; the higher speed, distance, and reward show the policy surpasses it. The comparison against SAC, CQL, and GAIL is external and the baselines are standard, so the finding is not a self-citation artifact. The exploration-barrier premise is supported by the paper's own vanilla SAC 0% success result in Table V, not solely by the self-citation [16]. There are deferrals to prior work by overlapping authors: 'Detailed configurations of the traffic vehicles follow our previous work [16]' (Sec. III-A), 'A detailed explanation of the reward design can be found in our previous study [16]' (Sec. III-C3), and the lower-level lane-change controller is also deferred to [16] (Sec. III-D). These are missing-support/reproducibility concerns, not circular reductions: the paper does not define its target result in terms of [16], nor does it use [16] as a proof of the main claim. The ablation showing the offline-replay-buffer arm already achieves 100% success and that adding the soft constraint changes collision rate from 5% to 0% is weak statistical evidence, but that is a correctness/robustness issue, not circularity. Overall, I find no step in the derivation chain where a 'prediction' is equivalent to a fitted input or a self-citation chain; score 1 reflects the minor but non-load-bearing self-citation and deferred implementation details.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central result rests on the design of the suboptimal controller and the demonstration integration. The free parameters c, xi, p, beta, and t_safe are hand-chosen without sensitivity analysis. The axioms include the realism of the simulation, the correctness of the reward function, the appropriateness of the backward-gap demonstration, and the validity of deferring implementation details to prior work.

free parameters (5)
  • Reward augmentation constant c = 2
    Eq. (20): added to demonstrated actions in the replay buffer. Chosen empirically without sensitivity analysis.
  • KL soft constraint multiplier (xi = 200 * alpha) = 200
    Section IV-A: the weight of the KL divergence term is set to 200 times the learned entropy coefficient alpha. Hand-chosen.
  • Demonstration policy stochasticity p = 0.9
    Eq. (16): probability assigned to the deterministic rule-based action. Chosen by hand.
  • Initial offline sampling ratio beta = 0.6
    Section IV-B: annealed to 0 over 1000 episodes. Chosen by hand.
  • Safety time distance t_safe = not specified
    Eq. (13): used in the suboptimal controller's safety-distance metric; no value is provided, making the controller underspecified.
assumptions (4)
  • domain assumption The simulated environment (highway-env with IDM and MOBIL) accurately models highway traffic dynamics sufficient for evaluating driving policies.
    Sections III-A and III-B describe the simulation setup; the conclusion that the method 'improves driving performance' depends on the simulator being a meaningful testbed.
  • domain assumption The hand-designed reward function (Eqs. 9-12) appropriately encodes driving objectives (speed, lane centering, smoothness) and that escaping the trap yields higher long-term reward.
    Section III-C3 defines the reward; if the reward misaligns with actual driving quality, the learned policy may be optimized for the wrong objective.
  • ad hoc to paper The suboptimal controller's backward-gap overtaking strategy is a feasible and beneficial behavior to demonstrate; guiding the agent toward it hastens discovery of a good policy.
    Section I argues this strategy is 'safe, general, and relatively easy for humans to identify and design', but no proof or comparison with other strategies is given.
  • domain assumption Details from the authors' prior work [16] (lower-level controller, environment configuration, reward design explanation) are correct and reproducible.
    Sections III-A, III-C3, III-D refer to [16] for critical implementation details; if [16] is flawed or differs, the current results are not self-contained.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bootstrapping Reinforcement Learning with Sub-optimal Policies for Autonomous Driving." pith.science (2026). https://pith.science/paper/7NWYSE2H

@misc{pith2026250904712,
  author       = {Pith},
  title        = {Pith review of: Bootstrapping Reinforcement Learning with Sub-optimal Policies for Autonomous Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7NWYSE2H}},
  note         = {Machine review of arXiv:2509.04712}
}
read the original abstract

Automated vehicle control using reinforcement learning (RL) has attracted significant attention due to its potential to learn driving policies through environment interaction. However, RL agents often face training challenges in sample efficiency and effective exploration, making it difficult to discover an optimal driving strategy. To address these issues, we propose guiding the RL driving agent with a demonstration policy that need not be a highly optimized or expert-level controller. Specifically, we integrate a rule-based lane change controller with the Soft Actor Critic (SAC) algorithm to enhance exploration and learning efficiency. Our approach demonstrates improved driving performance and can be extended to other driving scenarios that can similarly benefit from demonstration-based guidance.

Figures

Figures reproduced from arXiv: 2509.04712 by the authors.

Figure 1
Figure 1. In complex traffic conditions, multiple driving [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. A highway ”Trap” situation is represented by two vehicles moving slower than the typical traffic, forcing [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. We leverage the suboptimal policy in two ways: (1) as a soft constraint on the RL policy during initial [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Training comparison between integrating demonstration data via marginal loss and reward augmentation: (a) [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Performance during training episodes for SAC with soft constraint, SAC with offline replay buffer, and our [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Training performance comparison with other RL baseline methods: (a) Average success rate in escaping [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Comparison of (a) policy entropy, (b) effective [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 26 canonical work pages

  1. [16]

    Extensive Exploration in Complex Traffic Scenarios using Hierarchical Reinforcement Learning

    Z. Zhang, E. Yurtsever, and K. A. Redmill, “Extensive explo- ration in complex traffic scenarios using hierarchical reinforce- ment learning,”arXiv preprint arXiv:2501.14992, 2025

    work page Pith review arXiv 2025

  2. [1]

    A survey of autonomous driving: Common practices and emerging tech- nologies,

    E. Yurtsever, J. Lambert, A. Carballo, and K. Takeda, “A survey of autonomous driving: Common practices and emerging tech- nologies,”IEEE access, vol. 8, pp. 58 443–58 469, 2020

    work page 2020

  3. [2]

    Lane change and merge maneuvers for connected and automated vehicles: A survey,

    D. Bevly, X. Cao, M. Gordon, G. Ozbilgin, D. Kari, B. Nelson, J. Woodruff, M. Barth, C. Murray, A. Kurtet al., “Lane change and merge maneuvers for connected and automated vehicles: A survey,”IEEE Transactions on Intelligent Vehicles, vol. 1, no. 1, pp. 105–120, 2016

    work page 2016

  4. [3]

    Automated lane change controller design,

    C. Hatipoglu, U. Ozguner, and K. A. Redmill, “Automated lane change controller design,”IEEE transactions on intelligent transportation systems, vol. 4, no. 1, pp. 13–22, 2003

    work page 2003

  5. [4]

    Traffic dynam- ics: studies in car following,

    R. E. Chandler, R. Herman, and E. W. Montroll, “Traffic dynam- ics: studies in car following,”Operations research, vol. 6, no. 2, pp. 165–184, 1958

    work page 1958

  6. [5]

    A behavioural car-following model for computer simulation,

    P. G. Gipps, “A behavioural car-following model for computer simulation,”Transportation research part B: methodological, vol. 15, no. 2, pp. 105–111, 1981

    work page 1981

  7. [6]

    Congested traffic states in empirical observations and microscopic simulations,

    M. Treiber, A. Hennecke, and D. Helbing, “Congested traffic states in empirical observations and microscopic simulations,” Physical review E, vol. 62, no. 2, p. 1805, 2000

    2000

  8. [7]

    General lane-changing model mobil for car-following models,

    A. Kesting, M. Treiber, and D. Helbing, “General lane-changing model mobil for car-following models,”Transportation Research Record, vol. 1999, no. 1, pp. 86–94, 2007

    1999

Show all 38 references
  1. [8]

    Driving intention recognition and lane change prediction on the highway,

    T. Han, J. Jing, and ¨U. ¨Ozg¨uner, “Driving intention recognition and lane change prediction on the highway,” in2019 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2019, pp. 957–962

    2019

  2. [9]

    End to end learning for self-driving cars,

    M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller, J. Zhang et al., “End to end learning for self-driving cars,”arXiv preprint arXiv:1604.07316, 2016

    2016 arXiv

  3. [10]

    Explaining how a deep neural network trained with end-to-end learning steers a car,

    M. Bojarski, P. Yeres, A. Choromanska, K. Choromanski, B. Firner, L. Jackel, and U. Muller, “Explaining how a deep neural network trained with end-to-end learning steers a car,” arXiv preprint arXiv:1704.07911, 2017

    2017 arXiv

  4. [11]

    End-to-end driving via conditional imitation learning,

    F. Codevilla, M. M ¨uller, A. L ´opez, V . Koltun, and A. Dosovit- skiy, “End-to-end driving via conditional imitation learning,” in 2018 IEEE international conference on robotics and automation (ICRA). IEEE, 2018, pp. 4693–4700

    2018

  5. [12]

    Urban driving with conditional imitation learning,

    J. Hawke, R. Shen, C. Gurau, S. Sharma, D. Reda, N. Nikolov, P. Mazur, S. Micklethwaite, N. Griffiths, A. Shahet al., “Urban driving with conditional imitation learning,” in2020 IEEE Inter- national Conference on Robotics and Automation (ICRA). IEEE, 2020, pp. 251–257. 12

    2020

  6. [13]

    R. S. Sutton, A. G. Bartoet al.,Reinforcement learning: An introduction. MIT press Cambridge, 1998, vol. 1, no. 1

    1998

  7. [14]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  8. [15]

    Soft actor- critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor- critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,” inInternational conference on machine learning. Pmlr, 2018, pp. 1861–1870

    2018

  9. [17]

    Exploiting hier- archy for scalable decision making in autonomous driving,

    E. Sonu, Z. Sunberg, and M. J. Kochenderfer, “Exploiting hier- archy for scalable decision making in autonomous driving,” in 2018 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2018, pp. 2203–2208

    2018

  10. [18]

    Learning hierarchical behavior and motion planning for autonomous driv- ing,

    J. Wang, Y . Wang, D. Zhang, Y . Yang, and R. Xiong, “Learning hierarchical behavior and motion planning for autonomous driv- ing,” in2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020, pp. 2235–2242

    2020

  11. [19]

    Deep hierarchical rein- forcement learning for autonomous driving with distinct behav- iors,

    J. Chen, Z. Wang, and M. Tomizuka, “Deep hierarchical rein- forcement learning for autonomous driving with distinct behav- iors,” in2018 IEEE intelligent vehicles symposium (IV). IEEE, 2018, pp. 1239–1244

    2018

  12. [20]

    A rein- forcement learning approach to autonomous decision making of intelligent vehicles on highways,

    X. Xu, L. Zuo, X. Li, L. Qian, J. Ren, and Z. Sun, “A rein- forcement learning approach to autonomous decision making of intelligent vehicles on highways,”IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 50, no. 10, pp. 3884–3897, 2018

    2018

  13. [21]

    Integrating deep reinforcement learning with model-based path planners for automated driving,

    E. Yurtsever, L. Capito, K. Redmill, and U. Ozgune, “Integrating deep reinforcement learning with model-based path planners for automated driving,” in2020 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2020, pp. 1311–1316

    2020

  14. [22]

    Lane change decision-making through deep reinforcement learning with rule- based constraints,

    J. Wang, Q. Zhang, D. Zhao, and Y . Chen, “Lane change decision-making through deep reinforcement learning with rule- based constraints,” in2019 International Joint Conference on Neural Networks (IJCNN). IEEE, 2019, pp. 1–6

    2019

  15. [23]

    Combining reinforcement learning with rule- based controllers for transparent and general decision-making in autonomous driving,

    A. Likmeta, A. M. Metelli, A. Tirinzoni, R. Giol, M. Restelli, and D. Romano, “Combining reinforcement learning with rule- based controllers for transparent and general decision-making in autonomous driving,”Robotics and Autonomous Systems, vol. 131, p. 103568, 2020

    2020

  16. [24]

    A combined reinforcement learning and model predictive control for car- following maneuver of autonomous vehicles,

    L. Wang, S. Yang, K. Yuan, Y . Huang, and H. Chen, “A combined reinforcement learning and model predictive control for car- following maneuver of autonomous vehicles,”Chinese Journal of Mechanical Engineering, vol. 36, no. 1, p. 80, 2023

    2023

  17. [25]

    Combining reinforcement learning with model predic- tive control for on-ramp merging,

    J. Lubars, H. Gupta, S. Chinchali, L. Li, A. Raja, R. Srikant, and X. Wu, “Combining reinforcement learning with model predic- tive control for on-ramp merging,” in2021 IEEE International Intelligent Transportation Systems Conference (ITSC). IEEE, 2021, pp. 942–947

    2021

  18. [26]

    A hierarchical architecture for sequential decision-making in autonomous driving using deep reinforcement learning,

    M. Moghadam and G. H. Elkaim, “A hierarchical architecture for sequential decision-making in autonomous driving using deep reinforcement learning,”arXiv preprint arXiv:1906.08464, 2019

    1906 arXiv

  19. [27]

    Driving decision and control for automated lane change behavior based on deep reinforcement learning,

    T. Shi, P. Wang, X. Cheng, C.-Y . Chan, and D. Huang, “Driving decision and control for automated lane change behavior based on deep reinforcement learning,” in2019 IEEE intelligent trans- portation systems conference (ITSC). IEEE, 2019, pp. 2895– 2900

    2019

  20. [28]

    Prioritized experience- based reinforcement learning with human guidance for au- tonomous driving,

    J. Wu, Z. Huang, W. Huang, and C. Lv, “Prioritized experience- based reinforcement learning with human guidance for au- tonomous driving,”IEEE Transactions on Neural Networks and Learning Systems, vol. 35, no. 1, pp. 855–869, 2022

    2022

  21. [29]

    Learning to drive in a day,

    A. Kendall, J. Hawke, D. Janz, P. Mazur, D. Reda, J.-M. Allen, V .-D. Lam, A. Bewley, and A. Shah, “Learning to drive in a day,” in2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019, pp. 8248–8254

    2019

  22. [30]

    Efficient deep reinforcement learning with imitative expert priors for autonomous driving,

    Z. Huang, J. Wu, and C. Lv, “Efficient deep reinforcement learning with imitative expert priors for autonomous driving,” IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 10, pp. 7391–7403, 2022

    2022

  23. [31]

    Imitation is not enough: Robustifying imitation with reinforcement learning for challenging driving scenarios,

    Y . Lu, J. Fu, G. Tucker, X. Pan, E. Bronstein, R. Roelofs, B. Sapp, B. White, A. Faust, S. Whitesonet al., “Imitation is not enough: Robustifying imitation with reinforcement learning for challenging driving scenarios,” in2023 IEEE/RSJ International Conference on Intelligent ...

    2023

  24. [32]

    Boosted bellman residual minimization handling expert demonstrations,

    B. Piot, M. Geist, and O. Pietquin, “Boosted bellman residual minimization handling expert demonstrations,” inJoint European Conference on machine learning and knowledge discovery in databases. Springer, 2014, pp. 549–564

    2014

  25. [33]

    Deep q-learning from demonstrations,

    T. Hester, M. Vecerik, O. Pietquin, M. Lanctot, T. Schaul, B. Piot, D. Horgan, J. Quan, A. Sendonaris, I. Osbandet al., “Deep q-learning from demonstrations,” inProceedings of the AAAI conference on artificial intelligence, vol. 32, no. 1, 2018

    2018

  26. [34]

    Reward learning from human preferences and demonstrations in atari,

    B. Ibarz, J. Leike, T. Pohlen, G. Irving, S. Legg, and D. Amodei, “Reward learning from human preferences and demonstrations in atari,”Advances in neural information processing systems, vol. 31, 2018

    2018

  27. [35]

    Sqil: Imitation learning via reinforcement learning with sparse rewards,

    S. Reddy, A. D. Dragan, and S. Levine, “Sqil: Imitation learning via reinforcement learning with sparse rewards,”arXiv preprint arXiv:1905.11108, 2019

    1905 arXiv

  28. [36]

    An environment for autonomous driving decision- making,

    E. Leurent, “An environment for autonomous driving decision- making,” https://github.com/eleurent/highway-env, 2018

    2018

  29. [37]

    Conservative q- learning for offline reinforcement learning,

    A. Kumar, A. Zhou, G. Tucker, and S. Levine, “Conservative q- learning for offline reinforcement learning,”Advances in neural information processing systems, vol. 33, pp. 1179–1191, 2020

    2020

  30. [38]

    Generative adversarial imitation learning,

    J. Ho and S. Ermon, “Generative adversarial imitation learning,” Advances in neural information processing systems, vol. 29, 2016

    2016

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.