Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

Sharpening the Spear: Adaptive Expert-Guided Adversarial Attack Against DRL-based Autonomous Driving Policies

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that an adversarial driving attacker guided by cloned demonstrations of past attacks achieves higher collision rates, better attack efficiency, and more stable training than existing low-frequency attack baselines.

desk verdict A promising but overclaimed expert-guided adversarial attack method; the empirical headline fails on one of its own six evaluation cells, and the theory section needs a rewrite. read the letter →

arxiv 2506.18304 v1 pith:JJHYKJ7H submitted 2025-06-23 cs.LG cs.AI

Junchao Fan , Xuyang Lei , Xiaolin Chang This is my paper · ORCID
classification cs.LGcs.AI
keywords adversarialattacksdeepreinforcementlearningautonomousdrivinglow-frequencyexpert-guidedpolicyoptimizationmixture-of-expertsbehaviorcloningKL-divergenceregularization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that an adversarial agent can attack a deep-reinforcement-learning (DRL) driving policy more effectively and more stably by bootstrapping from successful past attacks instead of learning sparse attack timing from scratch. It claims that a behavior-cloned expert policy, trained on successful demonstration trajectories and reinforced with an ensemble mixture-of-experts architecture, gives a reliable reference that a PPO-based adversary can be steered toward with a KL-divergence regularizer. Since the expert can be suboptimal in unseen situations, a performance-aware annealing schedule gradually weakens the regularizer as the adversary improves. If the claim holds, vulnerability testing of autonomous-driving policies could be done with fewer perturbations and less unstable training.

What carries the argument

The load-bearing mechanism is an imitation-learned expert policy that shapes a PPO adversary without pinning it down. The expert is an ensemble of mixture-of-experts networks: each MoE routes each state to specialist networks, and the ensemble's output mean and variance act as a confidence estimate, so the adversary is pulled hard toward the expert only where the ensemble is confident. The guidance itself is a KL-divergence regularizer, a measure of how far two action distributions are apart, inserted into the PPO objective, and a performance-aware annealing coefficient beta_e decays as measured episode returns approach an optimal reference return, so the adversary gradually outgrows the expert. A policy-improvement lemma is used to justify the early constraint, and a short argument shows that keeping the regularizer fixed would cap performance below the optimal policy.

What would settle it

A decisive check would retrain the expert on demonstrations produced by a different kind of attacker, or on a held-out scenario type not represented in the demonstration set, and rerun the low-budget comparison; if the collision-rate advantage over the vanilla baseline shrinks or disappears, the expert guidance is mostly re-weighting the baseline's own trajectories. The paper itself contains one discordant observation already: for the TD3 victim in the on-ramp merging scenario, the method's collision rate (0.560) falls below the vanilla baseline (0.655).

Watch

Extended reading notes

Core claim

The central claim is that expert guidance converts a sparse-budget adversarial attacker from an unstable trial-and-error learner into a stable, higher-performing one. Concretely, successful attack trajectories from the paper's earlier low-frequency attack algorithm are filtered, balanced, and used to train an ensemble of mixture-of-experts networks by behavior cloning, yielding a stochastic expert policy with state-dependent mean and variance. During PPO training, the adversarial policy's objective includes a KL-divergence penalty pulling it toward the expert; the penalty weight is annealed according to the gap between the adversary's current episode return and an optimal return. The paper reports that this recipe achieves collision rates and attack efficiency about ten percent above the vanilla low-frequency baseline in the lowest-budget case, and that the gains persist across two driving scenarios and three victim algorithms, with lower variance across random seeds.

Load-bearing premise

The load-bearing assumption is that demonstrations harvested from the paper's own earlier attack algorithm transfer as genuine expertise to new scenarios; if those demonstrations mostly resemble the evaluation conditions, the reported gains could be the baseline supplying its own successes rather than a true expert advantage.

Editorial extensions

If this is right

  • For a fixed attack budget, the same number of perturbations produces more collisions: in the low-budget setting the guided adversary raises collision rate and attack efficiency by roughly ten percent over the vanilla low-frequency adversary.
  • Training becomes more dependable: the method converges faster and shows lower run-to-run variance, which matters if the attack is used to generate stress tests for driving policies.
  • The guidance transfers across victim policies: gains are reported against PPO, SAC, and TD3 agents and in two scenarios, unprotected left-turn and on-ramp merging, suggesting the expert encodes scenario-general attack timing.
  • The annealing mechanism protects against a bad expert: even when the expert is deliberately run in a suboptimal regime, the adversary outperforms baselines that keep the expert constraint fixed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next test the paper does not run is cross-algorithm demonstration transfer: train the expert on trajectories from a different attacker or from human-designed heuristics and check whether the collision-rate gains survive; if they do not, the advantage may be bootstrapping the same attack method's own successes.
  • The annealing schedule needs the adversary's optimal return R*, which is a simulation-only quantity; applying this attack outside a simulator would require a data-driven proxy for R*, an engineering step the paper leaves open.
  • The ensemble-variance gating idea is not specific to attacks: it could serve as a general mechanism for weighting any imitation prior inside PPO when demonstrations are known to be imperfect.
  • Because the two evaluated scenarios come from the same environment family used to collect demonstrations, a held-out scenario type such as a roundabout or a highway merge with different traffic patterns would be the sharper test of whether the mixture-of-experts actually generalizes or memorizes the demonstration conditions.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes an expert-guided adversarial attack method for DRL-based autonomous driving policies. The method first collects successful attack trajectories from the authors' previous low-frequency attack algorithm [13], trains an ensemble of Mixture-of-Experts policies via behavior cloning as an expert, and then uses that expert to regularize a PPO-based adversary through a KL-divergence term, with a performance-aware annealing coefficient that reduces reliance on the expert over time. Experiments in SUMO left-turn and on-ramp scenarios against PPO, SAC, and TD3 victim agents report collision rate, attack efficiency, and training-stability comparisons against vanilla PPO, value-penalty RL, and policy-constrained RL baselines.

Significance. If the stated claims were fully supported, the paper would offer a practical way to train low-frequency adversarial attacks with better sample efficiency and training stability, which is valuable for robustness evaluation of DRL driving policies. The framework is clearly organized, the ensemble MoE and annealing ideas are sensible, and the empirical study covers multiple victim algorithms and budget settings. The main weakness is that the central empirical claim is not consistently supported by the paper's own results, and the theoretical support contains a gap.

major comments (5)
  1. [Table II, Section V.F] The abstract and Section V.F claim that the proposed method 'consistently outperforms' baselines in collision rate and attack efficiency. This is contradicted by the TD3/Env-2 row of Table II, where the proposed method's collision rate is 0.560 ± 0.059 versus 0.655 ± 0.033 for the Vanilla baseline, with non-overlapping error bars. In the same table, the method ties Vanilla in the PPO/Env-2 cell (0.825 ± 0.031 vs 0.825 ± 0.006) and loses to VPRL in TD3/Env-2. Since this table is precisely the paper's imperfect-expert setting (ε=0.05, Γ=4), the central claim is not supported. The authors should either weaken the claim to 'outperforms in most cases' or provide statistical significance tests and a reasoned explanation for the regression.
  2. [Section IV.D, Proposition 1, Eqs. (17)-(21)] The proof of Proposition 1 is invalid as written. It assumes that a zero gradient of the regularized objective at π* implies a global optimum, and that a zero gradient of the KL divergence implies distributional equality. Neither implication holds for the parameterized policy class used in practice, where the gradient is taken with respect to parameters, not the full distribution, and stationary points need not be global optima. Moreover, the conclusion J(π_reg) < J(π*) requires π* to be the unique maximizer of J, which is not established anywhere. The proposition should be removed or re-proved under explicit, stated assumptions.
  3. [Sections IV.B and V.C] The expert demonstrations are generated by the authors' own prior attack algorithm [13], and the primary 'Vanilla RL' baseline is the same family of PPO adversary without expert guidance. This makes part of the evaluation self-referential: the expert is trained on trajectories produced by the baseline family, so the reported improvements may reflect bootstrapping from the baseline's own successes rather than a general advantage of expert-guided training. The paper's claim that the expert transfers 'generalizable knowledge' is therefore not independently evidenced. A concrete remedy is to include [13] itself as a baseline, or to train the expert on demonstrations from a different attack method, and to evaluate on scenarios not used to collect the demonstration data.
  4. [Section V.D, Eq. (22) and Algorithm 1] The performance-aware annealing mechanism is a central component of the method, but Eq. (22) and the surrounding text are not reproducible. The symbols for the annealing coefficient are inconsistent (β, β_e, τβ are used interchangeably), and the constants β_0, η, K, and R* are not defined unambiguously; the equation as typeset is garbled (e.g., the summation and exponent are unclear). Algorithm 1 also contains duplicate line numbers (two line 5) and an undefined 'PerturbationGenerate' reference to Eq. (2a). The authors should rewrite Eq. (22) and the algorithm pseudocode with clear notation.
  5. [Sections V.D-V.F] No statistical significance tests are reported anywhere in the experimental section. With only five random seeds and several cells showing overlapping error bars (e.g., PPO/Env-1 CR, SAC/Env-1 AE, TD3/Env-1 CR), the claims of 'significantly higher' collision rates and 'over 10% improvement' are not statistically substantiated. The authors should report confidence intervals, paired tests, or bootstrap comparisons across all relevant cells.
minor comments (6)
  1. [Abstract] The sentence 'our method achieves outperforms existing approaches' contains a grammatical error and should be corrected.
  2. [Section IV.A] The text says the three stages are 'detailed in Sections III.B-III.D', but the stages are actually described in Sections IV.B-IV.D; the cross-reference is wrong.
  3. [Section IV.D, Lemma 1] The paper refers to 'Lemma 2' from reference [51] but labels the displayed result as Lemma 1; the reference and numbering should be reconciled.
  4. [Table captions] The captions of Table I and Table II are incomplete; for example, Table I's caption reads 'CR, AND ANA OF THE EXPERT UNDER DIFFERENT CASES', missing the perturbation and attack budget notation and the metric definitions.
  5. [Section V.F] The sentence 'We first evaluate the performance of different attack methods under the setting of an imperfect expert. We first evaluate the expert's performance...' repeats 'We first evaluate'; one of the two occurrences should be reworded.
  6. [Algorithm 1] There is a typo in line 4 ('Ovserve' instead of 'Observe'), and the duplicate line numbering for line 5 should be fixed.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: expert demonstrations come from the authors' own prior attack algorithm, but the comparison is empirical and externally benchmarked; the Table II regression is a correctness issue, not circularity.

full rationale

The paper's claimed contribution is an empirical method, not a derivation. The expert policy is obtained by behavior cloning from successful attack trajectories generated by the authors' own prior low-frequency adversary [13] (Section IV.B, 'we employ multiple attack models trained using the attack algorithm from [13]'), which introduces self-reference into the data source. However, this is not a circular reduction because the method's output (collision rate, attack efficiency, training stability) is measured against independently implemented baselines (Vanilla, VPRL, PCRL) in a SUMO environment, and the KL-regularized PPO objective (Eq. 15) is not definitionally equal to the expert's imitation-learning loss (Eq. 4). The performance-aware annealing schedule (Eq. 22) is a hand-specified rule with unspecified scaling and annealing factors, not a fitted parameter renamed as a prediction. The abstract's 'consistently outperforms' claim is contradicted by the TD3 Env-2 cell in Table II (Ours CR 0.560 ± 0.059 vs Vanilla 0.655 ± 0.033), but that is an empirical falsification/correctness risk, not circularity. No load-bearing step reduces to its own inputs by construction; the self-citation [13] supplies training data for the expert, which is a normal use of prior work rather than a circular derivation.

Assumptions & free parameters 7 free parameters · 3 assumptions · 0 invented entities

The method introduces no new physical or mathematical entities; the MoE expert and adaptive coefficient are internal components. The main external dependencies are the SUMO simulator, Stable Baselines3, and the authors' prior attack algorithm [13], with hyperparameters treated as free inputs.

free parameters (7)
  • beta (KL regularization coefficient) = not specified for the proposed method; VPRL uses 0.1
    Controls the strength of expert guidance in Eq. (15); no value or initial schedule is given for Ours beyond the adaptive formula.
  • eta (annealing scaling factor) = not reported
    Scaling factor in Eq. (22) for adaptive beta_e; the paper does not state its value.
  • Annealing window K = not reported
    Number of episodes averaged in Eq. (22); unspecified.
  • Optimal return R* = not reported
    Reference return used in the annealing formula; unclear how it is set per environment.
  • AE metric coefficient omega = 0.05
    Custom weight in the attack efficiency metric Eq. (24); affects the reported AE values.
  • Number of experts N and ensemble size M = N=3, M=5
    Architecture choices for the MoE expert; set by hand and only claimed to be ablated without showing the ablation results.
  • Attack budget Gamma and perturbation budget epsilon = Gamma in {4,7}, epsilon in {0.05,0.1}
    Constraints tested; the imperfect-expert evaluation is done at epsilon=0.05, Gamma=4, a choice that may favor the annealing design.
assumptions (3)
  • domain assumption The MDP formulation of the attack in Eq. (2) with constraints (2a)-(2d) is a faithful model of adversarial timing and content selection.
    The optimization problem assumes the adversary controls a binary attack flag plus a desired victim action; this is a modeling choice, not derived from first principles.
  • domain assumption Lemma 1 (Policy Improvement Bound, Eq. 16) from reference [51] is assumed to justify early KL guidance.
    Quoted from an external reference without proof; if the bound does not apply to this attack setting, the guidance benefit is not guaranteed.
  • ad hoc to paper The proof of Proposition 1 assumes unconstrained policy optimization with zero gradient at the optimal policy and zero KL gradient implying distributional equality.
    The paper does not justify these assumptions for parameterized neural policies; this makes the proof of suboptimality of fixed-beta regularization non-rigorous.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sharpening the Spear: Adaptive Expert-Guided Adversarial Attack Against DRL-based Autonomous Driving Policies." pith.science (2026). https://pith.science/paper/JJHYKJ7H

@misc{pith2026250618304,
  author       = {Pith},
  title        = {Pith review of: Sharpening the Spear: Adaptive Expert-Guided Adversarial Attack Against DRL-based Autonomous Driving Policies},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JJHYKJ7H}},
  note         = {Machine review of arXiv:2506.18304}
}
read the original abstract

Deep reinforcement learning (DRL) has emerged as a promising paradigm for autonomous driving. However, despite their advanced capabilities, DRL-based policies remain highly vulnerable to adversarial attacks, posing serious safety risks in real-world deployments. Investigating such attacks is crucial for revealing policy vulnerabilities and guiding the development of more robust autonomous systems. While prior attack methods have made notable progress, they still face several challenges: 1) they often rely on high-frequency attacks, yet critical attack opportunities are typically context-dependent and temporally sparse, resulting in inefficient attack patterns; 2) restricting attack frequency can improve efficiency but often results in unstable training due to the adversary's limited exploration. To address these challenges, we propose an adaptive expert-guided adversarial attack method that enhances both the stability and efficiency of attack policy training. Our method first derives an expert policy from successful attack demonstrations using imitation learning, strengthened by an ensemble Mixture-of-Experts architecture for robust generalization across scenarios. This expert policy then guides a DRL-based adversary through a KL-divergence regularization term. Due to the diversity of scenarios, expert policies may be imperfect. To address this, we further introduce a performance-aware annealing strategy that gradually reduces reliance on the expert as the adversary improves. Extensive experiments demonstrate that our method achieves outperforms existing approaches in terms of collision rate, attack efficiency, and training stability, especially in cases where the expert policy is sub-optimal.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Robust Driving Control for Autonomous Vehicles: An Intelligent General-sum Constrained Adversarial Reinforcement Learning Approach

    cs.LG 2025-10 conditional novelty 4.0 of 10

    IGCARL trains an autonomous driving policy against a DRL-based adversary that plans multi-step collision-oriented observation attacks, with Lagrangian constraints that preserve clean-environment behavior, and reports ...

Reference graph

Works this paper leans on

58 extracted references · 22 canonical work pages · cited by 1 Pith paper

  1. [13]

    Seeing is not Believing: Robust Reinforcement Learning ag ainst Spurious Correlation,

    W. Ding, L. Shi, Y. Chi, and D. Zhao, “Seeing is not Believing: Robust Reinforcement Learning ag ainst Spurious Correlation,” in Proc. Adv. Neural Inf. Process. Syst., vol. 36, 2024

    work page 2024

  2. [1]

    To the best of our knowledge, we are the first to integrate expert knowledge into adversarial attack training for DRL-based autonomous driving policies

    Expert-Guided Adversarial Attack Framework : We propose a novel expert -guided adversarial attack method for autonomous driving, which enhances existing low - frequency a ttack frameworks by incorporating expert knowledge to improve both training stability and attack performance. To the best of our knowledge, we are the first to integrate expert knowledge...

    work page

  3. [2]

    This design enables the expert to better capture diverse attack policies across various scenarios

    Mixture-of-Experts-Based Expert Policy Derivation : To improve the quality and generalizability of expert guidance, we introduce a Mixture -of-Experts (MoE) architecture to derive the expert policy from historical successful attack demonstrations. This design enables the expert to better capture diverse attack policies across various scenarios

    work page

  4. [3]

    Performance-aware Annealing for Adaptive Expert Reliance: To address the limit ations of imperfect expert policies, we introduce a performance-aware annealing mechanism that dynamically adjusts the reliance on the expert based on the adversary’s performance during training. This mechanism enables the adversary to flexible balance exp ert guidance with aut...

    work page

  5. [4]

    A survey of decision-making and planning methods for self- driving vehicles,

    J. Hu et al., “A survey of decision-making and planning methods for self- driving vehicles,” Front. Neurorobot., vol. 19, p. 1451923, Feb. 2025, doi: 10.3389/fnbot.2025.1451923

    work page arXiv 2025

  6. [5]

    However, effective attack opportunities are temporally sparse and context-dependent

    Overly Frequent Adversarial Attacks: Existing high- frequency attack methods typically apply perturbations continuously over time . However, effective attack opportunities are temporally sparse and context-dependent. These methods not only waste computational resources but also l imit the ability to expose key vulnerabilities , ultimately hindering progre...

    work page

  7. [6]

    However, such trial -and- error exploration is highly inefficient, especially under strict attack budgets and sparse rewards

    Exploration Inefficiency of DRL-based method: Existing low-frequency methods typically rely on DRL to explore adversarial policies from scratch. However, such trial -and- error exploration is highly inefficient, especially under strict attack budgets and sparse rewards

    work page

  8. [7]

    This su boptimality can mislead adversary training and limit its overall effectiveness if left unaddressed

    Ignoring Expert Suboptimality : Existing expert- guided DRL methods often assume the expert is near -optimal and overlook that, in our setting, the expert is inherently imperfect due to the diversity of environments and attack conditions. This su boptimality can mislead adversary training and limit its overall effectiveness if left unaddressed. Based on t...

    work page

Show all 58 references
  1. [8]

    Fig. 2(a) illustrates an unprotected left-turn scenario (Env-1), where the DRL -based autonomous driving policy guides the ego vehicle (the red car) to navigate a left turn at an intersection without traffic lights . Fig. 2(b) depicts an on-ramp merging scenario (Env-2), in wh...

  2. [9]

    A Survey on Recent Advancements in Autonomous Driving Using Deep Reinforcement Learning: Applications, Challenges, and Solutions,

    R. Zhao, Y. Li, Y. Fan, F. Gao, M. Tsukada, and Z. Gao, “A Survey on Recent Advancements in Autonomous Driving Using Deep Reinforcement Learning: Applications, Challenges, and Solutions,” IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 12, pp. 19365–19398...

    2024

  3. [10]

    Reinforcement Learning -Based Multi-Lane Cooperative Control for On -Ramp Merging in Mixed - Autonomy Traffic,

    L. Liu, X. Li, Y. Li, J. Li, and Z. Liu, “Reinforcement Learning -Based Multi-Lane Cooperative Control for On -Ramp Merging in Mixed - Autonomy Traffic,” IEEE Internet Things J., pp. 1– 1, 2024, doi: 10.1109/JIOT.2024.3447039

    2024

  4. [11]

    Research on Autonomous Driving Deci sion-making Strategies based Deep Reinforcement Learning,

    Z. Wang, H. Yan, C. Wei, J. Wang, S. Bo, and M. Xiao, “Research on Autonomous Driving Deci sion-making Strategies based Deep Reinforcement Learning,” in Proceedings of the 2024 4th International Conference on Internet of Things and Machine Learning, Nanchang China: ACM, Aug. 2...

    2024

  5. [14]

    Adversarial Machine Learning Attacks and Defences in Multi- Agent Reinforcement Learning,

    M. Standen, J. Kim, and C. Szabo, “Adversarial Machine Learning Attacks and Defences in Multi- Agent Reinforcement Learning,” ACM Comput. Surv., vol. 57, no. 5, pp. 1– 35, May 2025, doi: 10.1145/3708320

    2025 doi

  6. [15]

    Adversarial Attacks and Countermeasures on Image Classification-based Deep Learning Models in Autonomous Driving Systems: A Systematic Review,

    B. Badjie, J. Cecílio, and A. Casimiro, “Adversarial Attacks and Countermeasures on Image Classification-based Deep Learning Models in Autonomous Driving Systems: A Systematic Review,” ACM Comput. Surv., vol. 57, no. 1, pp. 1–52, Jan. 2025, doi: 10.1145/3691625

    2025 doi

  7. [16]

    Targeted Attack on Deep RL-based Autonomous Driving with Learned Visual Patterns,

    P. Buddareddygari, T. Zhang, Y. Yang, and Y. Ren, “Targeted Attack on Deep RL-based Autonomous Driving with Learned Visual Patterns,” in Proc. IEEE Int. Conf. Robot. Automat. (ICRA), May 2022, pp. 10571– 10577. doi: 10.1109/ICRA46639.2022.9811574

    2022

  8. [17]

    Adversarial Deep Reinforcement Learning for Improving the Robustness of Multi -agent Autonomous Driving Policies,

    A. Sharif and D. Marijan, “Adversarial Deep Reinforcement Learning for Improving the Robustness of Multi -agent Autonomous Driving Policies,” in Proc. 29th Asia– Pacific Softw. Eng. Conf. (APSEC), Dec. 2022, pp. 61–70. doi: 10.1109/APSEC57359.2022.00018

    2022

  9. [18]

    Deep learning adversarial attacks and defenses in autonomous vehicles: a systematic literature review from a safety perspective,

    A. D. M. Ibrahum, M. Hussain, and J.- E. Hong, “Deep learning adversarial attacks and defenses in autonomous vehicles: a systematic literature review from a safety perspective,” Artif. Intell. Rev., vol. 58, no. 1, p. 28, Nov. 2024, doi: 10.1007/s10462-024-11014-8

    2024 doi

  10. [20]

    Toward Trustworthy Decision -Making for Autonomous Vehicles: A Robust Reinforcement Learning Approach with Safety Guarantees,

    X. He, W. Huang, and C. Lv, “Toward Trustworthy Decision -Making for Autonomous Vehicles: A Robust Reinforcement Learning Approach with Safety Guarantees,” Engineering, vol. 33, pp. 77–89, Feb. 2024, doi: 10.1016/j.eng.2023.10.005

    2024 doi

  11. [21]

    J. Fan, X. Lei, X. Chang, J. Miši, V. B. Miši, and Y. Yao, “Less is More: A Stealthy and Efficient Adversarial Attack Method for DRL -based > REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) < Autonomous Driving Policies,” IEEE Internet of Things Jo...

    2025

  12. [22]

    Reinforcement Learning from Imperfect Demonstrations under Soft Expert Guidance,

    M. Jing et al., “Reinforcement Learning from Imperfect Demonstrations under Soft Expert Guidance,” AAAI, vol. 34, no. 04, pp. 5109– 5116, Apr. 2020, doi: 10.1609/aaai.v34i04.5953

    2020 doi

  13. [23]

    Recent advances in reinforcement learning-based autonomous driving behavior planning: A survey,

    J. Wu, C. Huang, H. Huang, C. Lv, Y. Wang, and F. -Y. Wang, “Recent advances in reinforcement learning-based autonomous driving behavior planning: A survey,” Transp. Res. C: Emerg. Technol., vol. 164, p. 104654, Jul. 2024, doi: 10.1016/j.trc.2024.104654

    2024

  14. [24]

    A Preference- Based Multi- Agent Federated Reinforcement Learning Algorithm Framework for Trustworthy Interactive Urban Autonomous Driving,

    S. Lu, Y. Cai, Z. Liu, Y. Lian, L. Chen and H. Wang, “A Preference- Based Multi- Agent Federated Reinforcement Learning Algorithm Framework for Trustworthy Interactive Urban Autonomous Driving,” in IEEE Trans. Intell. Transp. Syst., doi: 10.1109/TITS.2025.3543810

  15. [26]

    Boosting Offline Reinforcement Learning for Autonomous Driving with Hierarchical Latent Skills,

    Z. Li, F. Nie, Q. Sun, F. Da, and H. Zhao, “Boosting Offline Reinforcement Learning for Autonomous Driving with Hierarchical Latent Skills,” in Proc. IEEE Int. Conf. Robot. Automat. (ICRA), May 2024, pp. 18362–18369. doi: 10.1109/ICRA57147.2024.10611197

    2024

  16. [27]

    Event-Triggered Model Predictive Control with Deep Reinforcement Learning for Autonomous Driving,

    F. Dang, D. Chen, J. Chen, and Z. Li, “Event-Triggered Model Predictive Control with Deep Reinforcement Learning for Autonomous Driving,” IEEE Trans. Intell. Veh., vol. 9, no. 1, pp. 459– 468, Jan. 2024, doi: 10.1109/TIV.2023.3329785

    2024

  17. [29]

    Self -Learned Autonomous Driving at Unsignalized Intersections: A Hierarchical Reinforced Learning Approach for Feasible Decision-Making,

    M. Al-Sharman, R. Dempster, M. A. Daoud, M. Nasr, D. Rayside, and W. Melek, “Self -Learned Autonomous Driving at Unsignalized Intersections: A Hierarchical Reinforced Learning Approach for Feasible Decision-Making,” IEEE Transactions on Intelligent Transportation Systems, vol....

    2023

  18. [30]

    Trustworthy Human- AI Collaboration: Reinforcement Learning with Human Feedback and Physics Knowledge for Safe Autonomous Driving,

    Z. Huang, Z. Sheng, and S. Chen, “Trustworthy Human- AI Collaboration: Reinforcement Learning with Human Feedback and Physics Knowledge for Safe Autonomous Driving,” Sep. 05, 2024, arXiv: arXiv:2409.00858. Accessed: Nov. 22, 2024. [Online]. Available: http://arxiv.org/abs/2409.00858

    2024 arXiv

  19. [31]

    Human-Guided Deep Rein forcement Learning for Optimal Decision Making of Autonomous Vehicles,

    J. Wu, H. Yang, L. Yang, Y. Huang, X. He, and C. Lv, “Human-Guided Deep Rein forcement Learning for Optimal Decision Making of Autonomous Vehicles,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 54, no. 11, pp. 6595– 6609, Nov. 2024, doi: 10.1109/TSMC.2024.3384992

    2024

  20. [32]

    Reinfor cement Learning from Human Feedback for Lane Changing of Autonomous Vehicles in Mixed Traffic,

    Y. Wang, L. Liu, M. Wang, and X. Xiong, “Reinfor cement Learning from Human Feedback for Lane Changing of Autonomous Vehicles in Mixed Traffic,” Aug. 08, 2024, arXiv: arXiv:2408.04447. Accessed: Nov. 22, 2024. [Online]. Available: http://arxiv.org/abs/2408.04447

    2024 arXiv

  21. [33]

    Fear -Neuro-Inspired Reinforcement Learning for Safe Autonomous Driving,

    X. He et al., “Fear -Neuro-Inspired Reinforcement Learning for Safe Autonomous Driving,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, no. 1, pp. 267–279, Jan. 2024, doi: 10.1109/TPAMI.2023.3322426

    2024

  22. [34]

    Efficient learning of safe driving policy via human -ai copilot optimization,

    Q. Li, Z. Peng, and B. Zhou, “Efficient learning of safe driving policy via human -ai copilot optimization,” in Proc. Int. Conf. Learn. Represent., 2022, pp. 1-19

    2022

  23. [35]

    Attention-Based Highway Safety Planner for Autonomous Driving via Deep Reinforcement Learning,

    G. Chen, Y. Zhang, and X. Li, “Attention-Based Highway Safety Planner for Autonomous Driving via Deep Reinforcement Learning,” IEEE Transactions on Vehicular Technology, vol. 73, no. 1, pp. 162–175, Jan. 2024, doi: 10.1109/TVT.2023.3304530

    2024

  24. [36]

    Robust lane change decision for autonomous vehicles in mixed traffic: A safety-aware multi- agent adversarial reinforcement learning approach,

    T. Wang, M. Ma, S. Liang, J. Yang, and Y. Wang, “Robust lane change decision for autonomous vehicles in mixed traffic: A safety-aware multi- agent adversarial reinforcement learning approach,” Transportation Research Part C: Emerging Technologies, vol. 172, p. 105005, Mar. 202...

    2025

  25. [38]

    An Analysis of Adversarial Attacks and Defenses on Autonomous Driving Models,

    Y. Deng, X. Zheng, T. Zhang, C. Chen, G. Lou, and M. Kim, “An Analysis of Adversarial Attacks and Defenses on Autonomous Driving Models,” in Proc. IEEE Int. Conf. Pervasive Comput. Commun. (PerCom), Mar. 2020, pp. 1 –10. doi: 10.1109/PerCom45495.2020.9127389

    2020

  26. [39]

    Robust Decision Making for Autonomous Vehicles at Highway On -Ramps: A Constrained Adversarial Reinforcement Learning Approach,

    X. He, B. Lou, H. Yang, and C. Lv, “Robust Decision Making for Autonomous Vehicles at Highway On -Ramps: A Constrained Adversarial Reinforcement Learning Approach,” IEEE Trans. Intell. Transp. Syst., vol. 24, no. 4, pp. 4103– 4113, Apr. 2023, doi: 10.1109/TITS.2022.3229518

    2023

  27. [41]

    Ro bust Lane Change Decision Making for Autonomous Vehicles: An Observation Adversarial Reinforcement Learning Approach,

    X. He, H. Yang, Z. Hu, and C. Lv, “Ro bust Lane Change Decision Making for Autonomous Vehicles: An Observation Adversarial Reinforcement Learning Approach,” IEEE Trans. Intell. Veh., vol. 8, no. 1, pp. 184–193, Jan. 2023, doi: 10.1109/TIV.2022.3165178

    2023

  28. [42]

    employed behavior cloning to derive an imitative expert policy from human expert demonstrations, which then regularizes the RL agent's behavior. D. Research Gaps In contrast to existing work, our research focuses on addressing the following gaps:

  29. [43]

    Adversarial Stress Test for Autonomous Vehicle Via Series Reinforcement Learning Tasks with Reward Shaping,

    X. Cai, X. Bai, Z. Cui, P. Hang, H. Yu, and Y. Ren, “Adversarial Stress Test for Autonomous Vehicle Via Series Reinforcement Learning Tasks with Reward Shaping,” IEEE Trans. Intell. Veh., pp. 1 –16, 2024, doi: 10.1109/TIV.2024.3418887

    2024

  30. [44]

    CRASH: Challenging Reinforcement-Learning Based Adversarial Scenarios for Safety Hardening,

    A. Kulkarni, S. Zhang, and M. Behl, “CRASH: Challenging Reinforcement-Learning Based Adversarial Scenarios for Safety Hardening,” Nov. 26, 2024, arXiv: arXiv:2411.16996. doi: 10.48550/arXiv.2411.16996

    2024 doi arXiv

  31. [45]

    ATS -O2A: A state-based adversarial attack strategy on deep reinforcement learning,

    X. Li, Y. Li, Z. Feng, Z. Wang, and Q. Pan, “ATS -O2A: A state-based adversarial attack strategy on deep reinforcement learning,” Comput. Secur., vol. 129, p. 103259, Jun. 2023, doi: 10.1016/j.cose.2023.103259

    2023

  32. [46]

    Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning,

    J. Sun et al., “Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning,” in Proc. AAAI Conf. Artif. Intell., vol. 34, no. 04, Art. no. 04, Apr. 2020, doi: 10.1609/aaai.v34i04.6047

    2020 doi

  33. [47]

    Attacking Deep Reinforcement Learning with Decoupled Adversarial Policy,

    K. Mo, W. Tang, J. Li, and X. Yuan, “Attacking Deep Reinforcement Learning with Decoupled Adversarial Policy,” IEEE Trans. Dependable Secure Comput., pp. 1–1, Jan. 2022, doi: 10.1109/TDSC.2022.3143566

    2022

  34. [48]

    Large Language Model guided Deep Reinforcement Learning for Decision Making in Autonomous Driving,

    H. Pang, Z. Wang, and G. Li, “Large Language Model guided Deep Reinforcement Learning for Decision Making in Autonomous Driving,” Dec. 24, 2024, arXiv: arXiv:2412.18511. doi: 10.48550/arXiv.2412.18511

    2024 doi arXiv

  35. [49]

    Safe Driving via Expert Guided Policy Optimization,

    Z. Peng, Q. Li, C. Liu, and B. Zhou, “Safe Driving via Expert Guided Policy Optimization,” in Proceedings of the 5th Conference on Robot Learning, PMLR, Jan. 2022, pp. 1554– 1563. Accessed: Jan. 10, 2025. [Online]. Available: https://proceedings.mlr.press/v164/peng22a.html

    2022

  36. [50]

    Efficient Deep Reinforcement Learning With Imitative Expert Priors for Autonomous Driving,

    Z. Huang, J. Wu, and C. Lv, “Efficient Deep Reinforcement Learning With Imitative Expert Priors for Autonomous Driving,” IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 10, pp. 7391–7403, Oct. 2023, doi: 10.1109/TNNLS.2022.3142822

    2023

  37. [51]

    Measuring Robustness to Natural Distribution Shifts in Image Classification,

    R. Taori, A. Dave, V. Shankar, N. Carlini, B. Recht, and L. Schmidt, “Measuring Robustness to Natural Distribution Shifts in Image Classification,” in Advances in Neural Information Processing Systems, Curran Associates, Inc., 2020, pp. 18583– 18599. Access ed: May 27,

    2020

  38. [52]

    Stable- baselines3: Reliable reinforcement learning implementations,

    A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann, “Stable- baselines3: Reliable reinforcement learning implementations,” J. Mach. Learn. Res., vol. 22, no. 268, pp. 1–8, 2021

    2021

  39. [53]

    Adaptive Mixtures of Local Experts,

    R. A. Jacobs, M. I. Jordan, S. J. Nowlan and G. E. Hinton, "Adaptive Mixtures of Local Experts," in Neural Computation, vol. 3, no. 1, pp. 79- 87, March 1991, doi: 10.1162/neco.1991.3.1.79

    1991 doi

  40. [54]

    Ma, J., Zhao, Z., Yi, X., Chen, J., Hong, L., & Chi, E. H. (2018, July). Modeling task relationships in multi- task learning with multi- gate mixture-of-experts. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining (pp. 1930-1939)

    2018

  41. [55]

    Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles,

    B. Lakshminarayanan, A. Pritzel, and C. Blundell, “Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles,” in Advances in Neural Information Processing Systems, Curran Associates, Inc.,

  42. [57]

    Proximal Policy Optimization Algorithms

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal Policy Optimization Algorithms.” arXiv, Aug. 28, 2017. doi: 10.48550/arXiv.1707.06347. > REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) <

    2017 doi arXiv

  43. [58]

    Energy- Constrained Safe Path Planning for UAV- Assisted Data Collection of Mobile IoT Devices,

    J. Fan, X. Chang, J. Mišić, V. B. Mišić, T. Yang, and Y. Gong, “Energy- Constrained Safe Path Planning for UAV- Assisted Data Collection of Mobile IoT Devices,” IEEE Internet Things J., pp. 1 –1, 2024, doi: 10.1109/JIOT.2024.3448537

    2024

  44. [59]

    Soft Actor-Critic: Off- Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft Actor-Critic: Off- Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor,” in Proc. Int. Conf. Mach. Learn., Jul. 2018, pp. 1861– 1870

    2018

  45. [60]

    Addressing Function Approximation Error in Actor-Critic Methods,

    S. Fujimoto, H. Hoof, and D. Meger, “Addressing Function Approximation Error in Actor-Critic Methods,” in Proc. 35th Int. Conf. Mach. Learn. (ICML), Jul. 2018, pp. 1587–1596

    2018

  46. [61]

    Hybrid Policy Optimization from Imperfect Demonstrations

    H. Yang, C. Yu, P. Sun, and S. Chen, “Hybrid Policy Optimization from Imperfect Demonstrations”

  47. [63]

    Explainable Deep Adversarial Reinforcement Learning Approach for Robust Autonomous Driving,

    C. Wang and N. Aouf, “Explainable Deep Adversarial Reinforcement Learning Approach for Robust Autonomous Driving,” IEEE Trans. Intell. Veh., pp. 1–13, 2024, doi: 10.1109/TIV.2024.3379367

    2024

  48. [64]

    Adversarial examples in the physical world,

    A. Kurakin, I. Goodfellow, and S. Bengio, “Adversarial examples in the physical world,” Feb. 11, 2017, arXiv: arXiv:1607.02533. doi: 10.48550/arXiv.1607.02533

    2017 doi arXiv

  49. [2017]

    [Online]

    Accessed: May 27, 2025. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2017/hash/9ef2ed4b7f d2c810847ffa5fa85bce38-Abstract.html

    2025

  50. [2025]

    Available: https://proceedings.neurips.cc/paper_files/paper/2020/hash/d8330f857a 17c53d217014ee776bfd50-Abstract.html

    [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2020/hash/d8330f857a 17c53d217014ee776bfd50-Abstract.html

    2020

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.