Pith. sign in

REVIEW 3 major objections 6 minor 36 references

Spatiotemporally Constrained Action Space Attacks on Deep Reinforcement Learning Agents

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A look-ahead attack that allocates a fixed adversarial budget non-uniformly over time, guided by the agent's own dynamics, degrades a deep reinforcement learning agent's cumulative reward substantially more than a static per-step attack…

desk verdict A genuinely new combination of dynamics-aware planning with action-space attacks, but the central comparison rests on 10 episodes with no inferential statistics and the theory section is a placeholder. read the letter →

arxiv 1909.02583 v2 pith:PT64C3PY submitted 2019-09-05 cs.LG cs.AIcs.CRstat.ML

Xian Yeow Lee , Sambit Ghadai , Kai Liang Tan , Chinmay Hegde , Soumik Sarkar This is my paper · ORCID
classification cs.LGcs.AIcs.CRstat.ML
keywords actionspaceattacksdeepreinforcementlearningadversarialrobustnessprojectedgradientdescentmixed-normconstraintslook-aheadattackcyber-physicalsystemscontinuouscontrol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that an adversary who can spread a fixed attack budget over time, guided by the agent's dynamics, can cripple a deep reinforcement learning agent far more effectively than an adversary who spends a fixed amount each step. It frames both attacks as constrained optimization problems over the action space and solves them with projected gradient descent on the agent's learned value function or policy probabilities. The central empirical finding is that this look-ahead attack (LAS) consistently reduces cumulative reward more than the myopic attack (MAS) at the same total budget across several continuous-control environments. The authors also argue that the projection procedure on the learned proxy reward converges to the effect of optimizing the true reward, and that the attack patterns expose which actuators are most vulnerable.

What carries the argument

The central object is the temporally coupled attack budget, expressed as a mixed-norm constraint $\|\Delta\|_{p,q}\le B$ on the concatenated sequence of action perturbations over a horizon $H$, where $p$ and $q$ govern how the budget is split across action dimensions versus time steps. Projected gradient descent on the agent's network -- Q-values for value-based agents, action probabilities for policy-based agents, both treated as a noisy proxy for the true reward -- produces the perturbations, and the mixed-norm projection forces the attacker to allocate the total budget according to the agent's dynamics. A receding-horizon re-planning loop (plan $H$ steps ahead, apply only the first perturbation, then re-plan) corrects for divergence between the simulated and true trajectories.

What would settle it

Compute, over a batch of states, the cosine similarity between the gradient of the learned Q-function (or policy log-probability) with respect to the action and the gradient of the true environment reward with respect to the same action; if the average similarity is not clearly positive, the noisy-proxy assumption fails and the claimed convergence of projected gradient descent to a true-reward attack would not be expected to hold.

Watch

Extended reading notes

Core claim

Using the same total attack budget, allocating perturbations non-uniformly across a short receding horizon with the agent's dynamics (the Look-ahead Action Space attack, LAS) degrades a trained agent's cumulative reward substantially more than injecting a fixed per-step perturbation (the Myopic Action Space attack, MAS). The attack is generated by solving a constrained optimization problem that minimizes anticipated reward subject to a mixed-norm budget on the perturbation sequence, following projected gradient descent on the learned Q-function or policy distribution treated as a surrogate reward. Empirically, LAS dominates MAS across PPO and Double DQN agents in Lunar Lander, Bipedal-Walker, Hopper, Half-Cheetah, and Walker environments, at multiple budgets and horizons; at low budgets MAS can appear ineffective while LAS still drives the agent toward failure. The paper further shows that LAS concentrates its budget on particular action dimensions, exposing the most vulnerable actuators, and argues via a noisy projected-gradient-descent theorem that optimizing the surrogate matches optimizing the true reward.

Load-bearing premise

The load-bearing premise is that the learned network's gradient is an unbiased, i.i.d.-noisy proxy for the true reward gradient, a condition the authors themselves say is difficult to verify, together with the assumption that the adversary's simulation of the environment stays close enough to reality that re-planning each step corrects drift.

Editorial extensions

If this is right

  • An adversary with a modest total budget can make a trained agent fail by spending the budget at moments where the dynamics make it count, even when the same budget spread uniformly per step looks harmless.
  • The attack's non-uniform use of action dimensions identifies which actuators are most safety-critical, pointing to where defense or redundancy should be concentrated.
  • The convergence argument implies that optimizing the learned network with projected gradient descent is a sound way to discover attacks, because the network behaves as an unbiased, noisy estimate of the true reward under the stated assumptions.
  • Robustness evaluations of RL agents that only consider static per-step action perturbations will tend to underestimate the real threat from a budgeted adversary.
  • Horizon length interacts with budget: short horizons concentrate the attack and are more damaging in some environments, while in others longer horizons prevent the agent from recovering between perturbed steps.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: The dynamics-aware budgeting principle likely transfers to state-space (observation) attacks, which the authors list as future work, and could be paired with an imperceptibility metric to design stealthier attacks.
  • Inference: A natural defense suggested by the mixed-norm budget picture is to train agents against worst-case temporal allocations of a fixed budget (a min-max formulation); the paper does not test this.
  • Inference: Because LAS relies on a model of the environment, the advantage over MAS should shrink as the attacker's model becomes inaccurate; a testable extension is to measure attack efficacy under deliberately mismatched dynamics.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes two white-box action-space attacks on deep RL agents: a Myopic Action Space (MAS) attack that perturbs each action under a per-step norm budget, and a Look-ahead Action Space (LAS) attack that allocates a total budget across both action dimensions and time steps using an approximate model of the agent's dynamics, with a receding-horizon re-planning scheme. The optimization problems are solved with projected gradient descent on a surrogate reward signal (Q-values or policy probabilities). Experiments on Lunar Lander, BipedalWalker, Hopper, Half-Cheetah, and Walker with PPO and DDQN agents compare cumulative rewards under MAS, LAS, and random attacks. The paper claims that, at equal total budget, LAS deteriorates agent performance significantly more than MAS, and that projected gradient descent on the surrogate converges to the same effect as on the true reward. The authors provide code and supplementary results.

Significance. If the central empirical trend is confirmed, this is a useful contribution: dynamics-aware temporal allocation of action-space perturbations is a materially stronger threat class than static action-space attacks, and the proposed vulnerability-decomposition analysis is a plausible practical tool. Strengths include the public code, the variety of environments and agent types, and a clean problem formulation with explicit threat-model assumptions. The paper does not fit free constants to the attack results, and the convergence theorem is cited from external literature rather than derived. However, the headline comparison currently rests on ten episodes per condition without inferential statistics, and the theoretical contribution is explicitly conditional on unverified smoothness and noise assumptions. Both issues are load-bearing for the claims as stated, so they need to be addressed before the paper's conclusions can be accepted.

major comments (3)
  1. [Experimental Results & Discussion, Fig. 2 and supplementary Figs. 4-9] The central claim that "the LAS attack deteriorates the agent's performance significantly more than the MAS attack" (abstract) is supported only by box plots of 10 episodes per condition, with no standard errors, confidence intervals, or significance tests. The body text itself uses hedged language such as "generally more effective" and "at least as effective," and with n=10 the observed ordering could be sampling noise, particularly in low-budget conditions such as Fig. 2a/d and in high-variance Mujoco conditions such as Fig. 8. Please report per-condition statistics with confidence intervals and an appropriate paired or nonparametric test across seeds, or soften the abstract's "significantly" claim accordingly.
  2. [Comparison of MAS and LAS Attacks, Eqs. (1)-(2) and Algorithm 1] The equal-resource comparison assumes that LAS with total budget B and horizon H uses the same resources as MAS with per-step budget b = B/H. However, the paper does not report the realized cumulative perturbation norms. LAS re-plans each step with B <- max(0, B - delta'_{t,k}) and H-1, so the actual applied perturbations depend on whether the norm balls are saturated by the gradient updates; MAS may likewise fail to saturate its per-step ball in some states. Without reporting the actual l_p norms of the applied perturbations over an episode, the claim of "same amount of resources" is not verified. Please report realized attack magnitudes in each condition.
  3. [Theoretical Analysis, Theorem 1] Contribution 4 claims that projected gradient descent on the surrogate reward converges to the same effect as projected gradient descent on the true reward. This is not established by the cited theorem. Theorem 1 requires f to be beta-smooth, L-Lipschitz, and to have a rho-Lipschitz Hessian, and it requires the noise terms xi_t to be i.i.d. with zero mean and bounded variance; the paper itself states "the iid-ness assumption on xi is hard to verify" and "it is difficult to ascertain whether the assumptions of the above theorem are satisfied in specific cases." In addition, the constraint set S = {x | c(x) = 0} is not shown to be convex or to correspond to the mixed-norm ball used in LAS, and the supplementary's analysis of double projections only derives projection formulas for special cases, not a convergence guarantee. The convergence claim should be reframed as a heuristic motivation or supported by explicit verification on the actual network architectures and attack procedures.
minor comments (6)
  1. [Algorithm 1, line 19] The update B <- max(0, B - delta'_{t,k}) is dimensionally inconsistent because B is a scalar while delta'_{t,k} is a vector; write B <- max(0, B - ||delta'_{t,k}||_p) or clarify the intended norm.
  2. [Eq. (2)] The matrix Delta = [delta_t, delta_{t+1}, ..., delta_{t+H}] contains H+1 elements while the text says the horizon is H; please standardize the indexing for consistency.
  3. [Figure 2 and supplementary figures] The legend labels "MAS1, MAS2, LAS11, LAS21, LAS12, LAS22" are not defined in the main text; please explain in the captions which spatial/temporal norm combinations they denote.
  4. [Proposed Algorithms, DDQN implementation] The statement that "for value-based methods, we randomly sample adversarial actions to compute numerical gradients" omits the number of samples and the estimator used; please add these details or point to the corresponding code lines for reproducibility.
  5. [References] Reference [Ge et al. 2015] contains a typo in the title ("saddle pointsonline stochastic gradient") and should be checked against the published version.
  6. [Supplementary, Action Space Dimension Decomposition] The claims that certain action dimensions are "more vulnerable" are based on visual inspection of bar plots for a small number of episodes; please label these as anecdotal or provide aggregated statistics over many episodes.

Circularity Check

0 steps flagged · score 0.0 of 10

No material circularity: the claimed LAS advantage is an empirical benchmark result with no fitted inputs, and the cited convergence theorem is external to this paper.

full rationale

No fitted parameter is reused as a prediction: the attack budgets B and the comparison value b = B/H are fixed by the experimental protocol rather than learned from the measured rewards, and the central LAS-vs-MAS comparison is an external benchmark against pre-trained agents in OpenAI Gym environments. The convergence argument depends on the external noisy projected-gradient-descent theorem of Ge et al. (2015), and the paper explicitly acknowledges that the i.i.d. noise assumption is hard to verify, which is a stated limitation rather than a circular step. The self-citations (Havens, Jiang, and Sarkar 2018; Lee et al. 2019; Tan et al. 2019; Joshi et al. 2019) are contextual related-work citations and do not carry the derivation or the empirical claim. Although LAS's temporally coupled constraint is formally a relaxation of MAS's per-step constraint when b = B/H, the paper does not present the empirical superiority as a theorem derived from this containment, and the actual PGD-based attack algorithms are approximate; hence this is a property of the problem formulation, not a circular prediction. The paper is self-contained against external benchmarks and contains no self-citation chain that forces its central result.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical or model entities are introduced. The paper's claims depend on hand-chosen attack hyperparameters and on domain assumptions about surrogate reward validity, unverified smoothness and noise conditions for a cited convergence theorem, and environment-model accuracy. The convergence theorem is external, so this is not circular, but the premises are not established.

free parameters (3)
  • Projected gradient descent step size η and number of gradient iterations = Not reported in main text
    Both attack algorithms require these hyperparameters; attack strength can depend on them, and no values or tuning procedure are given in the main text.
  • Attack budget B and horizon H = Varies across environments; B typically 2 to 9, H=5 or 10, MAS budget b=B/H
    These define the threat model and the MAS/LAS comparison. They are experiment parameters, not fitted scientific constants, but the conclusion is conditional on them.
  • Projection norms p and q = p,q in {1,2}
    The choice of spatial and temporal norm changes attack sparsity and effectiveness; the paper reports results for all combinations, so the comparison is conditional on these choices.
assumptions (4)
  • domain assumption The learned Q-function or policy probability distribution is a faithful noisy proxy for the true reward function.
    Invoked in 'Proposed Algorithms'; the paper says only a 'noisy proxy' is available and assumes high-probability actions induce high expected reward.
  • ad hoc to paper The objective and constraint functions satisfy β-smoothness, L-Lipschitz, ρ-Lipschitz Hessian, and the gradient noise is i.i.d. with zero mean, bounded variance and bounded magnitude.
    Required by Theorem 1 from Ge et al. 2015; the authors explicitly say these conditions are difficult to ascertain and do not verify them for their DRL settings.
  • domain assumption The adversary's simulation environment matches the nominal environment over the attack horizon, and re-planning at each step corrects trajectory divergence.
    Algorithm 1 relies on the adversarial environment initialized with the same seed; the paper identifies state matching as a known implementation issue.
  • domain assumption Rewards after the look-ahead horizon can be represented by the nominal unperturbed trajectory.
    Eq. (2) uses nominal rewards for j > t+H, so effects of perturbed actions beyond the horizon are neglected.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Spatiotemporally Constrained Action Space Attacks on Deep Reinforcement Learning Agents." pith.science (2026). https://pith.science/paper/PT64C3PY

@misc{pith2026190902583,
  author       = {Pith},
  title        = {Pith review of: Spatiotemporally Constrained Action Space Attacks on Deep Reinforcement Learning Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PT64C3PY}},
  note         = {Machine review of arXiv:1909.02583}
}
read the original abstract

Robustness of Deep Reinforcement Learning (DRL) algorithms towards adversarial attacks in real world applications such as those deployed in cyber-physical systems (CPS) are of increasing concern. Numerous studies have investigated the mechanisms of attacks on the RL agent's state space. Nonetheless, attacks on the RL agent's action space (AS) (corresponding to actuators in engineering systems) are equally perverse; such attacks are relatively less studied in the ML literature. In this work, we first frame the problem as an optimization problem of minimizing the cumulative reward of an RL agent with decoupled constraints as the budget of attack. We propose a white-box Myopic Action Space (MAS) attack algorithm that distributes the attacks across the action space dimensions. Next, we reformulate the optimization problem above with the same objective function, but with a temporally coupled constraint on the attack budget to take into account the approximated dynamics of the agent. This leads to the white-box Look-ahead Action Space (LAS) attack algorithm that distributes the attacks across the action and temporal dimensions. Our results shows that using the same amount of resources, the LAS attack deteriorates the agent's performance significantly more than the MAS attack. This reveals the possibility that with limited resource, an adversary can utilize the agent's dynamics to malevolently craft attacks that causes the agent to fail. Additionally, we leverage these attack strategies as a possible tool to gain insights on the potential vulnerabilities of DRL agents.

Figures

Figures reproduced from arXiv: 1909.02583 by the authors.

Figure 1
Figure 1. Visual comparison of MAS and LAS. In MAS, [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Box plots of PPO Lunar Lander showing average cumulative reward across 10 episodes for each attack methods. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Time vs Attack magnitude along action dimension for LAS attacks with [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: Boxplots showing cumulative rewards of PPO agent in Bipedal-Walker environment under different attack strategies [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: DDQN Lunar Lander box plots showing average cumulative reward across 10 episodes for each attack method. Plots [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: DDQN Bipedal Walker box plots showing average cumulative reward across 10 episodes for each attack method. [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Box plots depicting distribution of rewards obtained by PPO agent in Mujoco Hopper environment under different [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Box plots depicting distribution of rewards obtained by PPO agent in Mujoco Half-Cheetah environment under [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Box plots depicting distribution of rewards obtained by PPO agent in Mujoco Walker environment under different [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Comparison of kδtk used across time for a single episode in PPO Lunar Lander for different spatial projections with `1 and `2 temporal projection. Left plot illustrates `1 spatial projection and right plot shows `2 spatial projection. In both plots, the magnitude of a…
Figure 11
Figure 11. Figure 11: kδtk usage plot of DDQN agent in Lunar Lander across time for a single episode. Left subplot illustrates `1 spatial attacks while right subplot shows `2 spatial attacks. In each subplot, attacks with `1 time projection attacks exhibit periodic spiked patterns while `2…
Figure 12
Figure 12. Figure 12: kδtk usage plot for PPO agent in Bipedal-Walker across time for a single episode. Left subplot illustrates `1 spatial attacks while right subplot shows `2 spatial attacks. In this figure, both `1 and ell2 are seemingly well distributed, although the magnitude of kδtk …
Figure 13
Figure 13. Figure 13: kδtk usage plot for DDQN agent in Bipedal-Walker across time for a single episode. Left subplot illustrates `1 spatial attacks while the right subplot shows `2 spatial attacks. A trend similar to [PITH_FULL_IMAGE:figures/full_fig_p016_13.png]
Figure 14
Figure 14. Figure 14: Ablation study for PPO Lunar Lander showing effectiveness of attacks comparing LAS with MAS for [PITH_FULL_IMAGE:figures/full_fig_p016_14.png]
Figure 15
Figure 15. Figure 15: Ablation study for DDQN Lunar Lander showing effectiveness of attacks comparing LAS with MAS. Left subplot [PITH_FULL_IMAGE:figures/full_fig_p016_15.png]
Figure 16
Figure 16. Figure 16: Ablation study for PPO Bipedal-Walker showing effectiveness of attacks comparing LAS with MAS. Left subplot [PITH_FULL_IMAGE:figures/full_fig_p017_16.png]
Figure 17
Figure 17. Figure 17: Ablation study for DDQN Bipedal-Walker showing effectiveness of attacks comparing LAS with MAS. Left subplot [PITH_FULL_IMAGE:figures/full_fig_p017_17.png]
Figure 18
Figure 18. Figure 18: Magnitude of attack with respect to different episodes for Lunar Lander environment with DoubleDQN RL agent. [PITH_FULL_IMAGE:figures/full_fig_p017_18.png]
Figure 19
Figure 19. Figure 19: Magnitude of attack with respect to different episodes for Bipedal Walker environment with DoubleDQN RL agent. [PITH_FULL_IMAGE:figures/full_fig_p018_19.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 29 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...

  2. [2]

    S., and Djouadi , S

    Ayas , M. S., and Djouadi , S. M. 2016. Undetectable sensor and actuator attacks for observer based controlled cyber-physical systems. In 2016 IEEE Symposium Series on Computational Intelligence (SSCI) , 1--7

    work page 2016

  3. [3]

    Bai, X.; Niu, W.; Liu, J.; Gao, X.; Xiang, Y.; and Liu, J. 2018. Adversarial examples construction towards white-box q table variation in dqn pathfinding training. In 2018 IEEE Third International Conference on Data Science in Cyberspace (DSC) , 781--787. IEEE

    work page 2018

  4. [4]

    Behzadan, V., and Munir, A. 2017. Vulnerability of deep reinforcement learning to policy induction attacks. In Perner, P., ed., Machine Learning and Data Mining in Pattern Recognition , 262--275. Cham: Springer International Publishing

    work page 2017

  5. [5]

    Boyd, S., and Vandenberghe, L. 2004. Convex optimization . Cambridge university press

    work page 2004

  6. [6]

    Brockman, G.; Cheung, V.; Pettersson, L.; Schneider, J.; Schulman, J.; Tang, J.; and Zaremba, W. 2016. Openai gym. arXiv preprint arXiv:1606.01540

    arXiv 2016

  7. [7]

    Condat, L. 2016a. Fast projection onto the simplex and the _1 ball. Mathematical Programming 158(1-2):575--585

    work page

  8. [8]

    Condat, L. 2016b. Fast projection onto the simplex and the l1 ball. Mathematical Programming 158(1-2):575--585

    work page

Show all 36 references
  1. [9]

    Ge, R.; Huang, F.; Jin, C.; and Yuan, Y. 2015. Escaping from saddle points—online stochastic gradient for tensor decomposition. In Conference on Learning Theory , 797--842

    2015

  2. [10]

    Goodfellow, I.; Shlens, J.; and Szegedy, C. 2015. Explaining and harnessing adversarial examples. In International Conference on Learning Representations

    2015

  3. [11]

    Gu, S.; Lillicrap, T.; Sutskever, I.; and Levine, S. 2016. Continuous deep q-learning with model-based acceleration. In International Conference on Machine Learning , 2829--2838

    2016

  4. [12]

    Havens, A.; Jiang, Z.; and Sarkar, S. 2018. Online robust policy learning in the presence of unknown adversaries. In Advances in Neural Information Processing Systems , 9916--9926

    2018

  5. [13]

    Hu, Z.; Liang, Y.; Zhang, J.; Li, Z.; and Liu, Y. 2018. Inference aided reinforcement learning for incentive mechanism design in crowdsourcing. In Bengio, S.; Wallach, H.; Larochelle, H.; Grauman, K.; Cesa-Bianchi, N.; and Garnett, R., eds., Advances in Neural Information Proc...

    2018

  6. [14]

    Huang , X., and Dong , J. 2018. Reliable control policy of cyber-physical systems against a class of frequency-constrained sensor and actuator attacks. IEEE Transactions on Cybernetics 48(12):3432--3439

    2018

  7. [15]

    Huang, S.; Papernot, N.; Goodfellow, I.; Duan, Y.; and Abbeel, P. 2017. Adversarial attacks on neural network policies. arXiv preprint arXiv:1702.02284

    2017 arXiv

  8. [16]

    Joshi, A.; Mukherjee, A.; Sarkar, S.; and Hegde, C. 2019. Semantic adversarial attacks: Parametric transformations that fool deep classifiers. In The IEEE International Conference on Computer Vision (ICCV)

    2019

  9. [17]

    Kim , J.; Park , G.; Shim , H.; and Eun , Y. 2016. Zero-stealthy attack for sampled-data control systems: The case of faster actuation than sensing. In 2016 IEEE 55th Conference on Decision and Control (CDC) , 5956--5961

    2016

  10. [18]

    Kurakin, A.; Goodfellow, I.; and Bengio, S. 2016. Adversarial machine learning at scale. arXiv preprint arXiv:1611.01236

    2016 arXiv

  11. [19]

    Y.; Balu, A.; Stoecklein, D.; Ganapathysubramanian, B.; and Sarkar, S

    Lee, X. Y.; Balu, A.; Stoecklein, D.; Ganapathysubramanian, B.; and Sarkar, S. 2019. A case study of deep reinforcement learning for engineering design: Application to microfluidic devices for flow sculpting. Journal of Mechanical Design 141(11)

    2019

  12. [20]

    Lin, Y.-C.; Hong, Z.-W.; Liao, Y.-H.; Shih, M.-L.; Liu, M.-Y.; and Sun, M. 2017. Tactics of adversarial attack on deep reinforcement learning agents. In Proceedings of the 26th International Joint Conference on Artificial Intelligence , 3756--3762. AAAI Press

    2017

  13. [21]

    Q., and Michalska , H

    Mayne , D. Q., and Michalska , H. 1990. Receding horizon control of nonlinear systems. IEEE Transactions on Automatic Control 35(7):814--824

    1990

  14. [22]

    A.; Veness, J.; Bellemare, M

    Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A. A.; Veness, J.; Bellemare, M. G.; Graves, A.; Riedmiller, M.; Fidjeland, A. K.; Ostrovski, G.; et al. 2015. Human-level control through deep reinforcement learning. Nature 518(7540):529

    2015

  15. [23]

    Pattanaik, A.; Tang, Z.; Liu, S.; Bommannan, G.; and Chowdhary, G. 2018. Robust deep reinforcement learning with adversarial attacks. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems , 2040--2042. International Foundation for Auto...

    2018

  16. [24]

    B.; Abbeel, P.; Levine, S.; and van de Panne, M

    Peng, X. B.; Abbeel, P.; Levine, S.; and van de Panne, M. 2018. Deepmimic: Example-guided deep reinforcement learning of physics-based character skills. ACM Transactions on Graphics (TOG) 37(4):143

    2018

  17. [25]

    J., and Badgwell, T

    Qin, S. J., and Badgwell, T. A. 2003. A survey of industrial model predictive control technology. Control engineering practice 11(7):733--764

    2003

  18. [26]

    Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; and Klimov, O. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

    2017 arXiv

  19. [27]

    Sinha, A.; Namkoong, H.; and Duchi, J. 2018. Certifiable distributional robustness with principled adversarial training. In International Conference on Learning Representations

    2018

  20. [28]

    Sra, S. 2012. Fast projections onto mixed-norm balls with applications. Data Mining and Knowledge Discovery 25(2):358--377

    2012

  21. [29]

    S.; McAllester, D

    Sutton, R. S.; McAllester, D. A.; Singh, S. P.; and Mansour, Y. 2000. Policy gradient methods for reinforcement learning with function approximation. In Advances in neural information processing systems , 1057--1063

    2000

  22. [30]

    L.; Poddar, S.; Sharma, A.; and Sarkar, S

    Tan, K. L.; Poddar, S.; Sharma, A.; and Sarkar, S. 2019. Deep reinforcement learning for adaptive traffic signal control. arXiv preprint arXiv:1911.06294

    2019 arXiv

  23. [31]

    Tessler, C.; Efroni, Y.; and Mannor, S. 2019. Action robust reinforcement learning and applications in continuous control. In International Conference on Machine Learning , 6215--6224

    2019

  24. [32]

    Tram \`e r, F.; Kurakin, A.; Papernot, N.; Goodfellow, I.; Boneh, D.; and McDaniel, P. 2017. Ensemble adversarial training: Attacks and defenses. arXiv preprint arXiv:1705.07204

    2017 arXiv

  25. [33]

    J.; and Fritz, M

    Tretschk, E.; Oh, S. J.; and Fritz, M. 2018. Sequential attacks on agents for long-term adversarial goals. In 2. ACM Computer Science in Cars Symposium

    2018

  26. [34]

    Van Hasselt, H.; Guez, A.; and Silver, D. 2016. Deep reinforcement learning with double q-learning. In Thirtieth AAAI Conference on Artificial Intelligence

    2016

  27. [35]

    Watkins, C. J. C. H., and Dayan, P. 1992. Q-learning. Machine Learning 8(3):279--292

    1992

  28. [36]

    Xiang, Y.; Niu, W.; Liu, J.; Chen, T.; and Han, Z. 2018. A pca-based model to predict adversarial examples on q-learning of path finding. In 2018 IEEE Third International Conference on Data Science in Cyberspace (DSC) , 773--780. IEEE

    2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.