Pith. sign in

REVIEW 4 major objections 6 minor 39 references

Even 2.31% of training data can make behavioral cloning policies fully controllable by a tiny visual trigger while the policies still appear to perform normally.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 20:06 UTC pith:QMPFQXAT

load-bearing objection Clean-label backdooring clearly works on BC with ~5% poisoning, but the new entropy-timing attack is under-supported and the single-environment evaluation should curb the claims. the 4 major comments →

arxiv 2511.20992 v2 pith:QMPFQXAT submitted 2025-11-26 cs.LG cs.CRcs.RO

Dataset Poisoning Attacks on Behavioral Cloning Policies

Akansha Kalra , Soumil Datta , Ethan Gilmore , Duc La , Guanhong Tao , Daniel S. Brown This is my paper
classification cs.LG cs.CRcs.RO
keywords behavioral cloningdataset poisoningbackdoor attacksclean-label attacksentropy-based trigger timingimitation learningvisual triggersautonomous driving security
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that behavioral cloning policies, trained purely by supervised learning on expert demonstrations, can be silently compromised by extremely small amounts of poisoned data: injecting a tiny visual trigger into observations that accompany the gas action, without ever changing the expert's labels, makes a trained policy almost perfectly obey the trigger at deployment while its mean episode reward stays near the clean baseline until poisoning exceeds 70–80%. The authors further claim that an attacker who can choose when to show the trigger does far more damage by selecting states where the policy's action distribution has low entropy but the confident action is not the target action. If true, standard reward-based evaluation of cloned policies hides a dangerous vulnerability, and any organization deploying behavioral cloning from public or lightly guarded datasets should assume a small poisoned fraction can hand an adversary near-total control.

Core claim

The paper's central claim is that behavioral cloning policies can be fully compromised by a tiny, clean-label poisoning budget: with 5% of gas-action frames poisoned (2.31% of the whole dataset), the backdoor control rate rises from chance to approximately 100%, while mean episode reward remains statistically indistinguishable from the clean baseline until poisoning reaches 70–80%. The authors call this 'deceptively high' performance: the policy appears normal under standard evaluation yet is entirely controllable by the trigger at deployment. The second claim is that a black-box attacker with a limited test-time budget does far better by triggering the backdoor at low-entropy states where t

What carries the argument

The attack mechanism is clean-label backdoor poisoning: a fixed n×n patch (e.g., a 3×3 red square or a Gaussian noise patch) is added to the image observation of frames whose expert action equals the target action (gas), creating a spurious visual-action correlation without altering labels. The test-time amplification mechanism is an entropy-based trigger timing rule: query the BC policy's action probabilities, compute the Shannon entropy of that distribution, and inject the trigger only on low-entropy states where the most confident action differs from the target, concentrating an attack budget on states likely to push the policy out of distribution and cause compounding errors.

Load-bearing premise

The entropy-based test-time attack rests on the assumption that a policy's low action-distribution entropy reliably marks states where forcing the target action will compound errors, an assumption validated only in CarRacing with discrete actions and a single threshold; if entropy does not track criticality elsewhere, the claimed advantage over random timing may not hold.

What would settle it

Run the same pipeline (5% gas-action poisoning with a red patch, behavioral cloning trained on expert demonstrations) in a second environment with continuous actions or a different expert distribution, and compare entropy-triggered versus random triggering under a fixed budget; the central claim weakens if backdoor control does not approach near 100% at 2.31% overall poisoning, or if entropy timing does not beat random timing.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Policies trained on poisoned datasets pass standard evaluation: near-baseline mean reward while backdoor control is near 100% at just 2.31% overall poisoning.
  • Attack efficacy is more sensitive to the fraction of poisoned actions than to trigger size; control rate plateaus at small patch sizes around 5×5 pixels.
  • Subtle triggers (Gaussian noise, color-space shift) are learnable but need higher poisoning rates; a high-contrast red patch gives maximal control at minimal poisoning.
  • Entropy-triggered timing substantially outperforms random timing, with more than 100% additional performance degradation for a red-patch backdoor under a budget of 100 triggers.
  • The attacks operate with black-box access: the adversary only needs dataset edit access plus query access to policy outputs, with no reward function or simulator.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The 'deceptively high' performance suggests a general evaluation blind spot: aggregate return can mask adversarial controllability, and selective test-time triggering could be used as a cheap diagnostic to expose hidden backdoors in cloned policies.
  • If the entropy-criticality link generalizes, the same timing idea could turn any perturbation attack (not just backdoors) into a budget-efficient black-box attack on sequential policies, including continuous-action policies via differential entropy.
  • The results imply that simple dataset defenses based on anomaly detection of patch-like artifacts may be easy to bypass at 2–5% poisoning, since the poisoning is clean-label and visually subtle; defenses may need to look at action-conditioned correlations rather than label anomalies.
  • A digital twin or simulator, as the authors hypothesize, could make trigger timing nearly optimal; this could be tested by comparing entropy-based timing with an oracle that knows environment dynamics.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper studies clean-label backdoor dataset poisoning for behavioral cloning (BC) in the CarRacing environment with a discrete action space. The attacker poisons a fraction of gas-labeled demonstration frames with a small visual trigger (red square or Gaussian patch) without changing labels, trains a CNN BC policy on the poisoned set, and measures (i) mean episode reward during clean rollouts and (ii) backdoor control rate when the trigger is applied at test time. The main empirical finding is that just 5% poisoning of gas actions (≈2.31% of the full dataset) yields near-total trigger control while mean reward stays near baseline, with performance collapsing only at very high poisoning rates (70–80%). The paper also introduces an entropy-based test-time trigger-timing strategy that selects low-entropy states where the policy's confident action is not the target action, and reports that this attack degrades reward much more than random trigger timing for the red-patch backdoor. Additional experiments compare trigger types (red, Gaussian, color-space shift) and patch sizes. The authors conclude that BC policies are highly vulnerable to simple, low-budget clean-label backdoor attacks and that standard reward-based evaluation can mask hidden control vulnerabilities.

Significance. If the results hold, this is a useful and security-relevant contribution: it is, to my knowledge, the first systematic study of clean-label backdoor poisoning specifically for behavioral cloning, and the 5%-poisoning/≈100%-control result is a clean, reproducible empirical demonstration. The paper ships code and videos, uses multiple seeds in the poisoning-rate experiments, and provides patch-type and patch-size ablations, which strengthens the core poisoning finding. The main weakness is the entropy-based test-time attack, which is a central contribution but is supported by a single environment, a fixed threshold, and a mean/SE summary that is highly sensitive to outliers. If the entropy-attack result is not robust, the broader claim that BC policies are 'highly vulnerable during deployment' loses much of its force, although the poisoning/backdoor-control result would still stand. The paper is therefore significant but needs additional analysis to support its strongest deployment-vulnerability claim.

major comments (4)
  1. [§4.3, Table 2] The headline entropy-attack result for the red patch is a mean of -4731.69 with SE 1178.56 across 100 rollouts. The paper reports only mean and SE; no median, quantiles, per-rollout distribution, or significance test is given. With this magnitude of SE, the mean could be driven by a few catastrophic runs. I ask for the full distribution (e.g., boxplot or histogram), a robust statistic (median/interquartile range), and an explicit significance test (e.g., bootstrap or rank-sum) comparing Random vs. Entropy. The Gaussian-patch row shows a much smaller effect (577.43 vs. 858.38 unattacked), which further indicates that the advantage is trigger-dependent and cannot be asserted as a general property without more evidence.
  2. [§3.3, Table 2] The entropy threshold is fixed at 0.005 with no sensitivity sweep. The paper states the threshold is 'attacker-defined and can be tuned at test time,' but under the threat model in §3.1 the attacker has no access to a reward function or simulator. Without a specification of how the threshold is selected under this constraint, the reported entropy-attack performance may implicitly rely on privileged information (e.g., threshold chosen by evaluating reward on validation rollouts). At minimum, please provide a sweep over thresholds and show that the reported advantage over random timing is not confined to a narrow tuned value; ideally, state an unsupervised rule for choosing the threshold from policy outputs alone.
  3. [§4, Abstract, §5] All experiments use a single environment (CarRacing) and a single policy architecture (a classic CNN with three convolutional layers). The abstract and conclusion make unqualified statements such as 'BC policies are highly vulnerable' and 'behavioral cloning policies are highly vulnerable to minimal clean-label backdoor dataset poisoning attacks.' As the authors note in §5, future work should test other environments and IL algorithms; however, the current evidence is too narrow to support the unqualified generalization. I recommend either adding at least one additional environment/architecture or substantially qualifying the claims in the abstract and conclusion to make the scope explicit.
  4. [§4.3, §3.1] The threat model is black-box with a limited test-time attack budget (B=100). It is not clear how the entropy-based policy selects which states to trigger under this budget when the policy outputs a distribution; the paper should specify whether the attack triggers the first B states below the threshold or uses a different allocation rule, and whether the budget is a hard cap across an episode or across a fixed number of steps. This is needed to interpret the comparison with the random baseline and to assess real-world applicability.
minor comments (6)
  1. [§A.2, Table 3] The appendix text says 'Our results at 5% poisoning are shown in Table 1,' but the table in the appendix is labeled Table 3. Please correct the cross-reference.
  2. [§4.1, Fig. 1] The term 'chance' for backdoor control is used without definition. Since the action space has five actions, chance is roughly 20%, but Table 1 reports 30.80% for the unpoisoned 'None' condition. Please define what chance means and why the unpoisoned control rate is 30.8% rather than 20%.
  3. [§4.2, Table 1] The standard errors for backdoor control in Table 1 are formatted as 0.001 and 0.013 with only two decimal places shown, which obscures the actual precision. Report appropriate significant figures.
  4. [§5] The sentence 'We also hypothesize that if the adversary could use access to a digital twin of the environment to better optimize the timing of backdoor trigger injections' is grammatically incomplete; 'if' should be 'that' or the sentence should be completed.
  5. [§A.1, Fig. 4] The text in §A.1 says 'We repeat the sweep at a higher poisoning rate of 20% of gas-labeled frames' and Figure 4 shows this, but the main paper's §4.4 says 'We repeat the sweep at a higher poisoning rate of 20% of gas-labeled frames to rule out a low-rate artifact and observe the same results.' The appendix does not clearly report the effect of the 20% sweep on the conclusions; please make the comparison explicit.
  6. [General] There are several small typos and notation inconsistencies (e.g., 'D p' appears without a space; '100% degradation' in §4.3 is imprecise). A careful proofread would improve clarity.

Circularity Check

0 steps flagged

No significant circularity: empirical results are measured, not derived from attack definitions; same-author citations are background only.

full rationale

The paper's central claims are empirical: (1) BC policies poisoned with small fractions of trigger-labeled data retain near-baseline reward while exhibiting high backdoor control rates, and (2) entropy-timed trigger deployment degrades reward more than random timing. Neither claim is derived from the attack definitions by construction. The poisoning procedure intentionally creates a trigger–action correlation, but the paper does not claim to predict the control rate or reward from that construction; it trains policies and reports measured outcomes. The entropy-based timing attack is presented as a heuristic ('The intuition behind this trigger timing strategy is...') and is evaluated against random timing; the threshold 0.005 is an attacker-chosen hyperparameter, not a fitted quantity used to generate a prediction. The only same-author citations (refs [6], [28], and [35]) are background/methodology references and are not load-bearing; no uniqueness theorem or ansatz is imported from prior work by the same authors. Therefore no step in the paper's reasoning reduces to its own inputs. Concerns about single-environment evaluation, threshold sensitivity, and the high variance of the entropy-attack estimate are external-validity or statistical-reporting issues, not circularity.

Axiom & Free-Parameter Ledger

1 free parameters · 4 axioms · 0 invented entities

The reported results depend on a single environment and policy family; the paper itself flags generalization to other environments as future work. The entropy heuristic is an ad hoc assumption of the novel attack, and the entropy threshold is a tuned parameter.

free parameters (1)
  • entropy threshold = 0.005
    The entropy-based test-time attack selects states with entropy below this threshold. The threshold is attacker-defined and tuned, and no sensitivity analysis is provided in the main text.
axioms (4)
  • domain assumption CarRacing is a representative sequential decision-making environment for studying IL vulnerabilities.
    Used throughout; the paper extrapolates conclusions to real-world cyber-physical systems from this single simulator.
  • domain assumption The PPO-trained expert and fixed CNN BC architecture are representative of typical BC pipelines.
    Justifies the generality of the empirical findings; only one expert algorithm and one policy architecture are tested.
  • domain assumption Backdoor control rate (fraction of poisoned observations classified as the target action) is an appropriate measure of backdoor efficacy.
    Used in Section 4.1 as the primary metric for attacker control.
  • ad hoc to paper Action-distribution entropy is a valid proxy for state criticality.
    Introduced in Section 3.3 as intuition; no formal argument or external metric connects entropy to attack impact.

pith-pipeline@v1.3.0-alltime-deepseek · 10896 in / 12931 out tokens · 132248 ms · 2026-08-03T20:06:48.239517+00:00 · methodology

0 comments
read the original abstract

Behavior Cloning (BC) is a popular framework for training sequential decision policies from expert demonstrations via supervised learning. As these policies are increasingly being deployed in the real world, their robustness and potential vulnerabilities are an important concern. In this work, we perform the first analysis of the efficacy of clean-label backdoor attacks on BC policies. Our backdoor attacks poison a dataset of demonstrations by injecting a visual trigger to create a spurious correlation that can be exploited at test time. We evaluate how policy vulnerability scales with the fraction of poisoned data, the strength of the trigger, and the trigger type. We also introduce a novel entropy-based test-time trigger attack that substantially degrades policy performance by identifying critical states where test-time triggering of the backdoor is expected to be most effective at degrading performance. We empirically demonstrate that BC policies trained on even minimally poisoned datasets exhibit deceptively high, near-baseline task performance despite being highly vulnerable to backdoor trigger attacks during deployment. Our results underscore the urgent need for more research into the robustness of BC policies, particularly as large-scale datasets are increasingly used to train policies for real-world cyber-physical systems. Videos and code are available at https://sites.google.com/view/dataset-poisoning-in-bc.

Figures

Figures reproduced from arXiv: 2511.20992 by Akansha Kalra, Daniel S. Brown, Duc La, Ethan Gilmore, Guanhong Tao, Soumil Datta.

Figure 1
Figure 1. Figure 1: Sensitivity to the percentage of poisoned training data. The fig￾ure displays mean episode reward (left) and backdoor control rate (right) for varying poisoning percentages, based on a fixed 3×3 red patch on gas-labeled frames, averaged across 5 seeds with 10 rollouts per seed with error bars de￾noting standard deviation. We show that BC policies can achieve near-baseline performance despite being poisoned… view at source ↗
Figure 2
Figure 2. Figure 2: Visual comparison of clean and poisoned frames using different attack types. All attacks shown are a 3×3 patch in the top left corner of the visual observation [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Patch size sensitivity for red pixel and Gaussian noise patches. All models are trained with 5% of gas-labeled frames poisoned using an N×N red square patch. We see that we have similar reward performance for both patch types across all N. We also see that the backdoor effect rises at a very small N and stays fairly stable afterwards. The Gaussian patch performs worse in terms of control rate as a small po… view at source ↗
Figure 4
Figure 4. Figure 4: Patch-size sweep comparison at a higher poisoning rate (20% of gas-labeled frames) for red and Gaussian patches. On the left, we see the mean reward being comparable between both patch types, while on the right we see a high control rate via the patch across the board. The red patch outperforms the Gaussian patch at smaller patch sizes, but the Gaussian patch still performs well with over 90% attack accura… view at source ↗
Figure 5
Figure 5. Figure 5: Visual comparison of poisoned frames using different attack types. All [PITH_FULL_IMAGE:figures/full_fig_p016_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Poisoning-rate sweeps for Gaussian and color-space triggers. Both patch types show similar performance across different ’gas’ action poi￾son percentages. We see that the accuracy of the backdoor patch seems to be higher for the gaussian patch at lower percentages, while the mean reward and backdoor accuracy for the rest of the poison percentages for both patches follow the same trend. patch attains the hig… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 13 linked inside Pith

  1. [1]

    arXiv preprint arXiv:1606.06565 (2016)

    Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., Mané, D.: Con- crete problems in ai safety. arXiv preprint arXiv:1606.06565 (2016)

    Pith/arXiv arXiv 2016

  2. [2]

    In: 30th USENIX Security Symposium (USENIX Security 21)

    Bagdasaryan, E., Shmatikov, V.: Blind backdoors in deep learning models. In: 30th USENIX Security Symposium (USENIX Security 21). pp. 1505–1521. USENIX Association (Aug 2021), https://www.usenix.org/conference/usenixsecurity21/ presentation/bagdasaryan

    2021

  3. [3]

    Bojarski,M.,Testa,D.D.,Dworakowski,D.,Firner,B.,Flepp,B.,Goyal,P.,Jackel, L.D., Monfort, M., Muller, U., Zhang, J., Zhang, X., Zhao, J., Zieba, K.: End to end learning for self-driving cars (2016), https://arxiv.org/abs/1604.07316

    Pith/arXiv arXiv 2016

  4. [4]

    In: 2019 IEEE International Conference on Embedded Software and Systems (ICESS)

    Boloor, A., He, X., Gill, C., Vorobeychik, Y., Zhang, X.: Simple physical adver- sarial examples against end-to-end autonomous driving models. In: 2019 IEEE International Conference on Embedded Software and Systems (ICESS). pp. 1–7. IEEE (2019)

    2019

  5. [5]

    Kalra et al

    Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: Openai gym (2016), https://arxiv.org/abs/1606.01540 12 A. Kalra et al

    Pith/arXiv arXiv 2016

  6. [6]

    Brown, D.S., Goo, W., Niekum, S.: Better-than-demonstrator imitation learn- ing via automatically-ranked demonstrations (2019), https://arxiv.org/abs/1907. 03976

    2019

  7. [7]

    In: Conference on robot learning

    Chen, L., Paleja, R., Gombolay, M.: Learning from suboptimal demonstration via self-supervised reward regression. In: Conference on robot learning. pp. 1262–1277. PMLR (2021)

    2021

  8. [8]

    Chen, X., Liu, C., Li, B., Lu, K., Song, D.: Targeted backdoor attacks on deep learning systems using data poisoning (2017), https://arxiv.org/abs/1712.05526

    Pith/arXiv arXiv 2017

  9. [9]

    Advances in Neural Infor- mation Processing Systems36, 40786–40798 (2023)

    Chen, X., Guo, W., Tao, G., Zhang, X., Song, D.: Bird: generalizable backdoor detection and removal for deep reinforcement learning. Advances in Neural Infor- mation Processing Systems36, 40786–40798 (2023)

    2023

  10. [10]

    ArXivabs/2405.19424(2024)

    Chen, Y., Xue, H., Chen, Y.: Diffusion policy attacker: Crafting adversarial attacks for diffusion-based policies. ArXivabs/2405.19424(2024)

    Pith/arXiv arXiv 2024

  11. [11]

    Chi, C., Feng, S., Du, Y., Xu, Z., Cousineau, E., Burchfiel, B., Song, S.: Diffusion policy:Visuomotorpolicylearningviaactiondiffusion.In:ProceedingsofRobotics: Science and Systems (RSS) (2023)

    2023

  12. [12]

    In: 2018 International Conference on Innovation and In- telligence for Informatics, Computing, and Technologies (3ICT)

    Farag, W., Saleh, Z.: Behavior cloning for autonomous driving using convolu- tional neural networks. In: 2018 International Conference on Innovation and In- telligence for Informatics, Computing, and Technologies (3ICT). pp. 1–7 (2018). https://doi.org/10.1109/3ICT.2018.8855753

    arXiv 2018

  13. [13]

    Conference on Robot Learning (CoRL) (2021)

    Florence, P., Lynch, C., Zeng, A., Ramirez, O., Wahid, A., Downs, L., Wong, A., Lee, J., Mordatch, I., Tompson, J.: Implicit behavioral cloning. Conference on Robot Learning (CoRL) (2021)

    2021

  14. [14]

    In: 2024 IEEE Symposium on Security and Privacy (SP)

    Gong, C., Yang, Z., Bai, Y., He, J., Shi, J., Li, K., Sinha, A., Xu, B., Hou, X., Lo, D., et al.: Baffle: Hiding backdoors in offline reinforcement learning datasets. In: 2024 IEEE Symposium on Security and Privacy (SP). pp. 2086–2104. IEEE (2024)

    2024

  15. [15]

    Ieee Access7, 47230–47244 (2019)

    Gu, T., Liu, K., Dolan-Gavitt, B., Garg, S.: Badnets: Evaluating backdooring at- tacks on deep neural networks. Ieee Access7, 47230–47244 (2019)

    2019

  16. [16]

    arXiv preprint arXiv:2507.00485 (2025)

    Guo, W., Liu, G., Zhou, Z., Wang, L.: Pnact: Crafting backdoor attacks in safe reinforcement learning. arXiv preprint arXiv:2507.00485 (2025)

    Pith/arXiv arXiv 2025

  17. [17]

    Advances in neural information processing systems30(2017)

    Hadfield-Menell, D., Milli, S., Abbeel, P., Russell, S.J., Dragan, A.: Inverse reward design. Advances in neural information processing systems30(2017)

    2017

  18. [18]

    In: 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI)

    Hall, G., Das, A., Quarles, J., Rad, P.: Studying adversarial attacks on behavioral cloning dynamics. In: 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI). pp. 452–459. IEEE (2020)

    2020

  19. [19]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Jiang, W., Li, H., Xu, G., Zhang, T.: Color backdoor: A robust poisoning attack in color space. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 8133–8142 (2023)

    2023

  20. [20]

    arXiv preprint arXiv:2403.12945 (2024)

    Khazatsky, A., Pertsch, K., Nair, S., Balakrishna, A., Dasari, S., Karamcheti, S., Nasiriany, S., Srirama, M.K., Chen, L.Y., Ellis, K., et al.: Droid: A large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403.12945 (2024)

    Pith/arXiv arXiv 2024

  21. [21]

    In: 2020 57th ACM/IEEE Design Automation Conference (DAC)

    Kiourti, P., Wardega, K., Jha, S., Li, W.: Trojdrl: evaluation of backdoor attacks on deep reinforcement learning. In: 2020 57th ACM/IEEE Design Automation Conference (DAC). pp. 1–6. IEEE (2020)

    2020

  22. [22]

    DeepMind Blog (2020)

    Krakovna, V., Uesato, J., Mikulik, V., Rahtz, M., Everitt, T., Kumar, R., Kenton, Z., Leike, J., Legg, S.: Specification gaming: the flip side of ai ingenuity. DeepMind Blog (2020)

    2020

  23. [23]

    Computers & Security129, 103259 (2023)

    Li, X., Li, Y., Feng, Z., Wang, Z., Pan, Q.: Ats-o2a: A state-based adversarial attack strategy on deep reinforcement learning. Computers & Security129, 103259 (2023). https://doi.org/https://doi.org/10.1016/j.cose.2023.103259, https://www. sciencedirect.com/science/article/pii/S0167404823001694 Dataset Poisoning Attacks on Behavioral Cloning Policies 13

    arXiv 2023

  24. [24]

    Li, Y., Jiang, Y., Li, Z., Xia, S.T.: Backdoor learning: A survey (2022), https: //arxiv.org/abs/2007.08745

    Pith/arXiv arXiv 2022

  25. [25]

    In: Proceedings of the 26th International Joint Conference on Artificial Intelligence

    Lin, Y.C., Hong, Z.W., Liao, Y.H., Shih, M.L., Liu, M.Y., Sun, M.: Tactics of ad- versarial attack on deep reinforcement learning agents. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence. p. 3756–3762. IJCAI’17, AAAI Press (2017)

    2017

  26. [26]

    In: Icml

    Ng, A.Y., Harada, D., Russell, S.: Policy invariance under reward transformations: Theory and application to reward shaping. In: Icml. vol. 99, pp. 278–287 (1999)

    1999

  27. [27]

    In: 2024 IEEE International Conference on Robotics and Automation (ICRA)

    O’Neill, A., Rehman, A., Maddukuri, A., Gupta, A., Padalkar, A., Lee, A., Pooley, A., Gupta, A., Mandlekar, A., Jain, A., et al.: Open x-embodiment: Robotic learn- ing datasets and rt-x models: Open x-embodiment collaboration 0. In: 2024 IEEE International Conference on Robotics and Automation (ICRA). pp. 6892–6903. IEEE (2024)

    2024

  28. [28]

    Patil, B., Kalra, A., Tao, G., Brown, D.S.: How vulnerable is my policy? adversarial attacks on modern behavior cloning policies (2025), https://arxiv.org/abs/2502. 03698

    2025

  29. [29]

    In: ICML 2021 Workshop on Ad- versarial Machine Learning (2021)

    Qiaoben, Y., Zhou, X., Ying, C., Zhu, J.: Strategically-timed state-observation attacks on deep reinforcement learning agents. In: ICML 2021 Workshop on Ad- versarial Machine Learning (2021)

    2021

  30. [30]

    Ross, S., Gordon, G.J., Bagnell, J.A.: A reduction of imitation learning and struc- tured prediction to no-regret online learning (2011), https://arxiv.org/abs/1011. 0686

    2011

  31. [31]

    Russo, A., Proutiere, A.: Optimal attacks on reinforcement learning policies (2019), https://arxiv.org/abs/1907.13548

    Pith/arXiv arXiv 2019

  32. [32]

    Pro- ceedings of the AAAI Conference on Artificial Intelligence34(07), 11957–11965 (Apr 2020)

    Saha, A., Subramanya, A., Pirsiavash, H.: Hidden trigger backdoor attacks. Pro- ceedings of the AAAI Conference on Artificial Intelligence34(07), 11957–11965 (Apr 2020). https://doi.org/10.1609/aaai.v34i07.6871, https://ojs.aaai.org/index. php/AAAI/article/view/6871

    doi:10.1609/aaai.v34i07.6871 2020

  33. [33]

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms (2017), https://arxiv.org/abs/1707.06347

    Pith/arXiv arXiv 2017

  34. [34]

    Sutton, R.S., Barto, A.G., et al.: Reinforcement learning: An introduction, vol. 1. MIT press Cambridge (1998)

    1998

  35. [35]

    In: International Conference on Learning Representations (2023)

    Tien, J., He, J.Z.Y., Erickson, Z., Dragan, A.D., Brown, D.S.: Causal confusion and reward misidentification in preference-based reward learning. In: International Conference on Learning Representations (2023)

    2023

  36. [36]

    Torabi, F., Warnell, G., Stone, P.: Behavioral cloning from observation (2018), https://arxiv.org/abs/1805.01954

    Pith/arXiv arXiv 2018

  37. [37]

    Turner, A., Tsipras, D., Madry, A.: Clean-label backdoor attacks (2018)

    2018

  38. [38]

    Turner,A.,Tsipras,D.,Madry,A.:Label-consistentbackdoorattacks(2019),https: //arxiv.org/abs/1912.02771

    Pith/arXiv arXiv 2019

  39. [39]

    Kalra et al

    Zhao, T.Z., Kumar, V., Levine, S., Finn, C.: Learning fine-grained bimanual ma- nipulation with low-cost hardware (2023), https://arxiv.org/abs/2304.13705 14 A. Kalra et al. A Appendix A.1 Additional Patch Size Ablation Our results in Subsection 4.4 in the main paper show that small to moderate patch sizes (e.g., 5x5 to 25x25) occasionally yield higher me...

    Pith/arXiv arXiv 2023