Pith. sign in

REVIEW 4 major objections 6 minor 36 references

Robust In-Context Reinforcement Learning Under Reward Poisoning Attacks

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that adversarially training the in-context reinforcement-learning agent against a population of reward-poisoning attackers yields an agent that recovers optimal actions from poisoned contexts, outperforming…

desk verdict A legitimate but conditionally scoped empirical result: AT-DPT works when clean oracle actions are available, and the paper is honest about that. read the letter →

arxiv 2506.06891 v3 pith:5MUJFDWO submitted 2025-06-07 cs.LG cs.CR

Paulius Sasnauskas , Yiğit Yal{i}n , Goran Radanović This is my paper
classification cs.LGcs.CR
keywords in-contextreinforcementlearningrewardpoisoningadversarialtrainingDecision-PretrainedTransformercorruptionrobustnessmulti-armedbanditsMarkovdecisionprocessesmeta-RL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether an in-context reinforcement-learning agent can keep choosing near-optimal actions when an adversary corrupts a fraction of the rewards it observes at deployment time, and proposes AT-DPT, an adversarially trained Decision-Pretrained Transformer, as a yes. The method trains a population of reward-poisoning attackers alongside the agent, so the agent learns to infer optimal actions from contexts that contain poisoned rewards. The paper's experimental claim is that AT-DPT recovers from these attacks better than corruption-robust bandit baselines, including robust Thompson sampling and crUCB, and that the robustness survives adaptive attackers and carries over to MDPs. If correct, this shows that in-context RL can implement a corruption-robust learning algorithm without the hand-designed estimators that classical robust bandit algorithms rely on.

What carries the argument

The load-bearing object is the two-player training loop of Algorithm 1: for each task $M_i$, an attacker $\pi^\dagger_{\phi_i}$ is updated by REINFORCE to minimize the victim's true return subject to a mean-shift budget $B$ and a variance budget $B_\sigma$ (Equation 1), while the victim $\pi_\theta$ is updated by the supervised loss $\min_\theta \ell(\pi_\theta(\cdot|D^\dagger, s_q), a^\star)$ on the poisoned context $D^\dagger$ using oracle actions $a^\star$. The fixed point sought is a Nash equilibrium in which the victim maximizes expected true return against the trained attackers and each attacker minimizes the victim's true return against the trained victim. What carries the argument is the simultaneous optimization: the attacker forces the victim to read through corrupted rewards, and the victim learns to ignore the corruption while still using the in-context information that remains informative.

What would settle it

A concrete test: repeat the bandit training of AT-DPT but label the query states with actions from a suboptimal learned policy instead of the oracle, then measure cumulative regret against the same learned attackers; if the regret rises to the level of the frozen DPT, the claimed robustness is conditional on oracle labels rather than an intrinsic property of the method.

Watch

Extended reading notes

Core claim

AT-DPT simultaneously trains a population of reward-poisoning attackers, each minimizing the victim's true return under a soft budget constraint, and a transformer policy that is supervised to predict optimal actions from the corrupted contexts those attackers produce. The attack follows Huber's $\varepsilon$-contamination model: in an $\varepsilon$-fraction of timesteps the victim sees an attacker-chosen reward instead of the true reward, and the victim never sees clean rewards at test time. The experimental claim is that after adversarial training the policy implements a corruption-robust learning algorithm in-context, sustaining much lower regret under learned attackers than Thompson sampling, UCB, robust TS, crUCB, and the linear-bandit robust baselines, and retaining this robustness against an adaptive attacker and in the Darkroom2 and Miniworld MDPs.

Load-bearing premise

Everything rests on having oracle access to the optimal action for query states during adversarial training, because the victim is only as robust as the clean action labels it imitates from the poisoned contexts; if those labels are unavailable or wrong, the training signal that teaches robustness is gone.

Editorial extensions

If this is right

  • AT-DPT attains low cumulative regret under a learned attacker in 5-armed bandits with $\varepsilon=0.4$ and budget $B=3$, well below the regret of Thompson sampling, UCB, robust TS, and crUCB.
  • An AT-DPT trained against its own attacker generalizes to attackers trained for other algorithms, so it does not merely memorize one poisoning strategy.
  • AT-DPT trained against an adaptive attacker retains low regret against both adaptive and non-adaptive attackers, so the robustness is not limited to static poisoning.
  • The protocol transfers to linear bandits, where AT-DPT beats CRLinUCB under learned attacks, and to MDPs, where it at least matches the robust NPG baseline.
  • Adversarial training costs some performance in clean and uniformly random-attacked environments, so robustness to a learned adversary trades off against benign-condition performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Our read: the clean-environment performance drop visible in the reported tables implies a robustness-generalization trade-off that the paper does not quantify; a direct extension would adversarially train on a mixture of clean and poisoned contexts and test whether the trade-off shrinks.
  • The paper fixes the attacker's budget and contamination rate during training, so the strongest stress test would evaluate AT-DPT under budgets larger than those tested and under contamination rates beyond $\varepsilon=0.4$, where the appendix's budget study suggests baselines degrade more sharply.
  • Because AT-DPT is a meta-learner, its practical advantage over NPG and Q-learning may be adaptation speed rather than raw reward; comparing one-episode AT-DPT against NPG after varying numbers of episodes would make that explicit.
  • The comparisons use a learned attacker that observes the true reward and is trained to minimize the victim's return, so a different attacker class that poisons only a few critical timesteps, such as states near the goal in Darkroom2, could challenge the claimed out-of-distribution robustness.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper studies the robustness of in-context reinforcement learning (specifically the Decision-Pretrained Transformer, DPT) to test-time reward poisoning attacks. The authors propose AT-DPT, which alternates between training a set of reward-poisoning attackers (parameterized by phi, trained with REINFORCE on a soft-budget objective) and training the DPT victim via supervised learning to predict oracle optimal actions from contexts containing poisoned rewards. They evaluate AT-DPT in multi-armed bandit, linear bandit, adaptive-attacker, and MDP (Darkroom2 and Miniworld) settings, comparing it against standard and corruption-robust baselines (RTS, crUCB, CRLinUCB, NPG, Q-learning, PPO). The main reported findings are that AT-DPT achieves substantially lower cumulative regret than the baselines under trained attackers in the bandit/linear bandit settings, and is competitive or modestly better in the MDP settings.

Significance. If the empirical results are taken at face value, the paper makes a useful contribution: it is one of the first demonstrations that adversarial training can immunize an in-context RL agent against test-time reward corruption, and the evaluation is more thorough than is typical, with confidence intervals over 10 replications, cross-seed attacker evaluation, and out-of-distribution attackers. The paper is also honest about its main limitation (the need for oracle optimal actions at training time). However, the lack of code or data, the reliance on tuned/modified robust baselines, the soft-budget attack model, and the small MDP margins mean the central claims should be interpreted with care. If the oracle limitation cannot be relaxed, the practical impact is narrower than the abstract suggests.

major comments (4)
  1. [Section 4, Algorithm 1 line 10; Section 6] The supervised objective min_theta l(pi_theta(.|D^dagger, s_q), a*) requires an oracle-supplied a* for every query state. Section 6 acknowledges this as the main limitation, but it is load-bearing: all robustness results are obtained under this strong supervision assumption. The suggested alternative of using actions from a well-performing RL agent is not tested, and in an adversarial setting those actions could themselves be corrupted. Please either provide experiments with an approximate oracle (e.g., actions from a clean-trained RL policy) or explicitly scope the abstract and introduction to settings where optimal action labels are available.
  2. [Section 5.4, Table 4] The MDP results show AT-DPT only modestly outperforming NPG (e.g., AT-DPT 242.2 +/- 11.9 vs NPG 237.2 +/- 6.7 under the AT-DPT attacker), and several differences are within the confidence intervals. This weakens the claim that the bandit robustness 'generalizes to more complex environments such as ... MDPs.' Please report statistical significance tests or temper the generalization claim, and discuss the compute/prior-knowledge advantage of AT-DPT in this comparison.
  3. [Section 5.2, Appendix A] The main tables report only the best-performing variants of the robust baselines (RTS with C=0.5, crUCB modified per Algorithm 3, CRLinUCBv2), while the standard variants are relegated to the appendix. This makes the headline comparison hard to interpret. The modifications (especially the crUCB bonus-term change) should be described and justified in the main text, and the base variants should be shown in the main results so readers can see the full range of baseline performance.
  4. [Section 3.2, Equation (1); Section 5.2] The attacker is trained with a soft budget penalty, not a hard constraint. The realized mean deviation ||mu_phi - mu_R|| and variance ||sigma_phi|| of the trained attackers are not reported. Since the robust baselines (RTS, crUCB, CRLinUCB) have formal guarantees under hard corruption budgets, it is unclear whether the trained attackers violate those assumptions. Please report the realized attack statistics and the frequency of budget violations per epsilon, and discuss the implications for the baseline comparisons.
minor comments (6)
  1. [Title] The manuscript title is inconsistent between the arXiv metadata ('Robust In-Context Reinforcement Learning Under Reward Poisoning Attacks') and the PDF ('Can In-Context Reinforcement Learning Recover From Reward Poisoning Attacks?'); please unify.
  2. [Section 3.2] The notation \bar D is used both for the attacker's context and for the collection of clean trajectories; please define these clearly and avoid overloading.
  3. [Section 3.3] The Nash equilibrium formulation is stated without any existence or convergence results; please clarify that this is a design heuristic rather than a formal game-theoretic guarantee.
  4. [Section 5.2] The text says 'we cross-validate to prevent evaluation on the same attack,' but the procedure uses a different random seed for the attacker, not cross-validation; please rephrase to avoid confusion.
  5. [Section 5.4] The Miniworld results in the appendix include a PPO baseline, but the main text does not mention PPO when introducing the MDP baselines; please add a short description.
  6. [General] No code or data availability statement is provided. Given the custom attacker training, the modified baseline implementations, and the absence of a public repository, please include a statement on how the results can be reproduced.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: AT-DPT's robustness is an empirical result with an acknowledged oracle-supervision limitation, not a derivation that reduces to its inputs.

full rationale

The paper makes no formal derivation; its central claim is empirical. Algorithm 1, line 10 trains the victim by supervised learning on poisoned contexts D† with oracle actions a*, and Section 6 candidly states: "The main limitation of our method, also a limitation of DPT is the need of actions provided by the oracle for training." This is a real transferability constraint, but it is not circularity: the evaluation metric is cumulative regret on clean rewards under cross-seed and cross-algorithm attackers (Section 5.2 and Algorithm 2), which is not the training loss and is not forced by construction. The min-max formulation in Section 3.3 (attacker minimizes true reward subject to budget penalties; victim maximizes true reward) is a standard adversarial-training setup, and the attacker is trained with REINFORCE rather than being hand-set, so the comparison is not a fitted parameter renamed as a prediction. Baseline tuning of RTS and crUCB (Section A) is a baseline-selection concern, not circularity. Self-citations (Nika et al. 2023; Rakhsha et al. 2020; Mohammadi et al. 2023) appear only in related-work context and are not load-bearing. No equation in the paper reduces a claimed prediction to an input, a fit, or a prior self-cited result.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The method relies on standard RL assumptions plus several hand-chosen hyperparameters (lambda=10, B_sigma=1) and a strong oracle assumption (optimal actions at training time) stated as a limitation. Tuned baseline constants (RTS C=0.5, CRLinUCB bounds) are free parameters chosen to make baselines competitive. No invented entities.

free parameters (4)
  • Lambda penalty strength = 10
    Soft budget constraint weight in attacker's objective Eq. (1); chosen by hand, controls how much attacks may exceed budget.
  • Attacker covariance budget B_sigma = 1
    Variance penalty budget for attacker's output distribution; set by hand in bandit and linear bandit experiments.
  • RTS corruption parameter C (tuned) = 0.5
    Robust Thompson sampling hyperparameter tuned to make the baseline competitive; the paper reports the tuned variant in main tables, while the authors' recommended settings perform worse.
  • CRLinUCB budget bound (v2) = C' / H (approximately Theorem 1 values scaled)
    The paper tried three variants and reports the best one (v2) in the main text; the original recommended setting performed poorly.
assumptions (3)
  • domain assumption Oracle access to optimal actions a* for query states during adversarial training (Algorithm 1, line 10)
    AT-DPT training requires a clean oracle for optimal actions under poisoned contexts; stated explicitly as a limitation in Section 6.
  • domain assumption Huber epsilon-contamination model with attacker that observes true reward and can choose a different reward distribution per state-action
    The attack model in Section 3.2 assumes the attacker has access to the true reward and a bounded perturbation budget; all evaluations are within this model.
  • standard math Standard MDP and bandit assumptions: Gaussian rewards for bandits, linear features for linear bandits, finite grids with sparse rewards for MDPs
    Section 5 experimental setup specifies these environment assumptions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Robust In-Context Reinforcement Learning Under Reward Poisoning Attacks." pith.science (2026). https://pith.science/paper/5MUJFDWO

@misc{pith2026250606891,
  author       = {Pith},
  title        = {Pith review of: Robust In-Context Reinforcement Learning Under Reward Poisoning Attacks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5MUJFDWO}},
  note         = {Machine review of arXiv:2506.06891}
}
read the original abstract

We study the corruption-robustness of in-context reinforcement learning (ICRL), focusing on the Decision-Pretrained Transformer (DPT, Lee et al., 2023). To address the challenge of reward poisoning attacks targeting the DPT, we propose a novel adversarial training framework, called Adversarially Trained DPT (AT-DPT). Our method simultaneously trains a population of attackers to minimize the true reward of the DPT by poisoning environment rewards, and a DPT model to infer optimal actions from the poisoned data. We evaluate the effectiveness of our approach against standard bandit algorithms, including robust baselines designed to handle reward contamination. Our results show that AT-DPT significantly outperforms them in bandit settings under a learned attacker, and generalizes to more complex environments such as adaptive attackers and MDPs. It shows promise in ICRL as a meta-RL approach to learning effective corruption-robust algorithms.

Figures

Figures reproduced from arXiv: 2506.06891 by the authors.

Figure 1
Figure 1. The training procedure of AT-DPT. We use adversarial training to optimize the parameters [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Comparison of the cumulative regret per round (lower is better) of different methods throughout 20 rounds of adversarial training (si￾multaneously learning AT-DPT and an attacker) in the bandit setting. Within one round we perform H = 500 steps. The y axis indicates cumulative regret for that round. Mean and 95% confidence interval (2×SEM) over 10 experiment replications. Attack budget B = 3, ε = 0.4. DPT F indicate… view at source ↗
Figure 3
Figure 3. An illustration of the Darkroom2 environment with an attacker’s poisoning strategy during a [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Adversarial training curves for training the attacker in the bandit setting, for different values [PITH_FULL_IMAGE:figures/full_fig_p023_4.png]
Figure 5
Figure 5. Figure 5: Adversarial training curves for training the adaptive attacker in the bandit setting, for [PITH_FULL_IMAGE:figures/full_fig_p023_5.png]
Figure 6
Figure 6. Figure 6: Adversarial training curves for training the attacker in the Darkroom2 environment, for [PITH_FULL_IMAGE:figures/full_fig_p024_6.png]
Figure 7
Figure 7. Figure 7: Adversarial training curves for training the attacker in the Miniworld environment, for [PITH_FULL_IMAGE:figures/full_fig_p024_7.png]
Figure 8
Figure 8. Figure 8: A study of the effect of the budget B on the regret in the bandit setting. We run the experiments for B = 5 for more rounds to observe convergence. We observe that a larger budget for the attacker leads to a higher regret for TS and DPT F, although adversarial training…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 17 canonical work pages

  1. [3]

    Algorithm Attacker Target Unif. Rand. Attack Clean Env.AT-DPT DPT F LinUCB CRLinUCBv1 CRLinUCBv2 CRLinUCBv3 ε= 0.1 AT-DPT 2.55 ± 0.88 2.02 ± 0.92 2.28 ± 0.90 2.44 ± 0.96 1.55 ± 0.98 1.85 ± 0.94 4.60 ± 0.98 3.89 ± 0.86 DPT F 14.50 ± 2.30 14.42 ± 2.48 14.62 ± 2.34 14.06 ± 2.72 14.02 ± 2.74 13.19 ± 2.36 5.23 ± 1.02 3.35 ± 0.84 LinUCB 10.57 ± 2.00 7.92 ± 1.24...

    work page

  2. [4]

    Filtering Learning Histories Enhances In-Context Reinforcement Learning

    Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learning via sequence modeling.Advances in Neural Information Processing Systems, 34:15084–15097, 2021a. Weiqin Chen, Xinjie Zhang, Dharmashankar Subramanian, and Santiago Paternain. Filte...

    work page Pith review arXiv

  3. [5]

    We chose the α-trimmed variant, which performs best empirically

    A.2CRUCB Niss & Tewari (2020) provide a few variants of the crUCB algorithm. We chose the α-trimmed variant, which performs best empirically. We introduce a modification to the algorithm due to poor original variant empirical performance. The modified variant is shown in Algorithm 3, where f – α-trimmed mean function – if n is the number of rewards observ...

    work page 2020

  4. [9]

    Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned.arXiv preprint arXiv:2209.07858,

    11 Preprint Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, et al. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned.arXiv preprint arXiv:2209.07858,

    arXiv

  5. [11]

    Data poisoning for in-context learning.arXiv preprint arXiv:2402.02160,

    Pengfei He, Han Xu, Yue Xing, Hui Liu, Makoto Yamada, and Jiliang Tang. Data poisoning for in-context learning.arXiv preprint arXiv:2402.02160,

    arXiv

  6. [12]

    Decision Transformer under Random Frame Dropping

    Kaizhe Hu, Ray Chen Zheng, Yang Gao, and Huazhe Xu. Decision transformer under random frame dropping.arXiv preprint arXiv:2303.03391,

    work page Pith review arXiv

  7. [13]

    Adversarial attacks on neural network policies.arXiv preprint arXiv:1702.02284,

    Sandy Huang, Nicolas Papernot, Ian Goodfellow, Yan Duan, and Pieter Abbeel. Adversarial attacks on neural network policies.arXiv preprint arXiv:1702.02284,

    arXiv

  8. [17]

    Thodoris Lykouris, Vahab Mirrokni, and Renato Paes Leme

    URL https://proceedings.neurips.cc/paper_files/paper/ 2023/file/92d3d2a9801211ca3693ccb2faa1316f-Paper-Conference.pdf. Thodoris Lykouris, Vahab Mirrokni, and Renato Paes Leme. Stochastic bandits robust to adversarial corruptions. InProceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2018, pp. 114–122, New York, NY , USA,

    work page 2023

Show all 36 references
  1. [20]

    A survey of in-context reinforcement learning.arXiv preprint arXiv:2502.07978,

    Amir Moeini, Jiuqi Wang, Jacob Beck, Ethan Blaser, Shimon Whiteson, Rohan Chandra, and Shangtong Zhang. A survey of in-context reinforcement learning.arXiv preprint arXiv:2502.07978,

    arXiv

  2. [21]

    Implicit poisoning attacks in two-agent reinforcement learning: Adversarial policies for training- time attacks.arXiv preprint arXiv:2302.13851,

    Mohammad Mohammadi, Jonathan N¨other, Debmalya Mandal, Adish Singla, and Goran Radanovic. Implicit poisoning attacks in two-agent reinforcement learning: Adversarial policies for training- time attacks.arXiv preprint arXiv:2302.13851,

    arXiv

  3. [22]

    Learning to adapt in dynamic, real-world environments through meta-reinforcement learning.arXiv preprint arXiv:1803.11347,

    Anusha Nagabandi, Ignasi Clavera, Simin Liu, Ronald S Fearing, Pieter Abbeel, Sergey Levine, and Chelsea Finn. Learning to adapt in dynamic, real-world environments through meta-reinforcement learning.arXiv preprint arXiv:1803.11347,

    arXiv

  4. [23]

    Practical black-box attacks against machine learning

    Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. InProceedings of the 2017 ACM on Asia conference on computer and communications security, pp. 506–519,

    2017

  5. [24]

    Understanding the limits of poisoning attacks in episodic reinforcement learning.arXiv preprint arXiv:2208.13663,

    Anshuka Rangi, Haifeng Xu, Long Tran-Thanh, and Massimo Franceschetti. Understanding the limits of poisoning attacks in episodic reinforcement learning.arXiv preprint arXiv:2208.13663,

    arXiv

  6. [25]

    URLhttps://ojs.aaai.org/index.php/AAAI/article/view/6047

    doi: 10.1609/aaai.v34i04.6047. URLhttps://ojs.aaai.org/index.php/AAAI/article/view/6047. Yanchao Sun, Da Huo, and Furong Huang. Vulnerability-aware poisoning mechanism for online rl with unknown dynamics. InInternational Conference on Learning Representations,

    doi

  7. [28]

    Christopher JCH Watkins and Peter Dayan

    URL https://proceedings.neurips.cc/paper_files/paper/2017/ file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf. Christopher JCH Watkins and Peter Dayan. Q-learning.Machine learning, 8:279–292,

    2017

  8. [29]

    URL https://ojs.aaai

    doi: 10.1609/aaai.v38i14.29529. URL https://ojs.aaai. org/index.php/AAAI/article/view/29529. Jiawei Xu, Rui Yang, Feng Luo, Meng Fang, Baoxiang Wang, and Lei Han. Robust decision transformer: Tackling data corruption in offline rl via sequence modeling. InInternational Confere...

    doi

  9. [30]

    Robust thompson sampling algorithms against reward poisoning attacks.arXiv preprint arXiv:2410.19705,

    Yinglun Xu, Zhiwei Wang, and Gagandeep Singh. Robust thompson sampling algorithms against reward poisoning attacks.arXiv preprint arXiv:2410.19705,

    arXiv

  10. [31]

    Towards robust offline reinforcement learning under diverse data corruption.arXiv preprint arXiv:2310.12955,

    Rui Yang, Han Zhong, Jiawei Xu, Amy Zhang, Chongjie Zhang, Lei Han, and Tong Zhang. Towards robust offline reinforcement learning under diverse data corruption.arXiv preprint arXiv:2310.12955,

    arXiv

  11. [32]

    Universal vulnerabilities in large language models: In-context learning backdoor attacks.arXiv preprint arXiv:2401.05949,

    Shuai Zhao, Meihuizi Jia, Luu Anh Tuan, and Jinming Wen. Universal vulnerabilities in large language models: In-context learning backdoor attacks.arXiv preprint arXiv:2401.05949,

    arXiv

  12. [33]

    17 A.2 crUCB

    15 Preprint Supplementary Material: Can In-Context Reinforcement Learning Recover From Reward Poisoning Attacks? A Baseline algorithms 17 A.1 Robust TS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 A.2 crUCB . . . . . . . . . . . . . . . . . ...

    2024

  13. [35]

    A.3 CRLINUCB We source the CRLinUCB algorithm from Ding et al

    or (l.σ 0); 17 Preprint • the modified variant, indicated as crUCB (mod.) or (m.). A.3 CRLINUCB We source the CRLinUCB algorithm from Ding et al. (2022). The authors suggest setting the upper bound of the budget C ′ to equal εBH . We found that the algorithm did not perform we...

    2022

  14. [1933]

    URL http://www.jstor.org/stable/2332286

    ISSN 00063444. URL http://www.jstor.org/stable/2332286. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. V on Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vi...

  15. [1964]

    URL https://doi.org/10.1214/ aoms/1177703732

    doi: 10.1214/aoms/1177703732. URL https://doi.org/10.1214/ aoms/1177703732. L´eonard Hussenot, Matthieu Geist, and Olivier Pietquin. Copycat: Taking control of neural policies with constant attacks.arXiv preprint arXiv:1905.12282,

    1905

  16. [2001]

    Michael Laskin, Luyu Wang, Junhyuk Oh, Emilio Parisotto, Stephen Spencer, Richie Steigerwald, DJ Strouse, Steven Hansen, Angelos Filos, Ethan Brooks, et al

    MIT Press. Michael Laskin, Luyu Wang, Junhyuk Oh, Emilio Parisotto, Stephen Spencer, Richie Steigerwald, DJ Strouse, Steven Hansen, Angelos Filos, Ethan Brooks, et al. In-context reinforcement learning with algorithm distillation.arXiv preprint arXiv:2210.14215,

    arXiv

  17. [2002]

    doi: 10.1023/A: 1013689704352

    ISSN 0885-6125. doi: 10.1023/A: 1013689704352. URLhttps://doi.org/10.1023/A:1013689704352. Jacob Beck, Risto Vuorio, Evan Zheran Liu, Zheng Xiong, Luisa Zintgraf, Chelsea Finn, and Shimon Whiteson. A survey of meta-reinforcement learning.arXiv preprint arXiv:2301.08028,

    arXiv

  18. [2010]

    ISBN 9781605587998

    Association for Computing Machinery. ISBN 9781605587998. doi: 10.1145/1772690.1772758. URL https: //doi.org/10.1145/1772690.1772758. Yen-Chen Lin, Zhang-Wei Hong, Yuan-Hong Liao, Meng-Li Shih, Ming-Yu Liu, and Min Sun. Tactics of adversarial attack on deep reinforcement learni...

  19. [2015]

    Badnets: Identifying vulnerabilities in the machine learning model supply chain.arXiv preprint arXiv:1708.06733,

    Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain.arXiv preprint arXiv:1708.06733,

    arXiv

  20. [2017]

    Improved corruption robust algorithms for episodic reinforcement learning

    Yifang Chen, Simon Du, and Kevin Jamieson. Improved corruption robust algorithms for episodic reinforcement learning. In Marina Meila and Tong Zhang (eds.),Proceedings of the 38th Inter- national Conference on Machine Learning, volume 139 ofProceedings of Machine Learning Rese...

    2024

  21. [2018]

    ISBN 9781450355599

    Association for Computing Machinery. ISBN 9781450355599. doi: 10.1145/3188745.3188918. URL https://doi.org/10.1145/ 3188745.3188918. Yuzhe Ma, Xuezhou Zhang, Wen Sun, and Jerry Zhu. Policy poisoning in batch reinforcement learning and control. In H. Wallach, H. Larochelle, A. ...

  22. [2019]

    Shike Mei and Xiaojin Zhu

    URL https://proceedings.neurips.cc/paper_files/paper/ 2019/file/315f006f691ef2e689125614ea22cc61-Paper.pdf. Shike Mei and Xiaojin Zhu. Using machine teaching to identify optimal training-set attacks on machine learners. InProceedings of the aaai conference on artificial intell...

    2019

  23. [2020]

    Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901,

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901,

    1901

  24. [2021]

    Intriguing properties of neural networks.arXiv preprint arXiv:1312.6199,

    Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks.arXiv preprint arXiv:1312.6199,

    arXiv

  25. [2022]

    Juncheng Dong, Moyang Guo, Ethan X Fang, Zhuoran Yang, and Vahid Tarokh

    URL https://proceedings.mlr.press/v151/ding22c.html. Juncheng Dong, Moyang Guo, Ethan X Fang, Zhuoran Yang, and Vahid Tarokh. In-context reinforce- ment learning without optimal action labels. InICML 2024 Workshop on In-Context Learning,

    2024

  26. [2023]

    Evasion attacks against machine learning at test time

    Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim ˇSrndi´c, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. InMachine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2013, Pr...

    2013

  27. [2024]

    RL2: Fast reinforcement learning via slow reinforcement learning.arXiv preprint arXiv:1611.02779,

    Yan Duan, John Schulman, Xi Chen, Peter L Bartlett, Ilya Sutskever, and Pieter Abbeel. RL2: Fast reinforcement learning via slow reinforcement learning.arXiv preprint arXiv:1611.02779,

    arXiv

  28. [2025]

    Targeted backdoor attacks on deep learning systems using data poisoning.arXiv preprint arXiv:1712.05526,

    Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning.arXiv preprint arXiv:1712.05526,

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.