Pith. sign in

REVIEW 3 major objections 4 minor 43 references

BLAST: A Stealthy Backdoor Leverage Attack against Cooperative Multi-Agent Deep Reinforcement Learning based Systems

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read BLAST: one backdoored agent can make the whole cooperative team fail

desk verdict BLAST's attack recipe is concrete and evaluation is broad, but the headline 'leverage' claim isn't separated from plain self-sabotage by the backdoored agent. read the letter →

arxiv 2501.01593 v2 pith:MVLNMY66 submitted 2025-01-03 cs.AI cs.CRcs.LG

Jing Fang , Saihao Yan , Xueyu Yin , Yinbo Yu , Chunwei Tian , Jiajia Liu This is my paper · ORCID
classification cs.AIcs.CRcs.LG
keywords backdoorattackcooperativemulti-agentreinforcementlearningspatiotemporaltriggerrewardhackingsingle-agentpoisoningleverageVDNQMIX
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a cooperative multi-agent deep reinforcement learning team can be brought down by a backdoor planted in only one agent. The proposed attack, BLAST, uses a spatiotemporal behavior pattern as the trigger rather than a single visual patch, and delays malicious behavior until after the trigger period. The attack is trained by hacking only the backdoored agent's reward so that it steers its clean teammates toward failure states and away from optimal actions, exploiting the mutual influence between teammates. Across VDN, QMIX, and MAPPO on SMAC and Pursuit, the paper reports attack success rates up to 100%, effectiveness up to 88.5%, and clean-performance variance as low as 1.5%.

What carries the argument

The load-bearing machinery is the spatiotemporal trigger combined with a unilateral-influence reward hack. The trigger is a pair $T := (\Psi, \zeta)$, where $\Psi$ is a logical formula over the relative positions of the attacker's unit and the BLAST agent across a short window and $\zeta$ is the sequence of controllable actions the attacker performs; the attack period $L$ is decoupled from the trigger window. The hacked reward combines $r^{FS}_t$, the negative $\ell^2$ distance from clean teammates' next observations to pre-mined target failure observations, with $r^{AD}_t$, the deviation of their next actions from the frozen clean policy, and a parameter $\lambda$ balances the two. Training keeps the clean agents frozen, rolls the simulator back to compute both terms, and uses separate clean and poisoned replay buffers so the backdoored policy can be updated while clean behavior is preserved.

What would settle it

Run BLAST on the same algorithms and maps but with the simulator made non-rollback-able, so the action-deviation term $r^{AD}_t$ cannot be computed; if attack success stays high without counterfactual rollback, the paper's stated mechanism is not necessary, and if it collapses, the claimed leverage depends on a simulator capability that many real-world multi-agent systems do not provide.

Watch

Extended reading notes

Core claim

The central claim is that one compromised agent is sufficient to make the entire cooperative team fail, provided its reward is hacked to exert unilateral influence. BLAST treats a short sequence of an adversary-controlled unit's positions and actions as the trigger; once this sequence completes, the backdoored agent executes disruptive actions for a controllable attack period. Training minimizes a combined hacked reward: a failure-state term pushing clean teammates' next observations toward the lowest-reward observations mined from clean trajectories, and an action-deviation term pushing their next actions away from what the frozen clean policy would choose. The paper reports that this single-agent leverage degrades the team's reward and win rate sharply under trigger while leaving trigger-free performance nearly unchanged, and that the decoupled trigger-action timing lets the backdoor evade existing defenses in most tested settings.

Load-bearing premise

The attack works only if the attacker can rewind the environment simulator to compute counterfactual next observations, and if those counterfactuals predict what clean teammates will actually do during deployment; if the simulator is not rollback-able or the mined failure observations do not generalize, the single-agent leverage mechanism loses its training signal.

Editorial extensions

If this is right

  • A single agent retrained during deployment is enough to let an adversary-controlled unit in the environment force the team to fail, so third-party fine-tuning of one teammate model becomes a system-wide risk.
  • Trigger sequences can be chosen from natural adversary behavior, such as moving an enemy unit in a pattern, rather than from anomalous visual patches, making the backdoor harder to spot during normal operation.
  • The measured trade-off, with attack success rates up to 100% and clean-performance variance as low as 1.5%, means a deployer cannot infer compromise from clean-task performance alone.
  • Defenses that assume the trigger and the malicious action appear together will miss BLAST, because the attack acts in a later, separate time window.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same leverage mechanism points to a general single point of failure: any cooperative team trained with mutual-influence shaping should treat each agent's post-deployment training as a trust boundary, since one compromised policy can redirect collective behaviour.
  • Editorial inference: a natural adaptive defense follows from the paper's own timing: monitor delayed cross-correlations between an external unit's scripted movement and later team-level reward anomalies, rather than performing per-observation trigger detection.
  • Editorial inference: in physical systems without environment rollback, BLAST would need to approximate counterfactual observations from collected data; whether that approximation preserves the attack is a testable open question.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This manuscript proposes BLAST, a backdoor attack against cooperative multi-agent deep reinforcement learning (c-MADRL) in which a spatiotemporal behavior pattern of an external attacker-controlled unit serves as the trigger and only a single agent is implanted with the backdoor. The injection procedure hacks the poisoned agent's reward using two terms: target failure-state guidance (Eq. 8) and next-step action deviation of clean teammates (Eq. 9), balanced by hyperparameter λ (Eq. 10). The authors evaluate BLAST on SMAC and Pursuit against VDN, QMIX, and MAPPO, reporting attack success rates up to 100%, attack effectiveness rates up to 88.5%, and clean performance variance as low as 1.5%, together with partial evasion of activation clustering, spectral signatures, and BIRD. The core claimed contribution is a 'leverage attack effect' in which one backdoored agent unilaterally steers clean teammates into failure; the manuscript's main empirical weakness is that this mechanism is not separated from the direct disruptive effect of one malfunctioning agent.

Significance. The threat model is timely and the trigger design is a genuine contribution: decoupling the trigger from the attack action and distributing it over time is concrete and falsifiable, and Algorithm 2 specifies a usable injection procedure. If the leverage effect were established quantitatively, the work would provide a meaningful advance over all-agent backdoors and over prior single-agent attacks [14], [15]. However, the current evidence supports the attack's effectiveness as a one-agent mode of sabotage; it does not yet establish the claimed unilateral-influence mechanism. The λ ablation in Section V-E is a useful internal check, and the defense evaluation is a reasonable start. Reproducibility would be improved by releasing code, seeds, and normalization details.

major comments (3)
  1. [V-B (Table 2; Figs. 5–6)] The central claim of the paper is the 'leverage attack effect' (Abstract; Section I), but the experiments never test it against a control that would rule out self-sabotage. Table 2 compares BLAST only to a clean model, and the ablation in Section V-E varies λ and poisoning rate within the hacked-reward family. There is no condition in which the poisoned agent is trained with a non-hacked, equally disruptive reward (e.g., minimizing its own reward, fixed 'leave battlefield' script, or random actions). In SMAC, losing one unit in 8m/3m can itself produce team failure; the snapshots in Fig. 5 show the BLAST agent leaving the battlefield, so the reported ASR up to 100% and AER up to 88.5% could result from team-strength reduction rather than from BLAST's unilateral influence on clean agents. The action-distribution and per-agent reward analyses (Figs. 6 and 8) are qualitative and compare clean vs poisoned episodes, not the counterfactual trajectories that would isolate the mechanism. Please add such control conditions and measure, quantitatively, whether clean agents' observations/actions deviate more under BLAST than under the controls.
  2. [V-A (Table 2; Figs. 4, 7)] All headline numbers in Table 2 (and the curves in Figs. 4, 7) appear to come from a single run or a single seed; the Pursuit per-agent analysis in Fig. 8 is explicitly for 'the same random seed.' Since c-MADRL training is stochastic, the claimed margins—e.g., ASR 96.7% with CPVR 1.6% for 8m/VDN, or the λ=0.5 vs λ=0 difference in Fig. 11(a)—cannot be assessed without variance information. Report mean and standard deviation over at least five seeds and provide a statistical test (e.g., permutation or bootstrap) for ASR, AER, CPVR, and WRVR.
  3. [V (overall; Table 3)] The paper distinguishes BLAST from prior single-agent backdoor attacks [14], [15] in Table 3, but Section V contains no experimental comparison against these attacks under matched settings (same environment, same poisoning budget, same trigger conditions). The claim that BLAST is superior in stealth and effectiveness because it accounts for mutual influence is therefore not empirically substantiated. At minimum, include [14] and [15] (or a reimplementation) as baselines in Table 2, or state plainly that no comparison was performed and weaken the corresponding claims.
minor comments (4)
  1. [Abstract vs. V-D] The Abstract says '2 existing defense mechanisms' while the introduction and Section V-D evaluate three defenses (activation clustering, spectral signatures, BIRD); harmonize the count.
  2. [V-D] The sentence 'the detection results of spectral signature are shown in Figure 9' should refer to Figure 10, which contains the spectral-signature plots.
  3. [IV-B] The text says the method 'introduce[s] the unilateral influence filter [17]', but Eqs. (7)–(10) do not contain an explicit filter, only the two reward terms; clarify how [17] is used or remove the reference from this description.
  4. [IV-B] The normalization of r_FS and r_AD 'into the same range of values as the original reward' is stated but never defined; please give the exact normalization (e.g., min-max, z-score, or a scaling constant).

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the attack is evaluated against external environments and defenses, the reward-hack objective is not equivalent to the measured attack metrics, and self-citations are acknowledged extensions rather than load-bearing premises.

full rationale

The paper's derivation chain is self-contained rather than circular. The target failure observations are mined from clean trajectories by sorting transfer tuples by ascending reward and selecting the tuple with minimum R_t (Algorithm 1, Section IV-B), so they are inputs from clean data, not from the attack outcome. The hacked reward r_t = (1-lambda)*r_t^FS + lambda*r_t^AD (Equ. 10) is a training objective that minimizes distance to those failure observations and action deviation; it is not itself the evaluation metric. Attack success is measured externally by running the trained BLAST policy in SMAC and Pursuit and comparing ASR/AER against a clean model (Table 2). The ablation in Section V-E further shows the objective is not trivially equivalent to success: with lambda=1 (action-deviation only), ASR drops to 16.4%, so the measured attack success is not forced by the reward definition. The paper's self-citations ([5], [26], [37]) concern spatiotemporal trigger designs and are explicitly acknowledged as extensions ('This work extends our previous work [26]'), while the unilateral-influence concept is attributed to an external prior work [17]; neither reduces the central claim to a self-referential premise. The absence of a random-action or no-reward-hacking control weakens causal attribution of the 'leverage' effect, but that is an experimental-validity concern, not a circularity, because no equation equates the reward-hack objective with the measured attack-success metric by construction.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a small set of domain assumptions about attacker control of external units, frozen clean policies, and a rollback-able simulator, plus several hand-chosen hyperparameters (lambda, p, L, trigger thresholds). No new physical entities are introduced.

free parameters (6)
  • lambda (reward balance) = 0.5
    Weights the failure-state guidance vs action-deviation reward terms; selected via ablation on 8m VDN (Fig. 11a).
  • Poisoning rate p = 0.05 for 8m, 3m, Pursuit; 0.02 for 2s3z
    Fraction of poisoned episodes during retraining; chosen via ablation balancing ASR and CPVR (Fig. 11b).
  • Attack period L = 20 for 8m/3m, 40 for 2s3z/Pursuit
    Length of time steps after the trigger during which the backdoor agent behaves maliciously; set per map to control stealth and effectiveness.
  • Trigger period N_t = 4 or 5
    Length of the spatiotemporal trigger window; defined in Triggers 1-3 (Section V-A3).
  • Trigger spatial thresholds = e.g., 0.98 < x_e - x_b < 1.00
    Hand-crafted relative-position bands in the trigger definitions; specific to each environment's coordinate scale.
  • Exploration factor sigma = 0.05
    Greedy factor for the BLAST agent during retraining (Algorithm 2, Line 6).
assumptions (5)
  • domain assumption The environment transition F allows one-step lookahead and rollback to a previous state, including agent RNN hidden states.
    Required to compute the action-deviation reward r_AD in Equ. (9) (Section IV-B).
  • domain assumption Clean agents' policies remain frozen during backdoor injection and are known to the attacker.
    Used to compute pi_c(hat_o_{i,t+1}) in Equ. (9) and to isolate the backdoor agent's influence (Section IV-C).
  • ad hoc to paper The lowest-reward observation transition in clean data is a valid target failure state toward which pushing teammates causes team failure.
    Algorithm 1 selects o_fail as the observation after the minimum-reward tuple; no proof links this state to eventual team failure.
  • domain assumption The attacker can control an external unit (enemy in SMAC, evader in Pursuit) to execute the trigger sequence during both training and deployment.
    Trigger injection in Section V-A3 requires taking control of a non-agent entity; deployment assumes a user can act as such a unit in the environment.
  • domain assumption Mutual influence between agents can be reshaped by modifying one agent's reward so that its influence dominates.
    Borrowed from social influence literature [16] and adversarial minority influence [17]; not verified separately here.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BLAST: A Stealthy Backdoor Leverage Attack against Cooperative Multi-Agent Deep Reinforcement Learning based Systems." pith.science (2026). https://pith.science/paper/MVLNMY66

@misc{pith2026250101593,
  author       = {Pith},
  title        = {Pith review of: BLAST: A Stealthy Backdoor Leverage Attack against Cooperative Multi-Agent Deep Reinforcement Learning based Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MVLNMY66}},
  note         = {Machine review of arXiv:2501.01593}
}
read the original abstract

Recent studies have shown that cooperative multi-agent deep reinforcement learning (c-MADRL) is under the threat of backdoor attacks. Once a backdoor trigger is observed, it will perform malicious actions leading to failures or malicious goals. However, existing backdoor attacks suffer from several issues, e.g., instant trigger patterns lack stealthiness, the backdoor is trained or activated by an additional network, or all agents are backdoored. To this end, in this paper, we propose a novel backdoor leverage attack against c-MADRL, BLAST, which attacks the entire multi-agent team by embedding the backdoor only in a single agent. Firstly, we introduce adversary spatiotemporal behavior patterns as the backdoor trigger rather than manual-injected fixed visual patterns or instant status and control the period to perform malicious actions. This method can guarantee the stealthiness and practicality of BLAST. Secondly, we hack the original reward function of the backdoor agent via unilateral guidance to inject BLAST, so as to achieve the \textit{leverage attack effect} that can pry open the entire multi-agent system via a single backdoor agent. We evaluate our BLAST against 3 classic c-MADRL algorithms (VDN, QMIX, and MAPPO) in 2 popular c-MADRL environments (SMAC and Pursuit), and 2 existing defense mechanisms. The experimental results demonstrate that BLAST can achieve a high attack success rate while maintaining a low clean performance variance rate.

Figures

Figures reproduced from arXiv: 2501.01593 by the authors.

Figure 1
Figure 1. A comparison of backdoor attacks against c-MADRL. (a) [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The framework of our proposed BLAST backdoor attack. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The illustration of the experimental environments. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Training curves of average episode rewards and average winning rates of BLAST models attacking against VDN, QMIX, and MAPPO [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: The snapshots of agents’ behaviors in a poisoned episode. The [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Action distribution of all clean agents in a clean episode and [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: The average episode rewards of the BLAST model of VDN [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 9
Figure 9. Figure 9: Backdoor detection results via activation clustering [18]. “Pre” [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: Backdoor detection results via spectral signature [19]. [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]
Figure 11
Figure 11. Figure 11: ASR and CPVR achieved under different parameters. [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 34 canonical work pages

  1. [14]

    Backdoor attacks on multiagent collaborative systems,

    S. Chen, Y . Qiu, and J. Zhang, “Backdoor attacks on multiagent collaborative systems,”arXiv preprint arXiv:2211.11455, 2022

    arXiv 2022

  2. [15]

    One4all: Ma- nipulate one agent to poison the cooperative multi-agent reinforcement learning,

    H. Zheng, X. Li, J. Chen, J. Dong, Y . Zhang, and C. Lin, “One4all: Ma- nipulate one agent to poison the cooperative multi-agent reinforcement learning,”Computers & Security, vol. 124, p. 103005, 2023

    work page 2023

  3. [1]

    Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning,

    X. Chen, H. Zhang, C. Wu, S. Mao, Y . Ji, and M. Bennis, “Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning,”IEEE Internet of Things Journal, vol. 6, no. 3, pp. 4005–4018, 2018

    work page 2018

  4. [2]

    Joint computation offloading and resource allocation for leo satellite networks using hierarchical multi-agent reinforcement learning,

    J. Lai, H. Liu, G. Xu, W. Jiang, X. Wang, and D. Jiang, “Joint computation offloading and resource allocation for leo satellite networks using hierarchical multi-agent reinforcement learning,”IEEE TCCN, pp. 1–14, 2024

    work page 2024

  5. [3]

    The starcraft multi-agent challenge,

    M. Samvelyan, T. Rashid, C. S. De Witt, G. Farquhar, N. Nardelli, T. G. Rudner, C.-M. Hung, P. H. Torr, J. Foerster, and S. Whiteson, “The starcraft multi-agent challenge,”arXiv preprint arXiv:1902.04043, 2019

    arXiv 1902

  6. [4]

    Efficient twin migration in vehicular metaverses: Multi- agent split deep reinforcement learning with spatio-temporal trajectory generation,

    J. Chen, J. Kang, M. Xu, F. Wu, H. Zhang, H. Huang, D. Niyato, and S. Mao, “Efficient twin migration in vehicular metaverses: Multi- agent split deep reinforcement learning with spatio-temporal trajectory generation,”IEEE TMC, pp. 1–14, 2025

    work page 2025

  7. [5]

    A spatiotemporal backdoor attack against behavior-oriented decision makers in metaverse: From perspective of autonomous driving,

    Y . Yu, J. Liu, H. Guo, B. Mao, and N. Kato, “A spatiotemporal backdoor attack against behavior-oriented decision makers in metaverse: From perspective of autonomous driving,”IEEE JSAC, 2023

    work page 2023

  8. [6]

    Design of intentional backdoors in sequential models,

    Z. Yang, N. Iyer, J. Reimann, and N. Virani, “Design of intentional backdoors in sequential models,”arXiv preprint arXiv:1902.09972, 2019

    arXiv 1902

Show all 43 references
  1. [7]

    Backdoorl: Backdoor attack against competitive reinforcement learning,

    L. Wang, Z. Javed, X. Wu, W. Guo, X. Xing, and D. Song, “Backdoorl: Backdoor attack against competitive reinforcement learning,” inIJCAI, 2021, pp. 3699–3705

    2021

  2. [8]

    Poisoning deep reinforcement learning agents with in-distribution triggers,

    C. Ashcraft and K. Karra, “Poisoning deep reinforcement learning agents with in-distribution triggers,”arXiv preprint arXiv:2106.07798, 2021

    2021 arXiv

  3. [9]

    Stop-and- go: Exploring backdoor attacks on deep reinforcement learning-based traffic congestion control systems,

    Y . Wang, E. Sarkar, W. Li, M. Maniatakos, and S. E. Jabari, “Stop-and- go: Exploring backdoor attacks on deep reinforcement learning-based traffic congestion control systems,”IEEE TIFS, vol. 16, pp. 4772–4787, 2021

    2021

  4. [10]

    BAFFLE: Hiding backdoors in offline reinforcement learning datasets,

    C. Gong, Z. Yang, Y . Bai, J. Shi, J. He, K. Li, B. Xu, A. Sinha, X. Hou, D. Lo, and T. Wang, “BAFFLE: Hiding backdoors in offline reinforcement learning datasets,” inIEEE S&P, 2024, pp. 218–218

    2024

  5. [11]

    Pilot backdoor attack against deep reinforcement learning empowered intelligent reflection surface for smart radio,

    Y . Huang, H.-M. Wang, Z. Wang, and W. Liu, “Pilot backdoor attack against deep reinforcement learning empowered intelligent reflection surface for smart radio,”IEEE TWC, vol. 24, no. 6, pp. 4891–4903, 2025

    2025

  6. [12]

    Backdoor attacks on multi-agent reinforcement learning-based spectrum management,

    H. Zhang, M. Liu, and Y . Chen, “Backdoor attacks on multi-agent reinforcement learning-based spectrum management,” inIEEE GLOBE- COM, 2023, pp. 3361–3365

    2023

  7. [13]

    Marnet: Backdoor attacks against cooperative multi-agent reinforcement learning,

    Y . Chen, Z. Zheng, and X. Gong, “Marnet: Backdoor attacks against cooperative multi-agent reinforcement learning,”IEEE TDSC, vol. 20, no. 5, pp. 4188–4198, 2023

    2023

  8. [16]

    Social influence as intrinsic motivation for multi-agent deep reinforcement learning,

    N. Jaques, A. Lazaridou, E. Hughes, C. Gulcehre, P. Ortega, D. Strouse, J. Z. Leibo, and N. De Freitas, “Social influence as intrinsic motivation for multi-agent deep reinforcement learning,” inICML, 2019, pp. 3040– 3049

    2019

  9. [17]

    Attacking cooperative multi-agent reinforcement learning by adversarial minority influence,

    S. Li, J. Guo, J. Xiu, P. Feng, X. Yu, A. Liu, W. Wu, and X. Liu, “Attacking cooperative multi-agent reinforcement learning by adversarial minority influence,”arXiv preprint arXiv:2302.03322, 2023

    2023 arXiv

  10. [18]

    Detecting backdoor attacks on deep neural networks by activation clustering,

    B. Chen, W. Carvalho, N. Baracaldo, H. Ludwig, B. Edwards, T. Lee, I. M. Molloy, and B. Srivastava, “Detecting backdoor attacks on deep neural networks by activation clustering,” inAAAI, ser. CEUR Workshop Proceedings, vol. 2301, 2019

    2019

  11. [19]

    Spectral signatures in backdoor attacks,

    B. Tran, J. Li, and A. Madry, “Spectral signatures in backdoor attacks,” inNeurIPS, 2018, pp. 8011–8021

    2018

  12. [20]

    Value-decomposition networks for cooperative multi-agent learning based on team reward,

    P. Sunehag, G. Lever, A. Gruslys, W. M. Czarnecki, V . F. Zambaldi, M. Jaderberg, M. Lanctot, N. Sonnerat, J. Z. Leibo, K. Tuyls, and T. Graepel, “Value-decomposition networks for cooperative multi-agent learning based on team reward,” inAAMAS, 2018, pp. 2085–2087

    2018

  13. [21]

    QMIX: monotonic value function factorisation for deep multi-agent reinforcement learning,

    T. Rashid, M. Samvelyan, C. S. de Witt, G. Farquhar, J. N. Foerster, and S. Whiteson, “QMIX: monotonic value function factorisation for deep multi-agent reinforcement learning,” inICML, vol. 80, 2018, pp. 4292–4301

    2018

  14. [22]

    Counterfactual multi-agent policy gradients,

    J. N. Foerster, G. Farquhar, T. Afouras, N. Nardelli, and S. Whiteson, “Counterfactual multi-agent policy gradients,” inAAAI, 2018, pp. 2974– 2982

    2018

  15. [23]

    The surprising effectiveness of PPO in cooperative multi-agent games,

    C. Yu, A. Velu, E. Vinitsky, J. Gao, Y . Wang, A. M. Bayen, and Y . Wu, “The surprising effectiveness of PPO in cooperative multi-agent games,” inNeurIPS, 2022

    2022

  16. [24]

    Deep recurrent q-learning for partially observable mdps,

    M. J. Hausknecht and P. Stone, “Deep recurrent q-learning for partially observable mdps,” inAAAI Fall Symposia, 2015, pp. 29–37

    2015

  17. [25]

    Privacy-preserving multi- agent deep reinforcement learning for effective resource auction in multi- 13 access edge computing,

    F. You, X. Yuan, W. Ni, and A. Jamalipour, “Privacy-preserving multi- agent deep reinforcement learning for effective resource auction in multi- 13 access edge computing,”IEEE TCCN, vol. 11, no. 3, pp. 1887–1901, 2025

    1901

  18. [26]

    A spatiotemporal stealthy backdoor attack against cooperative multi-agent deep reinforcement learning,

    Y . Yu, S. Yan, and J. Liu, “A spatiotemporal stealthy backdoor attack against cooperative multi-agent deep reinforcement learning,” inIEEE GLOBECOM, 2024

    2024

  19. [27]

    Optimal and approxi- mate q-value functions for decentralized pomdps,

    F. A. Oliehoek, M. T. J. Spaan, and N. Vlassis, “Optimal and approxi- mate q-value functions for decentralized pomdps,”J. Artif. Intell. Res., vol. 32, pp. 289–353, 2008

    2008

  20. [28]

    Multi-agent reinforcement learning as a rehearsal for decentralized planning,

    L. Kraemer and B. Banerjee, “Multi-agent reinforcement learning as a rehearsal for decentralized planning,”Neurocomputing, vol. 190, pp. 82–94, 2016

    2016

  21. [29]

    Prox- imal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  22. [30]

    Deep multi-agent reinforcement learning for highway on- ramp merging in mixed traffic,

    D. Chen, M. R. Hajidavalloo, Z. Li, K. Chen, Y . Wang, L. Jiang, and Y . Wang, “Deep multi-agent reinforcement learning for highway on- ramp merging in mixed traffic,”IEEE TITS, vol. 24, no. 11, pp. 11 623– 11 638, 2023

    2023

  23. [31]

    Conser- vative and risk-aware offline multi-agent reinforcement learning,

    E. Eldeeb, H. Sifaou, O. Simeone, M. Shehab, and H. Alves, “Conser- vative and risk-aware offline multi-agent reinforcement learning,”IEEE TCCN, vol. 11, no. 3, pp. 1913–1926, 2025

    1913

  24. [32]

    Badrl: Sparse targeted backdoor attack against reinforcement learning,

    J. Cui, Y . Han, Y . Ma, J. Jiao, and J. Zhang, “Badrl: Sparse targeted backdoor attack against reinforcement learning,” inAAAI, 2024, pp. 11 687–11 694

    2024

  25. [33]

    Cooperative multi- agent control using deep reinforcement learning,

    J. K. Gupta, M. Egorov, and M. J. Kochenderfer, “Cooperative multi- agent control using deep reinforcement learning,” inAAMAS Workshops, vol. 10642, 2017, pp. 66–83

    2017

  26. [34]

    Pettingzoo: Gym for multi-agent reinforcement learning,

    J. K. Terry, B. Black, N. Grammel, M. Jayakumar, A. Hari, R. Sullivan, L. S. Santos, C. Dieffendahl, C. Horsch, R. Perez-Vicente, N. L. Williams, Y . Lokesh, and P. Ravi, “Pettingzoo: Gym for multi-agent reinforcement learning,” inNeurIPS, 2021, pp. 15 032–15 043

    2021

  27. [35]

    BIRD: generalizable backdoor detection and removal for deep reinforcement learning,

    X. Chen, W. Guo, G. Tao, X. Zhang, and D. Song, “BIRD: generalizable backdoor detection and removal for deep reinforcement learning,” in NeurIPS, 2023

    2023

  28. [36]

    Trojdrl: Evaluation of backdoor attacks on deep reinforcement learning,

    P. Kiourti, K. Wardega, S. Jha, and W. Li, “Trojdrl: Evaluation of backdoor attacks on deep reinforcement learning,” inDAC, 2020, pp. 1–6

    2020

  29. [37]

    A temporal-pattern backdoor attack to deep reinforcement learning,

    Y . Yu, J. Liu, S. Li, K. Huang, and X. Feng, “A temporal-pattern backdoor attack to deep reinforcement learning,” inIEEE GLOBECOM, 2022, pp. 2710–2715

    2022

  30. [38]

    Backdoor learning: A survey,

    Y . Li, Y . Jiang, Z. Li, and S. Xia, “Backdoor learning: A survey,”IEEE TNNLS, vol. 35, no. 1, pp. 5–22, 2024

    2024

  31. [39]

    Neural cleanse: Identifying and mitigating backdoor attacks in neural networks,

    B. Wang, Y . Yao, S. Shan, H. Li, B. Viswanath, H. Zheng, and B. Y . Zhao, “Neural cleanse: Identifying and mitigating backdoor attacks in neural networks,” inIEEE S&P, 2019, pp. 707–723

    2019

  32. [40]

    Neuralsanitizer: Detecting backdoors in neural networks,

    H. Zhu, Y . Zhao, S. Zhang, and K. Chen, “Neuralsanitizer: Detecting backdoors in neural networks,”IEEE TIFS, vol. 19, pp. 4970–4985, 2024

    2024

  33. [41]

    Policycleanse: Backdoor detection and mitigation for competitive reinforcement learning,

    J. Guo, A. Li, L. Wang, and C. Liu, “Policycleanse: Backdoor detection and mitigation for competitive reinforcement learning,” inICCV, 2023, pp. 4676–4685

    2023

  34. [42]

    Mitigating deep reinforcement learning backdoors in the neural activation space,

    S. Vyas, C. Hicks, and V . Mavroudis, “Mitigating deep reinforcement learning backdoors in the neural activation space,” inIEEE S&P Workshops, 2024, pp. 76–86

    2024

  35. [43]

    Shine: Shielding backdoors in deep reinforcement learning,

    Z. Yuan, W. Guo, J. Jia, B. Li, and D. Song, “Shine: Shielding backdoors in deep reinforcement learning,” inICML, 2024

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.