Pith. sign in

REVIEW 3 major objections 4 minor 78 references

Multi-Agent Reinforcement Learning in Cybersecurity: From Fundamentals to Applications

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MARL offers a path to self-learning cyber defense, survey argues

desk verdict A serviceable MARL-for-cyber-defense survey that needs fact-checking and a more disciplined conclusion before it earns reference status. read the letter →

arxiv 2505.19837 v1 pith:RXA3QNNV submitted 2025-05-26 cs.MA cs.GTcs.LG

Christoph R. Landolt , Christoph Würsch , Roland Meier , Alain Mermoud , Julian Jang-Jaccard This is my paper
classification cs.MAcs.GTcs.LG
keywords Multi-AgentReinforcementLearningAutomatedCyberDefenseAutonomousIntelligentCyber-defenseAgentsGymsIntrusionDetectionLateralMovementGameTheoryDecentralized
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This survey tries to establish that multi-agent reinforcement learning (MARL) is more than an incremental upgrade to single-agent RL for cybersecurity: it is the enabling technology for autonomous, adaptive, and decentralized cyber defense. The paper argues that MARL overcomes three concrete limitations of single-agent approaches—exponential scaling problems in large networks, inability to model interacting attackers and defenders, and lack of coordination across distributed sensors—by casting defense as a game among agents and training them in cyber gyms. Its central use cases are intrusion detection and lateral movement containment, the two phases where coordinated, sophisticated attacks (including advanced persistent threats) do the most damage. A sympathetic reader would care because the survey identifies a concrete route from today's static, rule-based intrusion detection to a future of collaborating self-learning defense agents, while also naming the reality gap between simulation and live networks as the main obstacle.

What carries the argument

The load-bearing machinery is the game-theoretic model hierarchy that scales from repeated normal-form games up to partially observable stochastic games (POSGs), and specifically the decentralized POMDP (Dec-POMDP) subclass in which multiple defenders act on local observations while optimizing a shared reward. This formal frame is what answers the survey's three stated objections to single-agent RL. On top of it sits centralized training with decentralized execution (CTDE), the design pattern—used by algorithms such as MADDPG and value decomposition networks—in which agents learn with a global critic that sees joint actions but execute using only local policies. The second half of the machinery is the cyber gym: simulation and emulation environments that generate attack graphs, lateral movement, and red-vs-blue interactions, which the survey treats as the indispensable bridge between MARL algorithms and deployed AICA. These two components together carry the argument from 'MARL exists' to 'MARL can defend networks'.

What would settle it

Train a defender policy in a lateral-movement simulator, then deploy it unmodified in an emulated enterprise network with a topology and background traffic it never saw, against a red team using novel exploit sequences. If its containment rate falls to the level of a static rule-based intrusion detection system, the survey's core promise that MARL enables autonomous cyber defense would be refuted.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that MARL supplies the missing adaptive layer for automated cyber defense. Where single-agent RL agents must observe and control an entire environment, MARL agents are trained as a distributed team under partial observability—each defender sees only part of the network and must coordinate with others through shared rewards or competitive pressure. The survey marshals evidence that this formulation fits the structure of real attacks: red agents learning lateral movement against blue agents that block, isolate, and reconfigure, with the interaction formalized as a partially observable stochastic game (a Dec-POMDP when defenders share a joint reward). It further claims that existing cyber gyms—simulation and emulation platforms for network attack and defense—are the training grounds where such agents can be developed, and that the AICA reference architecture with its sensing, planning, acting, collaboration, and learning functions is the natural home for MARL-based defenders. The conclusion is that MARL is a viable foundation for autonomous cyber defense, with scalability and adversarial robustness as open research problems rather than disqualifying failures.

Load-bearing premise

The load-bearing premise is that MARL agents trained in simulated cyber gyms will perform well enough on real operational networks, even though the survey itself acknowledges a persistent 'reality gap' between simulation and practice.

Editorial extensions

If this is right

  • Intrusion detection shifts from a single central RL agent analyzing all network traffic to a swarm of local defender agents that share observations and jointly decide when to alert or act, improving scalability and resilience.
  • Defenders can be trained adversarially against autonomous red agents, so that blue-team policies are continually sharpened by increasingly sophisticated attacks rather than fixed signatures.
  • Lateral movement containment becomes a Dec-POMDP problem, giving formal grounding for agents that detect credential reuse and privilege escalation and cut off the attacker's path through the network.
  • The combination of simulation and emulation in cyber gyms provides a testable path to close the reality gap, making it possible to validate AICA before deployment.
  • Scalability and adversarial robustness remain the principal open problems, so the near-term contribution of MARL is in assisted and semi-autonomous defense rather than fully unattended cyber operations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not say this, but its red-team/blue-team framing implies a dual-use dynamic: MARL training that produces stronger autonomous defenders also produces stronger autonomous attackers, so offensive and defensive capability rise together.
  • An unstated corollary of the reality-gap discussion is that simulator fidelity, not algorithm choice, is likely the binding constraint; a direct benchmark of one defender policy across a pure simulator, a hybrid emulator, and a live testbed would quantify how much of the survey's promise survives contact with real networks.
  • The survey lists generative models as future work; if adversarial scenario generation becomes cheap, cyber gyms could automatically produce the long-tail attack patterns that today make simulation-trained agents brittle, turning the reality gap from a training problem into a data-generation problem.
  • If CTDE methods scale to thousands of agents, the same architecture could defend not one enterprise network but a federation of networks under different administrative domains, which is the military coalition scenario that motivates much of the AICA work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This survey reviews the state of research on multi-agent reinforcement learning (MARL) for automated cyber defense, with emphasis on intrusion detection, lateral movement containment, AICA reference architectures, and cyber-gym training environments. It introduces game-theoretic models (repeated normal-form games, stochastic games, POMDPs, Dec-POMDPs), overviews common MARL algorithms (Q-learning, CTDE, VDN, IPPO, MAPPO, MADDPG), and surveys simulation environments such as CyberBattleSim, CybORG++, NASimEmu, and VINE. The paper argues that MARL enables adaptive, decentralized, and collaborative defense strategies while acknowledging open challenges in scalability, adversarial robustness, and simulation-to-reality transfer. It concludes that MARL is a transformative approach to automating cyber defense and identifies future research directions including more realistic training environments and integration with generative models.

Significance. If the survey's reading of the literature is accurate, it provides a useful, structured entry point for researchers seeking to apply MARL to cyber defense. Its strengths are the systematic organization of MARL algorithms and training environments, the inclusion of recent frameworks (CybORG++, CAGE 4, NASimEmu), and a clear separation of cooperative, competitive, and mixed-interest scenarios. The paper also honestly identifies key open problems, notably the reality gap between simulated training and real-world deployment. It does not offer new technical results, machine-checked proofs, or reproducible code; its contribution is synthetic. The usefulness of the survey is undermined, however, by a material factual error in the introduction, an internal contradiction about the reality gap, and a conclusion that overstates the evidence for deployment readiness.

major comments (3)
  1. [Section I, first paragraph] The statement that cybercrime will cost the global economy $10.5 billion in 2025 [1] is factually wrong; the cited WEF source, like common industry estimates, reports a figure in trillions (approximately $10.5 trillion). This is a material error in a central motivating statistic and should be corrected.
  2. [Section IV-B, second paragraph] The paragraph first claims that 'Tools like NASimEmu [53] and VINE [57] bridge this gap by providing realistic environments for policy evaluation,' then a few sentences later states that 'a key challenge remains in applying agents trained in simulated environments ... to real-world network environments due to the persistent reality gap.' These statements are contradictory: if the tools bridge the gap, the gap cannot be persistent. The paper should clarify whether these environments fully bridge the simulation-to-reality gap or merely reduce it, and it should reconcile this claim with the acknowledged persistent gap.
  3. [Section VI, first paragraph] The conclusion asserts that MARL 'offers a transformative approach' and 'can emerge as a critical asset' in real adversarial environments, but the surveyed evidence does not include a single example of a MARL policy transferred to a live production network. All training results described come from simulated or emulated testbeds, and the paper itself admits that deployment gaps limit MARL's current deployment. The conclusion should be tempered to reflect the current evidence, or the survey should cite work demonstrating a concrete path from simulation to deployment.
minor comments (4)
  1. [Sections IV-B and V-B] The tool name is spelled inconsistently: 'NASimEmu' appears in Section IV-B and later as 'NaSimEmu' in Section V-B, Red Teaming. Please standardize the spelling.
  2. [Reference [34]] Reference [34] is incorrectly formatted: Sutton and Barto's 'Reinforcement Learning: An Introduction' is a book, not an article in IEEE Transactions on Neural Networks, and the author list should not use 'et al.' for two authors. The in-text citation should also say 'Sutton and Barto [34]' rather than 'Sutton et al. [34]'.
  3. [Section I] Several words are missing spaces due to apparent formatting errors: 'staticrule-based,' 'dynamicanomaly-based,' and 'throughtrial and error' should be 'static rule-based,' 'dynamic anomaly-based,' and 'through trial and error.'
  4. [Section II-A, subsection 2] The phrase 'MARL uses frameworks like Nash Equilibria [30]' is imprecise: a Nash equilibrium is a solution concept, not a framework or method. Consider rephrasing to 'game-theoretic solution concepts such as Nash equilibrium provide a theoretical basis'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the survey's claims are grounded in external references and independent benchmarks; the two author self-citations are descriptive and not load-bearing.

full rationale

The paper is a survey, not a derivation. Its central claim that MARL offers a transformative approach to cyber defense is supported by a wide range of external references (e.g., [17]-[20], [24], [25], [53], [59], [60], [61]) and by descriptions of external benchmarks. The only author self-citations are Ref. [62] (Meier et al.) in the AICA reference-architecture section and Ref. [64] (Alavizadeh/Jang-Jaccard) in the intrusion-detection example. Both are descriptive and exemplary, not load-bearing: the AICA architecture is also attributed to Kott et al. [21], and the DQN intrusion-detection claim is one of several supporting examples alongside the survey [63]. No equation or prediction in the paper is fitted to, or defined in terms of, the claimed output; there is no imported uniqueness theorem and no ansatz smuggled in via citation. The paper's own admission of a "persistent reality gap" between simulation and deployment (Section IV-B) does undercut the strength of the concluding claim, but that is a consistency and evidence-strength concern, not a circularity. Accordingly, the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper is a survey and introduces no free parameters or new entities. Its claims rest on background assumptions about the suitability of game-theoretic RL models, the transferability of simulation-trained agents, and the accuracy of the cited literature.

assumptions (4)
  • domain assumption Game-theoretic models (POSG, Dec-POMDP, stochastic games) are appropriate formal frameworks for modeling cyber defense scenarios.
    Section III-A introduces these models as the foundation for MARL without justifying their fit to real cyber operations beyond illustrative examples.
  • domain assumption RL agents trained in simulated environments can generalize to real-world cyber defense tasks.
    The entire survey's policy recommendation depends on simulation-to-real transfer, which Section IV-B flags as an unresolved 'reality gap'.
  • domain assumption The cited primary studies are correctly represented and collectively substantiate the claim that MARL improves cyber defense.
    A narrative survey assumes its source selection and interpretations are accurate; no systematic verification is provided.
  • domain assumption The AICA reference architecture (from Kott et al. and Meier et al.) is a valid framework for integrating MARL into automated cyber defense.
    Section IV-C relies on this architecture, partly from a self-cited prior work [62], without independent evaluation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-Agent Reinforcement Learning in Cybersecurity: From Fundamentals to Applications." pith.science (2026). https://pith.science/paper/RXA3QNNV

@misc{pith2026250519837,
  author       = {Pith},
  title        = {Pith review of: Multi-Agent Reinforcement Learning in Cybersecurity: From Fundamentals to Applications},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RXA3QNNV}},
  note         = {Machine review of arXiv:2505.19837}
}
read the original abstract

Multi-Agent Reinforcement Learning (MARL) has shown great potential as an adaptive solution for addressing modern cybersecurity challenges. MARL enables decentralized, adaptive, and collaborative defense strategies and provides an automated mechanism to combat dynamic, coordinated, and sophisticated threats. This survey investigates the current state of research in MARL applications for automated cyber defense (ACD), focusing on intruder detection and lateral movement containment. Additionally, it examines the role of Autonomous Intelligent Cyber-defense Agents (AICA) and Cyber Gyms in training and validating MARL agents. Finally, the paper outlines existing challenges, such as scalability and adversarial robustness, and proposes future research directions. This also discusses how MARL integrates in AICA to provide adaptive, scalable, and dynamic solutions to counter the increasingly sophisticated landscape of cyber threats. It highlights the transformative potential of MARL in areas like intrusion detection and lateral movement containment, and underscores the value of Cyber Gyms for training and validation of AICA.

Figures

Figures reproduced from arXiv: 2505.19837 by the authors.

Figure 1
Figure 1. Reinforcement Learning (RL) control loop, inspired by Sutton et [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Nested structure of game models. The Partially Observable Stochastic Game includes n agents and m partially observable states. A simpler model is the Stochastic Game, which assumes the complete observability of the states, while the Repeated Normal-Form Game involves n agents interacting in a single state. Adapted from [24]. These game-theoretic frameworks allow modelling complex inter-agent dynamics and address unc… view at source ↗
Figure 4
Figure 4. Cooperative Defense of a System by n Distributed Defenders through a cooperative MARL-System, where the defender agents collaborate to maximize their expected reward Es,u[R(s, u)], with R(s, u) denoting the reward received when all agents take a joint action u in state s. Each defender agent operates independently, optimizing its policy πi based on local observations and rewards. This collaborative process can descr… view at source ↗
Figures from the paper (6 more)
Figure 3
Figure 3. Figure 3: Interaction between multiple agents, an optional communication [PITH_FULL_IMAGE:figures/full_fig_p004_3.png]
Figure 5
Figure 5. Figure 5: Zero-sum interaction between an attacker and defender agent in an [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Adversarial setup where n attack agents maximize their expected reward E[R(s, uA, uD)], while m defender agents minimize it. Both groups optimize their respective policies (πA,i and πD,j ) and interact through a shared environment, with joint actions uA and uD defining…
Figure 8
Figure 8. Figure 8: Comparison of SARL and MARL for Intrusion Detection System [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Multi-attacker scenarios in a network. Left: Attackers 1 and 2 share the same set of exploits but aim for different targets. Both attackers follow their respective optimal paths to achieve their goals. Right: Attacker 1 and Attacker 2 use different sets of exploits and…
Figure 11
Figure 11. Figure 11: Attack graph starting from the Internet. The attacker exploits a public [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

78 extracted references · 55 canonical work pages

  1. [1]

    Muggah and M

    R. Muggah and M. Margolis. (2023, Jan.) Why we need global rules to crack down on cybercrime. Accessed: 2024-11-19. [Online]. Available: https://www.weforum.org/stories/2023/01/global-rules-crack-down-cyb ercrime/

    work page 2023

  2. [53]

    Nasimemu: Network attack simulator & emulator for training agents generalizing to novel scenarios,

    J. Janisch, T. Pevn ´y, and V . Lis´y, “Nasimemu: Network attack simulator & emulator for training agents generalizing to novel scenarios,” 2023. [Online]. Available: https://arxiv.org/abs/2305.17246

    arXiv 2023

  3. [57]

    Vine: A cyber emulation environment for mtd experimentation,

    T. C. Eskridge, M. M. Carvalho, E. Stoner, T. Toggweiler, and A. Granados, “Vine: A cyber emulation environment for mtd experimentation,” inProceedings of the Second ACM Workshop on Moving Target Defense, ser. MTD ’15. New York, NY , USA: Association for Computing Machinery, 2015, p. 43–47. [Online]. Available: https://doi.org/10.1145/2808475.2808486

    work page arXiv 2015

  4. [2]

    Dynamic vs. static cybersecurity: Which approach is more effective?

    Cyber Insight, “Dynamic vs. static cybersecurity: Which approach is more effective?” https://cyberinsight.co/what-is-the-difference-between -dynamic-and-static-cybersecurity/, 2023, updated on: June 17, 2023. Accessed: 2024-12-11

    work page 2023

  5. [3]

    Does traditional security protect against modern threats?

    T. Kenyon, “Does traditional security protect against modern threats?” https://cybermagazine.com/cyber-security/does-traditional-security-p rotect-against-modern-threats, 2021, published on December 2, 2021. Accessed: 2024-12-11

    work page 2021

  6. [4]

    As the threat landscape changes, traditional cybersecurity approaches need to evolve,

    O. Vareilhes, “As the threat landscape changes, traditional cybersecurity approaches need to evolve,” 2024, published on August 12, 2024. Accessed: 2024-12-11. [Online]. Available: https://kudelskisecurity.c om/modern-ciso-blog/as-the-threat-landscape-changes-traditional-cyb ersecurity-approaches-need-to-evolve/

    work page 2024

  7. [5]

    Cyber grand challenge,

    DARPA, “Cyber grand challenge,” 2016, available at: https://www.darp a.mil/about-us/timeline/cyber-grand-challenge

    work page 2016

  8. [6]

    The mayhem cyber reasoning system,

    T. Avgerinos, D. Brumley, J. Davis, R. Goulden, T. Nighswander, A. Rebert, and N. Williamson, “The mayhem cyber reasoning system,” IEEE Security & Privacy, vol. 16, no. 2, pp. 52–60, 2018

    work page 2018

Show all 78 references
  1. [7]

    Xandra: An autonomous cyber battle system for the cyber grand challenge,

    A. Nguyen-Tuong, D. Melski, J. W. Davidson, M. Co, W. Hawkins, J. D. Hiser, D. Morris, D. Nguyen, and E. Rizzi, “Xandra: An autonomous cyber battle system for the cyber grand challenge,”IEEE Security & Privacy, vol. 16, no. 2, pp. 42–51, 2018

    2018

  2. [8]

    Mechanical phish: Resilient autonomous hacking,

    Y . Shoshitaishvili, A. Bianchi, K. Borgolte, A. Cama, J. Corbetta, F. Disperati, A. Dutcher, J. Grosen, P. Grosen, A. Machiry, C. Salls, N. Stephens, R. Wang, and G. Vigna, “Mechanical phish: Resilient autonomous hacking,”IEEE Security & Privacy, vol. 16, no. 2, 2018

    2018

  3. [9]

    The future of Cyber-Autonomy,

    D. Brumley, “The future of Cyber-Autonomy,” inEnigma 2018 (Enigma 2018). Santa Clara, CA: USENIX Association, Jan. 2018. [Online]. Available: https://www.usenix.org/node/208122

    2018

  4. [10]

    Orientation guide for the security of critical infrastructures,

    Federal Office for Information Security, “Orientation guide for the security of critical infrastructures,” Bonn, Germany, Sep. 2022, accessed: 2023-12-11. [Online]. Available: https://www.bsi.bund.de/Sh aredDocs/Downloads/EN/BSI/KRITIS/oh sza en.pdf

    2022

  5. [11]

    Fuzzing: Chal- lenges and reflections,

    M. Boehme, C. Cadar, and A. ROYCHOUDHURY , “Fuzzing: Chal- lenges and reflections,”IEEE Software, vol. 38, no. 3, pp. 79–86, 2021

    2021

  6. [12]

    Reinforcement learning for iot security: A comprehensive survey,

    A. Uprety and D. B. Rawat, “Reinforcement learning for iot security: A comprehensive survey,”IEEE Internet of Things Journal, vol. 8, no. 11, p. 8693–8706, Jun. 2021. [Online]. Available: http://dx.doi.org/10.1109/JIOT.2020.3040957

    2021

  7. [13]

    Cyber-security and reinforcement learning — a brief survey,

    A. M. K. Adawadkar and N. Kulkarni, “Cyber-security and reinforcement learning — a brief survey,”Engineering Applications of Artificial Intelligence, vol. 114, p. 105116, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0952197622002512

    2022

  8. [14]

    A review of machine learning-based zero-day attack detection: Challenges and future directions,

    Y . Guo, “A review of machine learning-based zero-day attack detection: Challenges and future directions,”Computer Communications, vol. 198, pp. 175–185, 2023. [Online]. Available: https://www.sciencedirect.com/ science/article/pii/S0140366422004248

    2023

  9. [15]

    Z. Hu, P. Chen, M. Zhu, and P. Liu,Reinforcement Learning for Adaptive Cyber Defense Against Zero-Day Attacks. Cham: Springer International Publishing, 2019, pp. 54–93. [Online]. Available: https://doi.org/10.1007/978-3-030-30719-6 4

    2019 doi

  10. [16]

    Reinforcement learning applications in cyber security: A review,

    E. Cengiz and M. G ¨ok, “Reinforcement learning applications in cyber security: A review,”Sakarya University Journal of Science, vol. 27, 02 2023

    2023

  11. [17]

    A multiagent cyberbattlesim for rl cyber operation agents,

    T. Kunz, C. Fisher, J. L. Novara-Gsell, C. Nguyen, and L. Li, “A multiagent cyberbattlesim for rl cyber operation agents,” 2023. [Online]. Available: https://arxiv.org/abs/2304.11052

    2023 arXiv

  12. [18]

    Automated cyber defence: A review,

    S. Vyas, J. Hannay, A. Bolton, and P. P. Burnap, “Automated cyber defence: A review,” 2023. [Online]. Available: https://arxiv.org/abs/23 03.04926

    2023

  13. [19]

    Deep reinforcement learning for autonomous cyber defence: A survey,

    G. Palmer, C. Parry, D. J. B. Harrold, and C. Willis, “Deep reinforcement learning for autonomous cyber defence: A survey,” 2024. [Online]. Available: https://arxiv.org/abs/2310.07745

    2024 arXiv

  14. [20]

    The path to autonomous cyber defense,

    S. Oesch, P. Austria, A. Chaulagain, B. Weber, C. Watson, M. Dixson, and A. Sadovnik, “The path to autonomous cyber defense,” 2024. [Online]. Available: https://arxiv.org/abs/2404.10788

    2024 arXiv

  15. [21]

    Autonomous intelligent cyber-defense agent (aica) reference architecture. release 2.0,

    A. Kott, P. Th ´eron, M. Dra ˇsar, E. Dushku, B. LeBlanc, P. Losiewicz, A. Guarino, L. Mancini, A. Panico, M. Pihelgas, K. Rzadca, and F. D. Gaspari, “Autonomous intelligent cyber-defense agent (aica) reference architecture. release 2.0,” 2023. [Online]. Available: https://arx...

    2023 arXiv

  16. [22]

    Multi-agent deep reinforcement learning: a survey,

    S. Gronauer and K. Diepold, “Multi-agent deep reinforcement learning: a survey,”Artificial Intelligence Review, vol. 55, no. 2, 2022

    2022

  17. [23]

    Multi-agent reinforcement learning: A comprehensive survey,

    D. Huh and P. Mohapatra, “Multi-agent reinforcement learning: A comprehensive survey,” 2024. [Online]. Available: https://arxiv.org/abs/ 2312.10256

    2024 arXiv

  18. [24]

    S. V . Albrecht, F. Christianos, and L. Sch¨afer,Multi-Agent Reinforcement Learning: Foundations and Modern Approaches. MIT Press, 2024. [Online]. Available: https://www.marl-book.com

    2024

  19. [25]

    Multi-agent reinforcement learning for cybersecurity: Approaches and challenges,

    S. Finistrella, S. Mariani, and F. Zambonelli, “Multi-agent reinforcement learning for cybersecurity: Approaches and challenges,” 2024

    2024

  20. [26]

    Design and analysis of decentralized interactive cyber defense approach based on multi-agent coordination,

    M. Liu, L. Ma, C. Li, W. Chang, Y . Wang, J. Cui, and Y . Ji, “Design and analysis of decentralized interactive cyber defense approach based on multi-agent coordination,” in2020 16th International Conference on Mobility, Sensing and Networking (MSN), 2020, pp. 659–664

    2020

  21. [27]

    A theory of abstraction in reinforcement learning,

    D. Abel, “A theory of abstraction in reinforcement learning,” 2022. [Online]. Available: https://arxiv.org/abs/2203.00397

    2022 arXiv

  22. [28]

    Near optimal behavior via approximate state abstraction,

    D. Abel, D. E. Hershkowitz, and M. L. Littman, “Near optimal behavior via approximate state abstraction,” 2017. [Online]. Available: https://arxiv.org/abs/1701.04113

    2017 arXiv

  23. [29]

    State abstraction as compression in apprenticeship learning,

    D. Abel, D. Arumugam, K. Asadi, Y . Jinnai, M. L. Littman, and L. L. Wong, “State abstraction as compression in apprenticeship learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, 2019, pp. 3134–3142

    2019

  24. [30]

    Non-cooperative games,

    J. F. Nash, “Non-cooperative games,” inThe Foundations of Price Theory Vol 4. Routledge, 2024, pp. 329–340

    2024

  25. [31]

    Value-decomposition networks for cooperative multi-agent learning,

    P. Sunehag, G. Lever, A. Gruslys, W. M. Czarnecki, V . Zambaldi, M. Jaderberg, M. Lanctot, N. Sonnerat, J. Z. Leibo, K. Tuyls, and T. Graepel, “Value-decomposition networks for cooperative multi-agent learning,” 2017. [Online]. Available: https://arxiv.org/abs/1706.05296

    2017 arXiv

  26. [32]

    State abstractions for lifelong reinforcement learning,

    D. Abel, D. Arumugam, L. Lehnert, and M. Littman, “State abstractions for lifelong reinforcement learning,” inInternational Conference on Machine Learning. PMLR, 2018, pp. 10–19

    2018

  27. [33]

    Value preserving state-action abstractions,

    D. Abel, N. Umbanhowar, K. Khetarpal, D. Arumugam, D. Precup, and M. Littman, “Value preserving state-action abstractions,” inInternational Conference on Artificial Intelligence and Statistics. PMLR, 2020

    2020

  28. [34]

    Reinforcement learning: An introduction,

    R. S. Sutton and A. G. Barto, “Reinforcement learning: An introduction,” IEEE Transactions on Neural Networks, vol. 16, pp. 285–286, 2005. [Online]. Available: https://api.semanticscholar.org/CorpusID:60035920

    2005

  29. [35]

    Shoham and K

    Y . Shoham and K. Leyton-Brown,Multiagent systems: Algorithmic, game-theoretic, and logical foundations. Cambridge Univ. Press, 2008

    2008

  30. [36]

    A game theoretic approach to decision and analysis in network intrusion detection,

    T. Alpcan and T. Basar, “A game theoretic approach to decision and analysis in network intrusion detection,” in42nd IEEE International Conference on Decision and Control, vol. 3, 2003, pp. 2595–2600 V ol.3

    2003

  31. [37]

    Computing optimal randomized resource allocations for massive se- curity games,

    C. Kiekintveld, M. Jain, J. Tsai, J. Pita, F. Ord ´o˜nez, and M. Tambe, “Computing optimal randomized resource allocations for massive se- curity games,” inProceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems - Volume 1, 2009

    2009

  32. [38]

    A review of attacker–defender games and cyber security,

    K. Hausken, J. W. Welburn, and J. Zhuang, “A review of attacker–defender games and cyber security,”Games, vol. 15, no. 4,

  33. [39]

    Dynamic games in cyber-physical security: An overview,

    S. R. Etesami and T. Bas ¸ar, “Dynamic games in cyber-physical security: An overview,”Dynamic Games and Applications, vol. 9, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:127700312

    2019

  34. [40]

    A pomdp approach to the dynamic defense of large-scale cyber networks,

    E. Miehling, M. Rasouli, and D. Teneketzis, “A pomdp approach to the dynamic defense of large-scale cyber networks,”IEEE Transactions on Information Forensics and Security, vol. 13, no. 10, 2018

    2018

  35. [41]

    Yu and R

    L. Yu and R. R. Brooks,Stochastic Tools for Network Intrusion Detection. Springer International Publishing, 2018, p. 197–205. [Online]. Available: http://dx.doi.org/10.1007/978-3-319-75683-7 15

    2018 doi

  36. [42]

    Multiagent cooperation and competition with deep reinforcement learning,

    A. Tampuu, T. Matiisen, D. Kodelja, I. Kuzovkin, K. Korjus, J. Aru, J. Aru, and R. Vicente, “Multiagent cooperation and competition with deep reinforcement learning,” 2015. [Online]. Available: https://arxiv.org/abs/1511.08779

    2015 arXiv

  37. [43]

    Pettingzoo: Gym for multi-agent reinforcement learning,

    J. K. Terry, B. Black, N. Grammel, M. Jayakumar, A. Hari, R. Sullivan, L. Santos, R. Perez, C. Horsch, C. Dieffendahl, N. L. Williams, Y . Lokesh, and P. Ravi, “Pettingzoo: Gym for multi-agent reinforcement learning,” 2021. [Online]. Available: https://arxiv.org/abs/2009.14471

    2021 arXiv

  38. [44]

    Multi- agent actor-critic for mixed cooperative-competitive environments,

    R. Lowe, Y . Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch, “Multi- agent actor-critic for mixed cooperative-competitive environments,”

  39. [45]

    Markov games as a framework for multi- agent reinforcement learning,

    M. L. Littman, “Markov games as a framework for multi- agent reinforcement learning,” inMachine Learning Proceedings 1994, W. W. Cohen and H. Hirsh, Eds. San Francisco (CA): Morgan Kaufmann, 1994, pp. 157–163. [Online]. Available: https: //www.sciencedirect.com/science/article...

    1994

  40. [46]

    Counterfactual multi-agent policy gradients,

    J. Foerster, G. Farquhar, T. Afouras, N. Nardelli, and S. Whiteson, “Counterfactual multi-agent policy gradients,” 2017. [Online]. Available: https://arxiv.org/abs/1705.08926

    2017 arXiv

  41. [47]

    Qmix: Monotonic value function factorisation for deep multi-agent reinforcement learning,

    T. Rashid, M. Samvelyan, C. S. de Witt, G. Farquhar, J. Foerster, and S. Whiteson, “Qmix: Monotonic value function factorisation for deep multi-agent reinforcement learning,” 2018. [Online]. Available: https://arxiv.org/abs/1803.11485

    2018 arXiv

  42. [48]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” 2017. [Online]. Available: https://arxiv.org/abs/1707.06347

    2017 arXiv

  43. [49]

    Is independent learning all you need in the starcraft multi-agent challenge?

    C. S. de Witt, T. Gupta, D. Makoviichuk, V . Makoviychuk, P. H. S. Torr, M. Sun, and S. Whiteson, “Is independent learning all you need in the starcraft multi-agent challenge?” 2020. [Online]. Available: https://arxiv.org/abs/2011.09533

    2020 arXiv

  44. [50]

    The surprising effectiveness of ppo in cooperative, multi-agent games,

    C. Yu, A. Velu, E. Vinitsky, J. Gao, Y . Wang, A. Bayen, and Y . Wu, “The surprising effectiveness of ppo in cooperative, multi-agent games,”

  45. [51]

    Deterministic policy gradient algorithms,

    D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. Riedmiller, “Deterministic policy gradient algorithms,” inProceedings of the 31st International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, E. P. Xing and T. Jebara, Eds., vol. 32,...

    2014

  46. [52]

    Cyberbattlesim,

    W. B. Christian Seifert, Michael Betser, “Cyberbattlesim,” https://gith ub.com/microsoft/cyberbattlesim, 2021

    2021

  47. [54]

    Marllib: A scalable and efficient multi-agent reinforcement learning library,

    S. Hu, Y . Zhong, M. Gao, W. Wang, H. Dong, X. Liang, Z. Li, X. Chang, and Y . Yang, “Marllib: A scalable and efficient multi-agent reinforcement learning library,” 2023. [Online]. Available: https://arxiv.org/abs/2210.13708

    2023 arXiv

  48. [55]

    The starcraft multi-agent challenge,

    M. Samvelyan, T. Rashid, C. S. de Witt, G. Farquhar, N. Nardelli, T. G. J. Rudner, C.-M. Hung, P. H. S. Torr, J. Foerster, and S. Whiteson, “The starcraft multi-agent challenge,” 2019. [Online]. Available: https://arxiv.org/abs/1902.04043

    2019 arXiv

  49. [56]

    Openspiel: A framework for reinforcement learning in games,

    M. Lanctot, E. Lockhart, J.-B. Lespiau, V . Zambaldi, S. Upadhyay, J. P ´erolat, S. Srinivasan, F. Timbers, K. Tuyls, S. Omidshafiei, D. Hennes, D. Morrill, P. Muller, T. Ewalds, R. Faulkner, J. Kram ´ar, B. D. Vylder, B. Saeta, J. Bradbury, D. Ding, S. Borgeaud, M. Lai, J. Sc...

  50. [58]

    Multi-agent reinforcement learning for maritime operational technology cyber security,

    A. Wilson, R. Menzies, N. Morarji, D. Foster, M. C. Mont, E. Turkbeyler, and L. Gralewski, “Multi-agent reinforcement learning for maritime operational technology cyber security,” 2024. [Online]. Available: https://arxiv.org/abs/2401.10149

    2024 arXiv

  51. [59]

    Cyborg++: An enhanced gym for the development of autonomous cyber agents,

    H. Emerson, L. Bates, C. Hicks, and V . Mavroudis, “Cyborg++: An enhanced gym for the development of autonomous cyber agents,” 2024. [Online]. Available: https://arxiv.org/abs/2410.16324

    2024 arXiv

  52. [60]

    Available: https://arxiv.org/abs/1908.09453

    [Online]. Available: https://arxiv.org/abs/1908.09453

    1908 arXiv

  53. [61]

    Exploring the efficacy of multi-agent reinforcement learning for au- tonomous cyber defence: A cage challenge 4 perspective,

    M. Kiely, M. Ahiskali, E. Borde, B. Bowman, D. Bowman, D. van Bruggen, K. Cowan, P. Dasgupta, E. Devendorf, B. Edwardset al., “Exploring the efficacy of multi-agent reinforcement learning for au- tonomous cyber defence: A cage challenge 4 perspective,” inProceed- ings of the A...

    2025

  54. [62]

    Towards an ai-powered player in cyber defence exercises,

    R. Meier, A. Lavrenovs, K. Hein ¨aaro, L. Gambazzi, and V . Lenders, “Towards an ai-powered player in cyber defence exercises,” in2021 13th International Conference on Cyber Conflict (CyCon), 2021

    2021

  55. [63]

    A survey for deep reinforcement learning based network intrusion detection,

    W. Yang, A. Acuto, Y . Zhou, and D. Wojtczak, “A survey for deep reinforcement learning based network intrusion detection,” 2024. [Online]. Available: https://arxiv.org/abs/2410.07612

    2024

  56. [64]

    Cyborg: An autonomous cyber operations research gym,

    C. Baillie, M. Standen, J. Schwartz, M. Docking, D. Bowman, and J. Kim, “Cyborg: An autonomous cyber operations research gym,”

  57. [65]

    Available: https://arxiv.org/abs/2002.10667

    [Online]. Available: https://arxiv.org/abs/2002.10667

    2002 arXiv

  58. [66]

    Csle: Cyber security learning environment,

    K. Hammar, “Csle: Cyber security learning environment,” 2024, gitHub repository. [Online]. Available: https://github.com/Limmen/csle

    2024

  59. [67]

    Combating advanced persistent threats: Challenges and solutions,

    Y . Wang, H. Liu, Z. Li, Z. Su, and J. Li, “Combating advanced persistent threats: Challenges and solutions,”IEEE Network, vol. 38, no. 6, p. 324–333, Nov. 2024. [Online]. Available: http://dx.doi.org/10. 1109/MNET.2024.3389734

    2024

  60. [68]

    Lateral movement (ta0008),

    MITRE ATT&CK, “Lateral movement (ta0008),” https://attack.mitre.o rg/tactics/TA0008/, 2019, accessed: 2024-12-11

    2019

  61. [69]

    Deep q-learning based reinforcement learning approach for network intrusion detection,

    H. Alavizadeh, J. Jang-Jaccard, and H. Alavizadeh, “Deep q-learning based reinforcement learning approach for network intrusion detection,”

  62. [70]

    Exploitation of remote services (t1210),

    ExtraHop, “Exploitation of remote services (t1210),” https://attack.mit re.org/techniques/T1210/, 2022, tactic: Lateral Movement; Version: 1.1; Created: 18 April 2018, Last Modified: 24 February 2022; Accessed: 2024-12-11

    2022

  63. [71]

    Cy- bershield: A competitive simulation environment for training ai in cybersecurity,

    J. A. F. Carrasco, I. A. Pagola, R. O. Urrutia, and R. Rom ´an, “Cy- bershield: A competitive simulation environment for training ai in cybersecurity,” in2024 11th International Conference on Internet of Things: Systems, Management and Security (IOTSMS), 2024, pp. 11– 18

    2024

  64. [72]

    Farsighted risk mitigation of lateral movement using dynamic cognitive honeypots,

    L. Huang and Q. Zhu, “Farsighted risk mitigation of lateral movement using dynamic cognitive honeypots,” 2020. [Online]. Available: https://arxiv.org/abs/2007.13981

    2020 arXiv

  65. [75]

    Os credential dumping (t1003),

    E. Williams, T. SpiderLabs, T. (Wadhwa-)Brown, V . L. Toux, and Y . Yonan, “Os credential dumping (t1003),” https://attack.mitre.org /techniques/T1003/, 2024, tactic: Credential Access; Platforms: Linux, Windows, macOS; Version: 2.2; Created: 31 May 2017, Last Modified: 15 Oct...

    2024

  66. [77]

    Privilege escalation (ta0004),

    MITRE ATT&CK, “Privilege escalation (ta0004),” https://attack.mitre.o rg/tactics/TA0004/, 2021, created: 17 October 2018, Last Modified: 06 January 2021; Accessed: 2024-12-11

    2021

  67. [2020]

    Available: https://arxiv.org/abs/1706.02275

    [Online]. Available: https://arxiv.org/abs/1706.02275

    arXiv

  68. [2021]

    Available: https://arxiv.org/abs/2111.13978

    [Online]. Available: https://arxiv.org/abs/2111.13978

    arXiv

  69. [2022]

    Available: https://arxiv.org/abs/2103.01955

    [Online]. Available: https://arxiv.org/abs/2103.01955

    arXiv

  70. [2024]

    Available: https://www.mdpi.com/2073-4336/15/4/28

    [Online]. Available: https://www.mdpi.com/2073-4336/15/4/28

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.