Pith. sign in

REVIEW 3 major objections 6 minor 22 references

Towards a Multi-Agent Simulation of Cyber-attackers and Cyber-defenders Battles

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read The paper proposes a Dec-POMDP simulation of cyber-attacker and cyber-defender agents acting on a property-based network model, demonstrated on a MITRE ATT&CK GALLIUM scenario with random, decision-tree, and Q-learning agents.

desk verdict A plausible multi-agent cyber simulator, but its only demonstration is hand-crafted, so the central result is internal consistency, not validated behavior. read the letter →

arxiv 2506.04849 v1 pith:V4CZE5FH submitted 2025-06-05 cs.AI

classification cs.AI
keywords cyber-defensemulti-agentsimulationDec-POMDPMITREATT&CKattack-defensetreesQ-learningnetworksecurityGALLIUM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a cyber network can be simulated as a property-based decentralized partially observable Markov decision process (Dec-POMDP) in which attacker and defender agents both act through the same mechanism of preconditions and postconditions. The payoff would be a shared experimental framework: different cyber-defense and cyber-attack strategies could be implemented, trained, and compared on identical network environments instead of in isolated, incompatible simulations. To demonstrate the idea, the authors build a GALLIUM-based scenario from MITRE ATT&CK, define 30 actions with hand-written pre/postconditions, and run random, decision-tree, and Q-learning agents in the resulting simulator. Their reported results show decision-tree defenders blocking decision-tree attackers, and Q-learning defenders progressively suppressing Q-learning attackers. If the model is accepted, the same environment can serve as a testbed for coordinated cyber-defense research.

What carries the argument

The central object is a property-based Dec-POMDP. A state is a set of property couples (identifier, value); an action is a relation mapping a conjunctive precondition subset of properties to a postcondition subset. When an agent plays an action whose precondition holds, the environment deletes any existing properties whose identifiers are overwritten and adds the postcondition properties; the agent then receives observable properties and a reward. This uniform action mechanism lets attackers and defenders be implemented as interchangeable behavior functions. The second load-bearing component is the attack-defense tree built from MITRE ATT&CK tactics, techniques, and mitigations for the GALLIUM scenario, which supplies the 30 actions with their pre/postconditions and the topology that connects outside hosts, a DMZ, and three internal subnets.

What would settle it

Replay the shortest 16-action attacker path from the simulator against a real or emulated version of the same five-subnet topology and compare whether the simulated property changes (new files, privileges, logs) match what actually occurs; any step where the real system does not gain the predicted access shows the hand-written action rules do not encode realistic cyber behavior.

Watch

Extended reading notes

Core claim

The paper's central claim is that a whole networked system—nodes, processes, files, privileges, and agent knowledge—can be represented as a set of property couples (identifier, value), and that every action, offensive or defensive, is a relation from a conjunctive precondition subset of properties to a postcondition subset. Applying an action that satisfies the precondition deletes properties whose identifiers are overwritten and adds the new properties, so the environment state advances deterministically. The agents are behavior functions that pick actions from the observations and rewards they receive; multiple agents play one action per turn in sequential cyclic order, and joint rewards are computed from network metrics at the end of each cycle. The paper instantiates this model with a manually built attack-defense tree for the GALLIUM advanced persistent threat, claims the shortest attacker path uses 16 of the 30 defined actions, and reports that in 1000-episode runs, decision-tree defenders prevent decision-tree attackers from reaching their goal while Q-learning defenders similarly reduce MARL attackers' success.

Load-bearing premise

The results assume that the hand-written preconditions and postconditions of the 30 actions, plus the attack-defense tree connecting them, faithfully represent how a real network reacts; if these rules are arbitrary, the observed defense success is an artifact of the modeler's coding rather than a property of the multi-agent approach.

Editorial extensions

If this is right

  • The same network environment and action set can host any agent implementation, so random, decision-tree, and Q-learning behaviors can be compared on identical ground.
  • A hand-crafted decision-tree defender placed on the web server and database blocks the decision-tree attackers' ultimate goal of installing spyware on the printer server and exfiltrating data from the database.
  • Q-learning attackers with curriculum learning converge to action paths nearly as efficient as the decision-tree reference, and Q-learning defenders then reduce the attackers' goal-reaching rate over episodes.
  • The manual pipeline from an APT's MITRE ATT&CK techniques to an attack-defense tree to simulator actions offers a reusable way to convert public threat intelligence into testable scenarios.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The property-based abstraction could become a shared benchmark interface for cyber-agent research if the action set and properties were standardized; the paper does not yet provide such a standard library, but nothing in the model prevents one.
  • Because the evaluation compares simulator runs against no real network logs or emulation, part of the reported defense success may come from the authors' choice of action rules; an independent ground truth consisting of real or emulated attacks would separate framework capability from scenario hand-coding.
  • The manual attack-defense-tree construction from MITRE ATT&CK could be partly automated with a parser that maps technique identifiers to pre/postcondition templates, a direct extension the paper leaves to future work.
  • The sequential cyclic execution is a simplification of real concurrent network activity; adopting asynchronous multi-action turns could change which defender policies succeed, which the paper itself lists as a limitation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a Dec-POMDP-based model for multi-agent cyber-attack/defense simulation, where network nodes are represented by property sets and actions are defined by precondition/postcondition relations. The authors implement this model in the MCAS simulator and evaluate it on a GALLIUM APT scenario using random, decision-tree, and Q-learning-based agents. The main claimed contribution is a common framework for implementing and evaluating cyber-attacker and cyber-defender agents on the same networked environment.

Significance. If validated, the framework could become a useful testbed for multi-agent cyber-defense research, particularly in the AICA context, and the open-source MCAS simulator is a positive step toward reproducibility. However, the formal model is a straightforward application of Dec-POMDP with deterministic transitions, and the experimental validation is largely qualitative. The reported results do not yet demonstrate either the realism of the scenario or the effectiveness of the defense approaches in a quantitative way. The paper's main value at this stage is as a platform description rather than a validated evaluation study.

major comments (3)
  1. [Section III-B] The transition function is written as a conditional transition probability, but its definition T(s,a,s') = I_P(s'|s,a) is an indicator function, making the environment deterministic. This contradicts the motivation in Sections II and III-A that emphasize uncertainty and stochastic observations; the Dec-POMDP framework is used only for partial observability, not for stochastic transitions. Please either introduce genuine stochasticity (e.g., probabilistic postconditions) or explicitly state that this is a deterministic Dec-POMDP and justify why the deterministic setting still captures the intended cyber-defense problems.
  2. [Section IV-B] The only quantitative result, Figure 4, shows attacker rewards with inactive cyber-defense; the defended-run results are reported only qualitatively as 'We observed the attackers to be unable to reach the ultimate goal' and 'we verified the attackers to be less and less able to reach the ultimate goal.' This does not support the claim that the framework assesses cyber-defenders. Please provide quantitative evidence for the defended conditions—for example, success rates, episode lengths, or reward curves with defenders active, including variance over at least several independent training runs.
  3. [Sections III-C and IV-B] The GALLIUM scenario's 30 actions and all defender mitigations are manually created by decorating the same AD tree used to define attack paths; the defenders' actions are designed to invalidate the very preconditions of the attack techniques. The observed defense success is therefore largely an artifact of construction rather than an independent validation. The limitations in Section V concede the need for automated scenario integration and transfer to emulation or real systems. To strengthen the central claim, the authors should compare against an independent ground truth (e.g., logs or emulation) or at least perform a sensitivity analysis, such as ablating defender actions or testing trained defenders against an unseen attack path.
minor comments (6)
  1. [Section III-B] The use of 'IP' as the name of the indicator function is confusing because 'IP' commonly stands for Internet Protocol; consider renaming it (e.g., 'Ind').
  2. [Figure 1] The written example is internally inconsistent: the sentence 'p15 is not added as id2 is not associated with any value' appears to refer to the wrong identifier, and 'which already respectively define p2 and p4' reverses the mapping (ID 4 defines p4, ID 2 defines p2); please correct the description.
  3. [Section IV-B] The statement that the shortest path has probability (1/30)^16 assumes uniform random selection over all 30 actions regardless of whether preconditions are satisfied; please clarify this as an illustration, not an exact probability.
  4. [Figure 4] The learning curves show no variance or number of seeds; please specify how many independent runs were averaged and include error bars or confidence bands.
  5. [Section II] The statement that few works address both cyber-attackers and cyber-defenders in the same networked environment is too strong given the works cited later (e.g., Kotenko, game-theoretic models, white-hat worm); please temper the related-work positioning.
  6. [Abstract / Section IV] The abstract promises assessing cyber-defenders' 'dynamic organizations,' but the experiments only use fixed teams of two defenders; either evaluate organizational changes or rephrase the claim to match the actual scope.

Circularity Check

2 steps flagged · score 5.0 of 10

Defense-effectiveness result in Section IV-B is built into the hand-authored AD-tree rules and rewards; the Dec-POMDP framework claim retains independent content, so circularity is partial.

  1. self definitional [Section III-C step 4 (rule authoring) → Section IV-B Decision Tree approach paragraph (evaluation)]
    "The decision tree was applied to get a reference when cyber-attackers or cyber-defenders already know the best action to take as the role of each agent is defined by a DT. ... Then we added the defenders DT _def ender1 which has to detect malicious logs on WS and DT _def ender2 which has to use privilege account management or monitor the executed commands and arguments on DB. We observed the attackers to be unable to reach the ultimate goal."

    Section III-C step 4 defines each defense action's pre/postconditions by decorating exactly the attack nodes of the same AD tree that encodes the attack techniques, and Section IV-B then equips the DT defenders with the best counter-actions by definition. Because the transition function is the deterministic indicator T(s,a,s')=I_P(s'|s,a) (Section III-B), the observed outcome — attackers unable to reach the ultimate goal — is a logical entailment of the authors' own rule set: the counter-actions were written to delete precisely the hand-written attacker preconditions.

  2. fitted input called prediction [Section IV-B Multi-Agent Reinforcement Learning paragraph, with reward definition in Section III-B]
    "When adding the defenders M ARL_def ender1 and M ARL_def ender2, we verified the attackers to be less and less able to reach the ultimate goal."

    The MARL defenders learn to maximize R = Eval ∘ Metrics (Section III-B), where the metrics are computed from the same hand-written property transitions and the same MITRE-derived action set used to build the attack scenario, and the environment is fully deterministic (indicator transitions and observations). The verified trend — attackers less and less able to reach the ultimate goal — is forced by the reward construction: it penalizes exactly the goal and step properties that the authors' own defender actions were written to delete. Since the authored rules are the sole source of both attack possibilities and defense countermeasures, the result measures internal consistency of the hand-coded simulator, not a prediction validated against an independent ground truth for GALLIUM.

full rationale

The paper's central contribution — the Dec-POMDP model of Section III and the MCAS simulator — is genuine, self-contained content: the formalism is a standard Dec-POMDP with deterministic indicator transition and observation functions, and no formal claim reduces to a self-citation. The existing self-citations (the AICA project report [13], a co-authored source of background use case, and the MCAS repository [21], the implementation itself) are not load-bearing for the derivation. The circular element is confined to the Section IV-B assessment. There, the defense actions were authored (Section III-C step 4) by decorating the same AD-tree attack nodes with MITRE mitigations, the decision-tree defenders are defined as already knowing the best action, and the deterministic indicator transition entails the reported 'attackers unable to reach the ultimate goal.' The MARL defenders' verified improvement is likewise an optimization outcome inside the author-authored reward and rule set, with no comparison to real network logs, emulation, or any independent representation of GALLIUM. The paper is transparent about this: Section V lists automating scenario integration and transferring simulation results to emulated or real systems as future work, explicitly acknowledging that the validation loop is not closed. Because the framework claim retains independent content while the demonstration of defense effectiveness is substantially built into its inputs, a score of 5 (partial circularity) is appropriate rather than 0 or 8.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

Everything observed in the scenario runs inside a closed world authored by the paper: the topology, the 30 actions, the property pre/post conditions, the AD tree, and the reward metrics. The Q-learning and decision-tree agents operate entirely within this world, and no part of the model is learned from or validated against real network data. The paper's experimental results are therefore properties of the authored model, not of the real cyber conflict.

free parameters (3)
  • Q-learning hyperparameters
    Learning rate, discount factor, exploration schedule, and curriculum learning stages are not reported; these determine the MARL results in Section IV-B but cannot be reproduced from the paper.
  • Reward function weights (Eval)
    The mapping from metrics to attacker/defender rewards is not specified; it directly shapes the learned behaviors.
  • Episode count = 1000
    Arbitrary choice for the experiment, with no sensitivity analysis.
assumptions (3)
  • domain assumption The state of the networked system is fully represented by the set of node properties, including agents' properties.
    Section III-A defines the environment state as sets of properties; states that cannot be expressed as property sets are invisible to the model.
  • ad hoc to paper The hand-defined action preconditions and postconditions correctly capture the effects of the selected MITRE ATT&CK techniques.
    Section IV-B manually creates 30 actions for the GALLIUM scenario; no validation against real systems, logs, or independent attack simulations is provided.
  • domain assumption Sequential turn-taking with one action per agent approximates the concurrent behavior of real attackers and defenders.
    Section III-A adopts the 'Agent Environment Cycle' and explicitly acknowledges it is 'not realistic' as a first approximation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards a Multi-Agent Simulation of Cyber-attackers and Cyber-defenders Battles." pith.science (2026). https://pith.science/paper/V4CZE5FH

@misc{pith2026250604849,
  author       = {Pith},
  title        = {Pith review of: Towards a Multi-Agent Simulation of Cyber-attackers and Cyber-defenders Battles},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V4CZE5FH}},
  note         = {Machine review of arXiv:2506.04849}
}
read the original abstract

As cyber-attacks show to be more and more complex and coordinated, cyber-defenders strategy through multi-agent approaches could be key to tackle against cyber-attacks as close as entry points in a networked system. This paper presents a Markovian modeling and implementation through a simulator of fighting cyber-attacker agents and cyber-defender agents deployed on host network nodes. It aims to provide an experimental framework to implement realistically based coordinated cyber-attack scenarios while assessing cyber-defenders dynamic organizations. We abstracted network nodes by sets of properties including agents' ones. Actions applied by agents model how the network reacts depending in a given state and what properties are to change. Collective choice of the actions brings the whole environment closer or farther from respective cyber-attackers and cyber-defenders goals. Using the simulator, we implemented a realistically inspired scenario with several behavior implementation approaches for cyber-defenders and cyber-attackers.

Figures

Figures reproduced from arXiv: 2506.04849 by the authors.

Figure 1
Figure 1. An illustrative view of the simulation model [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 3
Figure 3. Proposed small-scale company network topology [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 2
Figure 2. An overview of the proposed attack/defense AD Tree [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: An evolution of the rewards average according to [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 19 canonical work pages

  1. [1]

    MITRE ATTA&CK,

    “MITRE ATTA&CK,” https://attack.mitre.org/, accessed: 2023-04-11

  2. [2]

    A graph-based system for network- vulnerability analysis,

    C. Phillips and L. P. Swiler, “A graph-based system for network- vulnerability analysis,” in Proceedings of the 1998 Workshop on New Security Paradigms, 1998

  3. [3]

    Foundations of attack–defense trees,

    Kordy, Barbara et. al., “Foundations of attack–defense trees,” in Formal Aspects of Security and Trust , Berlin, Heidelberg, 2011, pp. 80–95

  4. [4]

    Modeling cyberattacks with extended petri nets,

    Petty, Mikel D et al., “Modeling cyberattacks with extended petri nets,” in Proc. of the 2022 ACM Southeast Conference , 2022

  5. [5]

    Machine learning cyberattack strategies with petri nets with players, strategies, and costs,

    Bland, John A. et al., “Machine learning cyberattack strategies with petri nets with players, strategies, and costs,” in National Cyber Summit (NCS) Research Track, Cham, 2020, pp. 232–247

  6. [6]

    White-hat worm to fight malware and its evaluation by agent-oriented petri nets,

    S. Yamaguchi, “White-hat worm to fight malware and its evaluation by agent-oriented petri nets,” Sensors, vol. 20, no. 2, 2020. [Online]. Available: https://www.mdpi.com/1424-8220/20/2/556

  7. [7]

    A game-theoretical approach to cyber-security of critical infrastructures based on multi-agent reinforcement learning,

    Panfili, Martina et. al., “A game-theoretical approach to cyber-security of critical infrastructures based on multi-agent reinforcement learning,” in 26th Mediterranean Conf. on Control and Automation , 2018

  8. [8]

    A game theoretic approach to model cyber attack and defense strategies,

    Afraa, Attiah et al., “A game theoretic approach to model cyber attack and defense strategies,” in IEEE Conf. on Communications , 2018, pp. 1–7

Show all 22 references
  1. [9]

    A markov game theory-based risk assessment model for network information system,

    Xiaolin, Cui et. al., “A markov game theory-based risk assessment model for network information system,” in 2008 Int. Conf. on Computer Science and Software Engineering , vol. 3, 2008, pp. 1057–1061

  2. [10]

    DEC-MDP / DEC-POMDP,

    Beynier, Aurélie et al., “DEC-MDP / DEC-POMDP,” in Markov Decision Processes in Artificial Intelligence , 2010, pp. 277–313. [Online]. Available: https://hal.science/hal-00969197

  3. [11]

    Pettingzoo: Gym for multi-agent reinforcement learning,

    Terry, J. K et al, “Pettingzoo: Gym for multi-agent reinforcement learning,” 2020

  4. [12]

    The complexity of decentralized control of markov decision processes,

    Daniel S. Bernstein et al., “The complexity of decentralized control of markov decision processes,” CoRR, vol. abs/1301.3836, 2013. [Online]. Available: http://arxiv.org/abs/1301.3836

  5. [13]

    Autonomous Intelli- gent Cyber Defence Agent Prototype 2021 - Project Report,

    P. Theron, N. Evans, M. Drasar, and A. Guarino, “Autonomous Intelli- gent Cyber Defence Agent Prototype 2021 - Project Report,” Dec. 2021

  6. [14]

    F. A. Oliehoek and C. Amato, A Concise Introduction to Decentralized POMDPs, ser. Springer Briefs in Intelligent Systems. Springer, 2016. [Online]. Available: https://doi.org/10.1007/978-3-319-28929-8

  7. [15]

    A master attack methodology for an ai-based automated attack planner for smart cities,

    Falco, Gregory et al., “A master attack methodology for an ai-based automated attack planner for smart cities,” IEEE Access , vol. 6, pp. 48 360–48 373, 2018

  8. [16]

    Agent-based network security simulation,

    Grunewald, Dennis et. al., “Agent-based network security simulation,” vol. 2, 01 2011, pp. 1325–1326

  9. [17]

    Multi-agent modelling and simulation of cyber-attacks and cyber-defense for homeland security,

    I. Kotenko, “Multi-agent modelling and simulation of cyber-attacks and cyber-defense for homeland security,” in 2007 4th IEEE Workshop on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications, 2007, pp. 614–619

  10. [18]

    Varga, OMNeT++

    A. Varga, OMNeT++. Berlin, Heidelberg: Springer Berlin Heidelberg, 2010, pp. 35–59. [Online]. Available: https: //doi.org/10.1007/978-3-642-12331-3_3

  11. [19]

    Session-level adversary intent-driven cyberattack simulator,

    Drasar, Martin et. al., “Session-level adversary intent-driven cyberattack simulator,” in2020 IEEE/ACM 24th Int. Symp. on Distributed Simulation and Real Time Applications , 2020

  12. [20]

    Cyberbattlesim,

    M. D. R. Team., “Cyberbattlesim,” 2021, created by C. Seifert et. al. [Online]. Available: https://github.com/microsoft/cyberbattlesim

  13. [21]

    Multi cyber agent simulator,

    “Multi cyber agent simulator,” https://github.com/julien6/MCAS, ac- cessed: 2023-04-17

  14. [22]

    Technical note: Q-learning,

    C. Watkins and P. Dayan, “Technical note: Q-learning,” Machine Learn- ing, vol. 8, pp. 279–292, 05 1992

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.