CyGym: A Simulation-Based Game-Theoretic Analysis Framework for Cybersecurity

Michael Lanier; Yevgeniy Vorobeychik

REVIEW 3 major objections 6 minor 39 references

CyGym: A Simulation-Based Game-Theoretic Analysis Framework for Cybersecurity

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CyGym pairs a moderate-fidelity network simulator with a formal game model and shows that approximate equilibrium strategies beat simple cyber-defense baselines in a Volt Typhoon scenario.

desk verdict A useful simulator and zero-day model, but the paper does not supply evidence that DOAR actually finds approximate equilibria, and the experiments are too thin to support the case-study conclusions. read the letter →

arxiv 2506.21688 v3 pith:ZBVYKVEP submitted 2025-06-26 cs.CR cs.GT

Michael Lanier , Yevgeniy Vorobeychik This is my paper

classification cs.CRcs.GT

keywords cybersecuritysimulationgametheorypartiallyobservablestochasticgamesdoubleoraclereinforcementlearningzero-dayexploitsadvancedpersistentthreatsnetworkdefense

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes CyGym, a network cyber-encounter simulator built at intermediate fidelity, paired with a formal game model and an equilibrium-solving algorithm. It claims that the combined framework supports game-theoretic analysis of advanced persistent threats, including zero-day exploits, in a way that simple abstract models or pure emulators do not. The central result is a demonstration, on a Volt Typhoon-style scenario, that approximate equilibrium strategies for both attacker and defender outperform random, do-nothing, and preset heuristic baselines. If the framework is right, cyber defenders can use it to reason about when to scan, reset, or do nothing, and how to price the risk of unknown attacks.

What carries the argument

The machinery is a pipeline of three connected pieces. First, CyGym, a simulation environment with hierarchical networks generated by scale-free preferential attachment, per-device workloads, exploits drawn from a public vulnerability database, and defender actions spanning detection, reset, update, and reconfiguration. Second, a formalization of the simulator as a partially observable stochastic game, with zero-days encoded as a commonly known prior over exploit parameters that the attacker observes privately. Third, DOAR (double oracle with actor response ascent), a policy-space response oracle style solver in which best responses are approximated by reinforcement learning using a critic-guided coordinate-ascent beam search to handle the combinatorial action space.

What would settle it

Replace the generated scale-free network with a real enterprise network topology (or run the same equilibrium policies in a higher-fidelity emulation or red-team exercise) and compare defender payoffs against the preset scan-and-reset baseline; if the equilibrium defense no longer outperforms the baseline, the framework's transfer claim fails.

Watch

Extended reading notes

Core claim

The paper's central discovery is that a moderate-fidelity simulator can be turned into a partially observable stochastic game whose approximate Bayes-Nash equilibria, computed by a double-oracle method with reinforcement-learning best responses, dominate simple baselines in the Volt Typhoon case study. It also claims that zero-day exploits are naturally represented as asymmetric information: both players share a prior over possible exploit parameters, the attacker alone knows which one it holds, and once used the exploit becomes known to the defender. In equilibrium, the framework yields qualitative predictions: defenders scan less when productive work is more valuable or defense is costlier, scan less and lose more as networks grow, and benefit from learning which specific zero-day the attacker holds most when the attacker's option set is small.

Load-bearing premise

The load-bearing premise is that the simulator's abstractions—scale-free preferential-attachment network generation, independent random workload tuples, and fixed exploit-success probabilities—faithfully capture the strategic choices that matter in real cyber operations; if real enterprises deviate, the equilibrium insights may not transfer.

Editorial extensions

If this is right

In the Volt Typhoon case study, the DOAR equilibrium profile gives both players higher expected payoff than random, do-nothing, and preset scan-and-reset baselines, which is evidence that the computed profile is an approximate equilibrium of the modeled game.
Equilibrium behavior responds to opportunity cost: the defender scans less when workloads are more valuable and when defensive actions are costlier, even though compromise rates rise.
Larger networks make scanning less cost-effective: equilibrium compromise counts rise, attacker payoff falls due to search cost, and the defender scans less often and ends with a worse payoff.
The value of knowing the attacker's zero-day is largest when the attacker's possible zero-day set is small; with a fixed vulnerability pool, a larger option set favors the defender, while a vulnerability pool that grows with the option set favors the attacker.
Because zero-day exploits become known to the defender once used, the framework turns 'unknown unknowns' into one-shot private information, allowing defense analysis without assuming the defender knows the attacker's full action set.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the simulator's abstractions transfer, the same pipeline could be configured for other APT profiles by swapping topology, workload, and exploit parameters; the case study is one instantiation of a general procedure.
The result that higher workload value reduces defensive scanning suggests an economic tradeoff that security operations centers could test directly: accept higher compromise risk to protect productivity.
The critic-guided beam search is a generic fix for combinatorial action decoding in reinforcement learning, so it could improve best-response computation in other games with large joint action spaces, not just cyber defense.
The zero-day knowledge result implies a concrete intelligence recommendation: prioritize learning which specific zero-day an adversary holds when the adversary's exploit option set is small.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The paper is a mixed bag. The CyGym simulator and the game model built on it are worth a look; the zero-day modeling via a common-knowledge distribution over exploit types (with the attacker knowing its draw) is a clean way to get at asymmetric information without pretending you know the strategy space. The DOAR algorithm, critic-guided coordinate-ascent beam search for combinatorial action spaces, is a reasonable engineering idea, and the paper ships code and data. That's real value.

What is not in the paper is any evidence for the central claim that DOAR's joint profile approximates a Bayes-Nash equilibrium of the POSG. The results in Section 6.1 show DOAR outperforms a few hand-picked baselines, and the text says this 'is simply a confirmation' that the profile is an approximate equilibrium. That's wrong as an inference. In a general-sum game, beating RandomInit, No Defense, and a preset heuristic says nothing about whether each player is best-responding to the other. The paper gives no epsilon, no exploitability, no NashConv, no fresh best response check. The stress-test note is right: the subsequent qualitative conclusions about zero-days (Figure 6) ride on an equilibrium concept that has not been validated.

The experimental reporting is also thin. The near-zero standard deviations in Table 2 look like the DOAR rows are deterministic, but there is no seed count or indication of how many runs produced the mean. There is no comparison to a standard PSRO or independent RL solver, so we don't know whether DOAR is actually better than what exists, or only better than 'do nothing.' And the fidelity assumptions—Barabási-Albert graphs, independent workload tuples, hand-set utilities—are heavy and unvalidated. The authors are up front about some of these abstractions, which I credit, but it means the paper is a demonstration of the framework, not yet a source of reliable defensive guidance for Volt Typhoon or any other real APT.

Where I'd land: this is a legitimate framework paper that deserves reviewer time, but a referee should push hard on the equilibrium verification. The modeling of zero-days and the simulator itself are the strongest parts. The DOAR algorithm is plausible but needs a proper PSRO baseline and an exploitability-style check before the equilibrium claim can be taken seriously. The paper is honest about the beam-search limitation (it can miss synergistic multi-device actions), which is a nice touch.

Recommend sending it out, with a request for major revision rather than rejection out of hand. The authors have the pieces; they need to supply the equilibrium evidence and sharper experiments.

Referee Report

3 major / 6 minor

Summary. The paper introduces CyGym, a Gym-based cybersecurity simulation environment with networked devices, workloads, NVD-sourced vulnerabilities and exploits, defensive actions, and a zero-day model based on asymmetric information. It formalizes the resulting interaction as a partially observable stochastic game (POSG) and proposes DOAR, a double-oracle/PSRO-style algorithm that uses critic-guided coordinate-ascent beam search to approximate best responses in combinatorial action spaces. The authors instantiate the framework on a Volt Typhoon scenario and report that DOAR outperforms heuristic baselines, followed by equilibrium-based sensitivity studies of workload value, defense cost, network size, and zero-day availability. The GitHub repository is provided.

Significance. If the equilibrium claims are substantiated, CyGym would be a useful, reusable tool for game-theoretic security analysis, and the treatment of zero-days as asymmetric information over a common-knowledge exploit distribution is a genuine modeling contribution. The use of NVD data and the public release of the simulator strengthen reproducibility. The Volt Typhoon case study offers interesting qualitative tradeoffs, but the paper's quantitative conclusions currently rest on an unverified equilibrium claim; the experiments also lack independent solver comparisons and explicit seed counts. These issues are fixable with additional evaluation, so the manuscript is worth a major revision rather than rejection.

major comments (3)

[Section 6.1, Tables 2-3, Fig. 2b] The central claim in the abstract and Section 1 that DOAR finds approximate equilibria is not supported by the reported experiments. The text in Section 6.1 says that DOAR's consistently outperforming baselines 'is simply a confirmation that the joint strategy profile obtained by DOAR is indeed an approximate equilibrium'; in a general-sum POSG this inference is invalid, because beating a small set of hand-picked baselines is neither necessary nor sufficient for an epsilon-BNE. The paper provides no exploitability or NashConv estimate, no epsilon value, no independent best-response check against the opponent's DOAR policy, and no comparison to a standard PSRO or independent RL solver. This is especially important because the DOAR best-response oracle is an approximate RL policy and Algorithm 1's merge step is explicitly acknowledged to 'miss synergistic, multi-dimensional actions' (Section 5), so the equilibrium property is not established by construction. Please add an explicit equilibrium-quality evaluation (for example, a best-response tournament, NashConv/exploitability estimate, or one-step deviation analysis) for both the game in Table 2 and the Bayes game in Table 3, and report the number of seeds; the DOAR rows report standard deviations of 0.000-0.002 while baseline rows report 35-63, which is difficult to interpret without knowing the number of independent runs.
[Section 6, reward shaping paragraph] The statement that the attacker's shaping term 'does not affect the long-run equilibrium, since such potential-based shaping is provably policy-invariant [21]' is not justified. Reference [21] (Ng et al. 1999) proves policy invariance for a single-agent MDP; the present setting is a finite-horizon (T=30) stochastic game with gamma=0.99, and with a nonzero terminal potential the telescoped sum includes a policy-dependent term gamma^T phi(s_T). Furthermore, in a two-player general-sum game, adding a shaping bonus to one player's reward can change the set of Nash equilibria unless the shaping is shown to preserve both players' best-response correspondences and the equilibrium of the underlying game. The caption 'Results reported without the reward shaping bonus' (Table 2) does not address this, since the policies were trained on the shaped rewards. Please provide a proof of equilibrium invariance for this POSG, or set beta_A=0 and verify that the reported equilibrium profiles and payoffs are unchanged.
[Sections 3.2, 3.3, and 6] The paper's stated goal is to provide 'valuable insight into optimal defensive posture' for real APTs, but all conclusions are derived from a single set of abstraction choices: Barabasi-Albert preferential attachment for network generation and independent workload tuples with independently distributed requirements, durations, and values. These choices determine the entire payoff structure, and Figures 3-6 are therefore conditional on them. The paper does not validate these abstractions against any real enterprise topology or workload trace, and it does not include a sensitivity analysis over the topology-generation parameters (for example, attachment exponent, lambda, p_add). Please add a limitations paragraph that states explicitly that the Volt Typhoon conclusions are insights about the simulator's model, not direct empirical findings about real networks, or add validation and sensitivity experiments.

minor comments (6)

[Section 4.5] In the definition after Eq. (5), U^beta is written as U^beta(sigma^alpha,sigma^beta;z), but the defender does not observe z in the zero-day model of Section 4.4; the expression should be the z-expectation U^beta(sigma^alpha,sigma^beta)=E_z[U^beta(pi^alpha,pi^beta)]. The current notation makes it appear that the defender's utility is z-dependent.
[References] Reference [29] cites 'Simulacra, J., Roboticus, J.', which appears to be a placeholder name rather than a real author list. Please replace it with the correct citation.
[Algorithm 1] The merge step can produce joint actions that were not evaluated by Q_phi at the joint level, because t* and p* are sampled from per-device candidate sets and e* is formed as a union; the returned action is therefore not necessarily a best response or even a feasible joint action. Please state how the environment validates the merged action and what happens if it is infeasible.
[Tables 2 and 3] Please report the number of independent runs and confidence intervals in addition to means and standard deviations; the near-zero variance in the DOAR rows makes the comparison with baseline rows difficult to interpret.
[Figure 2a and Section 5] The motivating comparison of beam search with DDPG is performed on a toy 2D Q-surface rather than on the full game; a sentence clarifying that this is illustrative would avoid over-interpretation.
[Section 6, Impact of System Size] The cross-references to 'Figure 4' in the system-size paragraph appear incorrect; the relevant figure for system size is Figure 5, while Figure 4 shows defensive costs.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the DOAR equilibrium claim is unsupported by the reported experiments, but it does not reduce to the paper's definitions or to any fitted input.

full rationale

CyGym's derivation chain is self-contained: payoffs are computed from its own simulator using the reward definitions in Section 4.3 and Table 1, zero-days are modeled as a common-knowledge prior over exploit parameters with an asymmetric action set (Section 4.4), and the epsilon-Bayes-Nash equilibrium definition in Section 4.5 is the standard condition. The DOAR algorithm is an approximate double-oracle/PSRO method whose output is not by construction equal to an equilibrium, because Algorithm 1 explicitly returns only a 'better response' and is acknowledged to miss synergistic multi-device actions (Section 5). The statement in Section 6.1 that DOAR outperforming baselines 'is simply a confirmation that the joint strategy profile obtained by DOAR is indeed an approximate equilibrium' is a validation gap: beating three hand-picked baselines is neither necessary nor sufficient for the epsilon-BNE condition, and no exploitability, NashConv, or fresh best-response check is reported. That is an unsupported inference about the solution concept, not circularity, because the claim does not reduce to the paper's own definitions or to a statistically forced fitted parameter. Load-bearing citations such as [3], [12], [4], and [21] are external standard results, and the authors' prior works appear only in related-work context and are not load-bearing. Therefore no circular step is identified.

Assumptions & free parameters 6 free parameters · 5 assumptions · 2 invented entities

The central claim rests on many hand-set simulation parameters and domain assumptions about network structure, workload behavior, and zero-day prior knowledge. These are not derived from empirical data but are chosen by the authors, which limits the external validity of the conclusions.

free parameters (6)

Initial Compromised Ratio = 0.4
Hand-set simulation parameter that determines the fraction of devices already compromised at game start; affects all results.
Number of attacker-owned nodes = 5
Hand-set constant representing persistent adversary footholds; affects network connectivity and compromise dynamics.
Workload value vw = 1.0 (base), 0.01 (zero-day)
Hand-set reward scale for completed workloads; directly sets defender utility and the tradeoff between work and security.
Action utilities (Clean +0.30, Checkpoint -0.50, Restore -1.00, etc.) = See Table 1
Hand-tuned payoff values for defender and attacker actions; these determine the equilibrium structure and are not derived from data.
Reward shaping weight beta_A = Small (not specified)
Chosen to accelerate learning; authors state it does not affect equilibrium, but the specific value influences the learned policies.
DOAR hyperparameters (greedy-K, soft-tau, learning rates) = See Table 4
Algorithmic hyperparameters selected by the authors; no sensitivity analysis is provided.

assumptions (5)

domain assumption Common knowledge prior distribution Dz over zero-day exploit parameters (Section 4.4)
The game-theoretic model assumes both players share a common prior over the space of possible zero-day exploits, which is central to the Bayesian POSG but may not hold in practice.
domain assumption Barabási-Albert model generates realistic enterprise networks (Section 3.2)
The simulator uses preferential attachment to create network topologies; if real enterprise networks are not scale-free, the results may not generalize.
domain assumption Workload model with independent distributions abstracts productivity (Section 3.3)
The simplification of workloads to independent tuples (requirements, duration, value) may not capture the complexity of real organizational productivity.
standard math Potential-based reward shaping does not alter equilibrium (Section 6, based on Ng et al.)
The authors rely on the policy-invariance theorem for potential-based shaping; this is a sound mathematical result, but the specific shaping function is an ad hoc choice.
domain assumption Simultaneous moves with attacker-first execution (Section 4)
The game model assumes players act simultaneously but the next state and rewards implement the attacker's action before the defender's; this ordering affects the interaction.

invented entities (2)

Zero-day type z with distribution Dz
purpose: Model asymmetric information about the attacker's secret exploit capabilities
Conceptual construct with no direct empirical handle; its validity rests on the realism of the prior distribution.
Attacker-owned nodes
purpose: Represent persistent adversary footholds outside the defender's control
Modeling device to capture pre-existing compromise; not independently verified against real-world data.

how reviews work

0 comments

Cite this review

Pith. "Pith review of CyGym: A Simulation-Based Game-Theoretic Analysis Framework for Cybersecurity." pith.science (2026). https://pith.science/paper/ZBVYKVEP

@misc{pith2026250621688,
  author       = {Pith},
  title        = {Pith review of: CyGym: A Simulation-Based Game-Theoretic Analysis Framework for Cybersecurity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZBVYKVEP}},
  note         = {Machine review of arXiv:2506.21688}
}

read the original abstract

We introduce a novel cybersecurity encounter simulator between a network defender and an attacker designed to facilitate game-theoretic modeling and analysis while maintaining many significant features of real cyber defense. Our simulator, built within the OpenAI Gym framework, incorporates realistic network topologies, vulnerabilities, exploits (including-zero-days), and defensive mechanisms. Additionally, we provide a formal simulation-based game-theoretic model of cyberdefense using this simulator, which features a novel approach to modeling zero-days exploits, and a PSRO-style approach for approximately computing equilibria in this game. We use our simulator and associated game-theoretic framework to analyze the Volt Typhoon advanced persistent threat (APT). Volt Typhoon represents a sophisticated cyber attack strategy employed by state-sponsored actors, characterized by stealthy, prolonged infiltration and exploitation of network vulnerabilities. Our experimental results demonstrate the efficacy of game-theoretic strategies in understanding network resilience against APTs and zero-days, such as Volt Typhoon, providing valuable insight into optimal defensive posture and proactive threat mitigation.

Figures

Figures reproduced from arXiv: 2506.21688 by the authors.

**Figure 2.** (a) Beam search vs. DDPG on a noisy 2D Q-surface. (b) DOAR average equilibrium payoff over [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗

**Figure 3.** Behavior of DOAR as a function of workload value. (a) Average number of scans performed. (b) [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Behavior of DOAR as a function of defensive costs. (a) Average number of scans performed. (b) [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]

**Figure 5.** Figure 5: Behavior of DOAR as a function of network size (all metrics per device). (a) Attacker payoff. (b) [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]

**Figure 6.** Figure 6: Comparison of key DOAR metrics versus |Dz| under both vulnerability models. Top row: fixedvulnerability. Bottom row: submartingale-vulnerability. 7 Conclusion Herein, we have developed a comprehensive game-theoretic framework to address the challenges posed by APTs su…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

39 extracted references · 36 canonical work pages

[21]

In: International Conference on Machine Learning (1999)

Ng, A., Harada, D., Russell, S.J.: Policy invariance under reward transformations: Theory and application to reward shaping. In: International Conference on Machine Learning (1999)

1999
[1]

Improving Homeland Security Decisions2, 485 (2017)

An, B., Tambe, M., Sinha, A.: Stackelberg security games (ssg) basics and application overview. Improving Homeland Security Decisions2, 485 (2017)

work page 2017
[2]

Banik, S., Bopardikar, S.D., Hovakimyan, N.: Flipdyn in graphs: Resource takeover games in graphs (2024), https://arxiv.org/abs/2406.16812

work page Pith review arXiv 2024
[3]

science 286, 509-512

Barabasi, A.L., Albert, R.: Albert, r.: Emergence of scaling in random networks. science 286, 509-512. Science (New York, N.Y.)286, 509–12 (11 1999)

work page 1999
[4]

arXiv preprint arXiv:2403.02227 (2024)

Bighashdel, A., Wang, Y., McAleer, S., Savani, R., Oliehoek, F.A.: Policy space response oracles: A survey. arXiv preprint arXiv:2403.02227 (2024)

arXiv 2024
[5]

Cybersecurity, (CISA), I.S.A.: Cisa and partners release advisory on prc-sponsored volt typhoon activity and supplemental living off the land guidance (2024),https://tinyurl.com/54uw3mre, accessed: 2024-06-05

work page 2024
[6]

In: Proceedings of the 2013 ACM SIGSAC Conference on Computer & Communications Security

van Dijk, M., Juels, A., Oprea, A., Rivest, R.L.: Flipit: The game of ’stealthy takeovers’. In: Proceedings of the 2013 ACM SIGSAC Conference on Computer & Communications Security. pp. 2–13 (2013)

work page 2013
[7]

In: IJCAI

Durkota, K., Lisy, V., Bošansky, B., Kiekintveld, C.: Optimal network security hardening using attack graph games. In: IJCAI. pp. 7–14 (2015)

work page 2015

Show all 39 references

[8]

arXiv preprint arXiv:1905.07059 (2019)

Gardner, C., Waliga, A., Thaw, D., Churchman, S.: Using camouflaged cyber simulations as a model to ensure validity in cybersecurity experimentation. arXiv preprint arXiv:1905.07059 (2019)

2019 arXiv
[9]

Decision Support Systems59, 93–102 (2014)

Holm, H., Sommestad, T., Almgren, M., Persson, M., Axelsson, J.: Cysemol: A tool for cyber security analysis of enterprises. Decision Support Systems59, 93–102 (2014)

2014
[10]

Networks: An International Journal40(2), 97–111 (2002)

Israeli, E., Wood, R.K.: Shortest-path network interdiction. Networks: An International Journal40(2), 97–111 (2002)

2002
[11]

In: AAAI conference on artificial intelligence

Jain, M., Kardes, E., Kiekintveld, C., Ordónez, F., Tambe, M.: Security games with arbitrary schedules: A branch and price approach. In: AAAI conference on artificial intelligence. pp. 792–797 (2010)

2010
[12]

In: Proceedings of the 10th International Conference on Autonomous Agents and Multiagent Systems-Volume 1

Jain, M., Korzhyk, D., Vanek, O., Conitzer, V., Pechoucek, M., Tambe, M.: Double oracle algorithm for zero- sum security games on graphs. In: Proceedings of the 10th International Conference on Autonomous Agents and Multiagent Systems-Volume 1. pp. 327–334 (2011)

2011
[13]

In: International Conference on Autonomous Agents and Multiagent Systems

Kiekintveld, C., Jain, M., Tsai, J., Pita, J., Ordóñez, F., Tambe, M.: Computing optimal randomized resource allocations for massive security games. In: International Conference on Autonomous Agents and Multiagent Systems. p. 689–696 (2009) 20 Michael Lanier and Yevgeniy Vorobeychik

2009
[14]

Advances in neural information processing systems 30(2017)

Lanctot, M., Zambaldi, V., Gruslys, A., Lazaridou, A., Tuyls, K., Pérolat, J., Silver, D., Graepel, T.: A unified game-theoretic approach to multiagent reinforcement learning. Advances in neural information processing systems 30(2017)

2017
[15]

ACM Computing Surveys47(2), 1–38 (2014)

Laszka, A., Felegyhazi, M., Buttyán, L.: A survey of interdependent information security games. ACM Computing Surveys47(2), 1–38 (2014)

2014
[16]

In: AAMAS

Letchford, J., Vorobeychik, Y.: Optimal interdiction of attack plans. In: AAMAS. pp. 199–206 (2013)

2013
[17]

Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D.: Continuous control with deep reinforcement learning (2019)

2019
[18]

IEEE International Conference on Data Mining pp

Liu, F.T., Ting, K.M., Zhou, Z.H.: Isolation forest. IEEE International Conference on Data Mining pp. 413–422 (2008)

2008
[19]

In: IEEE International Carnahan Conference on Security Technology

McInerney, J., Stubberud, S., Anwar, S., Hamilton, S.: Friars: a feedback control system for information assurance using a markov decision process. In: IEEE International Carnahan Conference on Security Technology. pp. 223– 228 (2001)

2001
[20]

Microsoft Research: Cyberbattlesim: A research simulation environment to investigate automated attack and de- fense strategies.https://www.microsoft.com/en-us/research/project/cyberbattlesim/(2021), [Online; ac- cessed 2025-02-06]

2021
[22]

In: IEEE International Con- ference on Communications

Nguyen, K.C., Alpcan, T., Basar, T.: Security games with incomplete information. In: IEEE International Con- ference on Communications. pp. 1–6 (2009)

2009
[23]

In: AAAI Conference on Artificial Intelligence

Nguyen, T., Yang, R., Azaria, A., Kraus, S., Tambe, M.: Analyzing the effectiveness of adversary modeling in security games. In: AAAI Conference on Artificial Intelligence. pp. 718–724 (2013)

2013
[24]

IEEE Conference on Communications and Network Security pp

Nguyen, T.H., Reddi, S., Zheng, B., Bertino, E.: Deep reinforcement learning for cyber security in software-defined networks. IEEE Conference on Communications and Network Security pp. 1–5 (2018)

2018
[25]

Panda,S.,Vorobeychik,Y.:Near-optimalinterdictionoffactoredmdps.In:ConferenceonUncertaintyinArtificial Intelligence (2017)

2017
[26]

In: International Joint Conference on Artificial Intelligence (2018)

Panda, S., Vorobeychik, Y.: Scalable initial state interdiction for factored mdps. In: International Joint Conference on Artificial Intelligence (2018)

2018
[27]

IEEE Transactions on power systems24(1), 96–104 (2009)

Salmeron, J., Wood, K., Baldick, R.: Worst-case interdiction analysis of large-scale electric power grids. IEEE Transactions on power systems24(1), 96–104 (2009)

2009
[28]

In: Joint Workshop on Cyber- Physical Security and Resilience in Smart Grids (10 2016)

Sanjab, A., Saad, W.: On bounded rationality in cyber-physical systems security: Game-theoretic analysis with application to smart grid protection. In: Joint Workshop on Cyber- Physical Security and Resilience in Smart Grids (10 2016)

2016
[29]

Journal of Cybersecurity (2019)

Simulacra, J., Roboticus, J.: Automated adversary emulation: A case for planning and acting with unknowns. Journal of Cybersecurity (2019)

2019
[30]

In: International Joint Conference on Artificial Intelligence

Sinha, A., Fang, F., An, B., Kiekintveld, C., Tambe, M.: Stackelberg security games: Looking beyond a decade of success. In: International Joint Conference on Artificial Intelligence. pp. 5494–5501 (2018)

2018
[31]

arXiv preprint arXiv:2108.09118 (2021)

Standen, M., Lucas, M., Bowman, D., Richer, T.J., Kim, J., Marriott, D.: Cyborg: A gym for the development of autonomous cyber agents. arXiv preprint arXiv:2108.09118 (2021)

2021 arXiv
[32]

The ns-3 Project: ns-3 Network Simulator (2024),https://www.nsnam.org/, version 3.35

2024
[33]

In: AAAI Conference on Artificial Intelligence

Tong, L., Laszka, A., Yan, C., Zhang, N., Vorobeychik, Y.: Finding needles in a moving haystack: Prioritizing alerts with adversarial reinforcement learning. In: AAAI Conference on Artificial Intelligence. pp. 946–953 (2020)

2020
[34]

and international partners publish cybersecurity advisory on people’s republic of china state- sponsored hacking of u.s

(TSA), T.S.A.: U.s. and international partners publish cybersecurity advisory on people’s republic of china state- sponsored hacking of u.s. critical infrastructure (2024),https://www.tsa.gov/news/press/releases/2024/02/ 07/us-and-international-partners-publish-cybersecurity-a...

2024
[35]

Sensors20, 7148 (12 2020)

Ukwandu, E., Farah, M., Hindy, H., Brosset, D., Kavallieros, D., Tachtatzis, C., Bures, M., Andonovic, I., Bellekens, X.: A review of cyber-ranges and test-beds: Current and future trends. Sensors20, 7148 (12 2020)

2020
[36]

arXiv preprint arXiv:2405.01046 (2024)

Vartiainen, T., Dang, D., Mekkanen, M., Anti, E.: Development of cybersecurity simulator-based platform for the protection of critical infrastructures. arXiv preprint arXiv:2405.01046 (2024)

2024 arXiv
[37]

Adaptive Autonomous Secure Cyber Systems pp

Vorobeychik, Y., Pritchard, M.: Plan interdiction games. Adaptive Autonomous Secure Cyber Systems pp. 159– 182 (2020)

2020
[38]

Xiong, J., Wang, Q., Yang, Z., Sun, P., Han, L., Zheng, Y., Fu, H., Zhang, T., Liu, J., Liu, H.: Parametrized deep q-networks learning: Reinforcement learning with discrete-continuous hybrid action space (2018)

2018
[39]

CoRRabs/1103.2491(2011),http://arxiv.org/abs/1103.2491

Zhu, Q., Tembine, H., Basar, T.: Heterogeneous learning in zero-sum stochastic games with incomplete informa- tion. CoRRabs/1103.2491(2011),http://arxiv.org/abs/1103.2491

2011 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [21]

In: International Conference on Machine Learning (1999)

Ng, A., Harada, D., Russell, S.J.: Policy invariance under reward transformations: Theory and application to reward shaping. In: International Conference on Machine Learning (1999)

1999

[2] [1]

Improving Homeland Security Decisions2, 485 (2017)

An, B., Tambe, M., Sinha, A.: Stackelberg security games (ssg) basics and application overview. Improving Homeland Security Decisions2, 485 (2017)

work page 2017

[3] [2]

Banik, S., Bopardikar, S.D., Hovakimyan, N.: Flipdyn in graphs: Resource takeover games in graphs (2024), https://arxiv.org/abs/2406.16812

work page Pith review arXiv 2024

[4] [3]

science 286, 509-512

Barabasi, A.L., Albert, R.: Albert, r.: Emergence of scaling in random networks. science 286, 509-512. Science (New York, N.Y.)286, 509–12 (11 1999)

work page 1999

[5] [4]

arXiv preprint arXiv:2403.02227 (2024)

Bighashdel, A., Wang, Y., McAleer, S., Savani, R., Oliehoek, F.A.: Policy space response oracles: A survey. arXiv preprint arXiv:2403.02227 (2024)

arXiv 2024

[6] [5]

Cybersecurity, (CISA), I.S.A.: Cisa and partners release advisory on prc-sponsored volt typhoon activity and supplemental living off the land guidance (2024),https://tinyurl.com/54uw3mre, accessed: 2024-06-05

work page 2024

[7] [6]

In: Proceedings of the 2013 ACM SIGSAC Conference on Computer & Communications Security

van Dijk, M., Juels, A., Oprea, A., Rivest, R.L.: Flipit: The game of ’stealthy takeovers’. In: Proceedings of the 2013 ACM SIGSAC Conference on Computer & Communications Security. pp. 2–13 (2013)

work page 2013

[8] [7]

In: IJCAI

Durkota, K., Lisy, V., Bošansky, B., Kiekintveld, C.: Optimal network security hardening using attack graph games. In: IJCAI. pp. 7–14 (2015)

work page 2015