Nash Q-Network for Multi-Agent Cybersecurity Simulation

Edward Koh; Peter Chin; Qintong Xie; Xavier Cadet

REVIEW 3 major objections 5 minor 24 references

Nash Q-Network for Multi-Agent Cybersecurity Simulation

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A Nash Q-Network—a centralized critic whose joint-action Q-matrix yields stage-game Nash equilibria used as policy targets—converges to a steady equilibrium and improves Blue's defensive performance in the CybORG CC2 cyber simulation.

desk verdict A credible algorithmic variant with a load-bearing gap between the Nash rhetoric and the actual learning rule, plus experiments too thin to support the claims. read the letter →

arxiv 2509.00678 v1 pith:K4UE4HSJ submitted 2025-08-31 cs.MA cs.GT

Qintong Xie , Edward Koh , Xavier Cadet , Peter Chin This is my paper

classification cs.MAcs.GT

keywords multi-agentreinforcementlearningNashQ-learningcybersecurityCybORGzero-sumMarkovgamepolicy-basedequilibriumautonomouscyberdefense

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a multi-agent reinforcement learning algorithm can converge onto stable, equilibrium-based policies in a complex adversarial cyber setting. The proposed Nash Q-Network adds a centralized critic that outputs a Q-value matrix over both agents' joint actions; from that matrix the algorithm computes a stage-game Nash equilibrium and updates each agent's policy to match it. A practical twist is that the critic's temporal-difference targets average over the agents' current policies rather than over equilibrium policies, which the authors argue reduces variance and stabilizes training. Trained in the CybORG CC2 environment against an aggressive B-line Red adversary, the resulting Blue agent improves its cumulative reward over training and settles into action preferences dominated by Restore and Analyse. If the claim holds, Nash-style equilibrium reasoning can be made to work at scale in partially observable, stochastic cybersecurity games.

What carries the argument

The load-bearing object is the joint Q-matrix Q_phi(s_B, s_R) in R^{|A_B| × |A_R|}, which turns each observed state into a finite two-player zero-sum stage game. Solving that stage game for a Nash equilibrium gives a mixed-strategy profile (σ_B, σ_R), and the policy update minimizes cross-entropy between the agent's softmax policy and that equilibrium profile. The critic is trained separately using the expected-policy TD target y_t = r_t + γ E_{π_B,π_R}[Q_phi(s'_B,s'_R,a'_B,a'_R)], which the paper introduces to avoid high-variance equilibrium bootstrapping. Decoupling these two updates—critic from policy, policy from critic—is the mechanism the paper credits with taming non-stationarity.

What would settle it

Train two otherwise identical Nash Q-Networks, one with the paper's expected-policy TD target and one bootstrapping the target from the stage-game Nash value (the minimax operator for this zero-sum game), and compare final performance against B-line Red on the same seeds. If the minimax-bootstrapped agent matches or exceeds the policy-bootstrapped agent, the claimed equilibrium targets are critic artifacts. A complementary check: compute the exploitability of the finished Blue policy against a best-response Red; a policy far from equilibrium would be beaten much more badly than B-line Red beat

Watch

Extended reading notes

Core claim

The central claim is that the Nash Q-Network learns Nash-optimal strategies in a two-player zero-sum Markov game by separating the critic's job from the policy's job. A centralized critic Q_phi(s_B,s_R,a_B,a_R) estimates expected returns for every joint action pair, so each observed state yields a payoff matrix for the stage game. The algorithm solves that matrix for a Nash equilibrium and updates the Blue and Red policy networks to minimize cross-entropy with the equilibrium mixed strategies, while the critic itself is updated with TD targets built from an expectation over the agents' current stochastic policies. On CybORG CC2, the trained Blue agent shows steadily rising mean episode rewar

Load-bearing premise

The whole equilibrium loop rests on the assumption that bootstrapping the critic's Q-values from an expectation over the agents' current policies—instead of from the equilibrium value—still produces Q-values whose stage-game Nash equilibria are valid targets for the stochastic game.

Editorial extensions

If this is right

In CybORG CC2, the Nash Blue agent's mean episode reward increases over roughly a thousand training epochs and stabilizes, indicating the equilibrium-alignment loss is a usable learning signal in a partially observable cyber environment.
The learned Blue policy concentrates probability on Restore and Analyse actions while largely abandoning Decoy, suggesting the stage-game equilibria in CC2 favor recovery and information gathering over deceptive placement.
Parallel rollout collection with multiple environment instances is compatible with the Nash critic/policy separation, so the method can leverage distributed computation without destabilizing equilibrium updates.
Because the critic outputs full joint-action values, the same mechanism can in principle be applied to any discrete-action zero-sum cyber scenario where a payoff matrix can be formed, not just the specific CC2 layout tested.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's 'Nash' targets are only as valid as the critic that produces them: since the critic bootstraps from current policies rather than from equilibrium values, a promising direct test is to replace the expected-policy target with a minimax (Nash-value) target and compare stability and final performance.
The comparison against a 'heuristic agent baseline' is not detailed; a fair reader would want the same Nash Blue evaluated against fixed Red policies of varying strength and against an independent PPO learner to see how much of the gain comes from equilibrium alignment rather than from policy-gradient training generally.
In multi-adversarial extensions the joint action matrix grows exponentially, so applying the same idea to more than two agents would likely require factored critics or hierarchical equilibrium computation rather than a simple scaling of this architecture.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The Nash Q-Network is a legitimate incremental variant: it separates policy and critic updates, computes stage-game Nash equilibria of a centralized Q-matrix, and distills those into policies via cross-entropy. That combination is not exactly in the cited Nash Q-learning or Nash-DQN work, so there is a modest new algorithmic idea here. The CybORG CC2 testbed is also a real, non-trivial environment, and the authors were honest enough to state in Section 5.4.2 that they bootstrap from the current policies rather than equilibrium targets.

The problem is that the central claim outruns the evidence. The critic in Eq. (2)-(3) learns a value function for the current joint policy, not the Nash-equilibrium value from Eq. (1). No theorem, contraction argument, or policy-iteration justification connects those two. The stage-game Nash equilibria are then computed from a critic that may be evaluating the wrong continuation values, so the self-referential loop has no external check. The paper explicitly acknowledges the choice but never addresses the gap. This would be acceptable as a heuristic if the paper said so; the abstract's "directly converge onto a steady equilibrium" is not supported.

The experiments are also too weak. There is no comparison against any other defender, no exploitability or deviation analysis, and the only reported opponent is the fixed B-line Red agent. The claim in the conclusion that the policies "consistently outperform heuristic agent baseline" is not demonstrated anywhere in the results. I also noticed a confusing metric: Section 7.4 reports "successful attack impacts" for the Blue agent and interprets it as offensive effectiveness, but Blue in CybORG CC2 is a defender with actions like Decoy, Analyse, Restore, Block. This suggests either a mislabeled experiment or a misunderstanding of the environment, and it makes the empirical story hard to trust.

Who is this for? Someone working on MARL-based cyber defense might find the architecture worth a look, but as a paper it reads like an early technical report rather than a submission-ready work. It needs either a real convergence guarantee for the algorithm as stated, or a much more careful empirical study with baselines and equilibrium verification. I would not send this to a serious referee in its current form; I would send it back with a clear list of missing pieces and let the authors decide whether to invest in them.

Referee Report

3 major / 5 minor

Summary. The paper proposes the Nash Q-Network (Nash Q-N), a centralized-training MARL algorithm for two-player zero-sum Markov games, applied to the CybORG CC2 cyber-defense simulation. A centralized critic outputs a joint action-value matrix Q_phi(s_B,s_R,a_B,a_R); stage-game Nash equilibria are computed from this matrix, and each agent's policy is trained by minimizing cross-entropy against those equilibrium mixed strategies. Training alternates between data collection, critic fitting with policy-based TD targets, and policy updates via Nash alignment. The authors claim the method 'directly converge[s] onto a steady equilibrium' and that the resulting Blue policy 'consistently outperform[s] heuristic agent baseline.' Experiments show improving cumulative reward for a trained Blue agent against a fixed B-line Red adversary over 1000 training epochs.

Significance. If the central claim were established, the paper would make a meaningful contribution by scaling Nash Q-learning with deep function approximation and applying it to a realistic cyber-defense benchmark. The problem formulation is clean, the architecture (centralized joint critic with decentralized policies) is sensible, and the use of parallel environment collection via Ray is a practical strength. However, the theoretical connection between the algorithm's updates and a Nash equilibrium of the stochastic game is missing, and the empirical evaluation contains no equilibrium check or comparison against any baseline. The manuscript's own Section 5.1 states that TD targets are computed by expectations over current policies 'instead of using equilibrium strategies for bootstrapping,' which directly contradicts the definition of Nash Q-values in Eq. (1). Since the abstract and conclusion make strong equilibrium and performance claims, the significance of the paper hinges on an unsupported step. The work would be a useful empirical study if reframed as an equilibrium-inspired heuristic, but as written the central claim is not substantiated.

major comments (3)

[§5.4.2 and §5.4.3 vs. §3.2, Eq. (1)] The critic target in Eq. (2)–(3) is y_t = r_t + γ E_{a'_B∼π_θB, a'_R∼π_θR}[Q_φ(s', a'_B, a'_R)]. This is a policy-evaluation bootstrap for the current joint policy, not the Nash Q-function defined in Eq. (1), whose continuation value is v_i(s', π_*) under the equilibrium strategy profile. The stage-game Nash equilibria computed in §5.4.3 from Q_φ are therefore equilibrium solutions of a payoff matrix that does not encode equilibrium continuation values, unless the current policies already coincide with the equilibrium. The paper provides no convergence theorem, contractivity argument, or policy-iteration justification connecting these alternating updates to a fixed point of the Nash Q-operator. The abstract's claim of 'directly converge onto a steady equilibrium' is thus unsupported. This is the load-bearing issue: without a formal or empirical link to equilibrium values, the 'Nash' targ
[§7, experimental evaluation] The evaluation compares the trained Nash Blue agent only against a fixed B-line Red adversary. There is no comparison against a heuristic Blue baseline, a random policy, independent PPO/DQN, or a previously published CybORG defender, even though the conclusion states that 'equilibrium-informed policies consistently outperform heuristic agent baseline.' Figure 2a shows final average reward approximately -5662.69 ± 4349.63 and a maximum of -3665.44, but without a baseline these numbers do not demonstrate that the learned policy is robust or equilibrium-like. No exploitability measurement, unilateral-deviation test, or best-response analysis is reported. Therefore the empirical claim of 'robust defenses' and 'consistent outperformance' is not established.
[§7.4 and §7.2] The 'Successful attack impacts' metric is defined using I[action_R = Impact], i.e., it counts Red's Impact actions, but the text in §7.4 attributes this to 'the Blue agent' and describes 'the agent continuously refines offensive capabilities.' This is either a mislabeling of the metric or a misinterpretation of the results. If the metric counts Red's successful impacts, then the near-linear increase indicates the attacker's success, not Blue's defense; if it is meant to measure Blue's attacks, the definition is wrong. This ambiguity undermines the interpretation of Figure 2b and the action-selection analysis in §7.5.

minor comments (5)

[§1, Contributions] The bullet list uses lowercase 'nash equilibrium' and 'cross entropy divergence'; please use consistent capitalization and 'cross-entropy' throughout.
[§7.2] The B-line Red adversary is never defined or referenced. Please provide a description or citation of this heuristic so the reader can assess the strength of the opponent and the meaning of 'outperform.'
[Eq. (2)–(3)] Notation is inconsistent: Eq. (2) uses a'_B, a'_R for actions sampled at s', while Eq. (3) sums over a_B, a_R. Use a single notation for next-state actions.
[§6.3 and §7.2] Table 1 lists rollout horizon T=2000, but §7.2 says each evaluation episode runs for 1000 timesteps. Clarify whether the rollout horizon differs between training and evaluation, and how T is used in Algorithm 1.
[§5.4.3] The stage-game Nash equilibrium solver is not specified in the algorithm or implementation. The related work mentions Lemke-Howson, but the manuscript should state which solver is used for the Q-matrix at each state and how ties among multiple Nash equilibria are broken.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the critic-policy loop is a bootstrap, not a definitional reduction.

full rationale

The paper's algorithm uses a centralized critic Q_phi trained with TD targets that are expectations over the agents' current policies (Eqs. 2–3), and then updates policies toward the stage-game Nash equilibrium of Q_phi's output matrix (Phase 3). This is a self-referential training loop common in actor-critic methods, but it is not circular in the sense defined here. The Nash equilibrium is computed from the critic, not defined as the critic; the critic is fitted to current-policy returns, not to the Nash value. There is no equation in which the predicted quantity (the stage-game Nash policy) is identical by construction to the fitted input (the current policy) or to the critic's definition. The gap between the current-policy Bellman operator and the Nash Q-operator of Eq. (1) is a convergence/correctness concern, not a definitional circularity. The paper also contains no load-bearing self-citations: the references to Nash Q-learning and deep Nash Q-learning are external prior work, and no uniqueness theorem or prior result by the same authors is invoked to force the choice. Therefore, no circular step can be exhibited from the text. The unsupported 'direct convergence to a steady equilibrium' claim is a weakness in empirical or theoretical validation, but that falls outside the scope of circularity analysis.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical or ontological entities are introduced; the Nash Q-Network is an algorithmic construct. The ledger instead shows that the empirical claims depend on several unstated modeling choices, especially a reward-shaping scheme that is never specified and a self-referential critic-policy loop whose convergence is not proven.

free parameters (4)

Reward shaping terms = Unspecified
Section 7.5 says 'reward shaping employed in this work' but never defines the shaping terms; these determine the learning signal and are tuned implicitly.
Discount factor gamma = 0.99
Standard in RL but chosen by hand; Table 1.
Policy and critic learning rates = 1e-3 each
Chosen based on empirical tuning, Section 6.2.
Critic update epochs K, rollout horizon T, batch size B = K=6, T=2000, B=64
Empirical choices in Table 1, no sensitivity analysis.

assumptions (5)

domain assumption CybORG CC2 is adequately modeled as a two-player zero-sum Markov game with finite discrete action spaces.
Sections 4 and 7.1 impose this model on a partially observable stochastic environment with action masks.
ad hoc to paper The centralized critic's Q-matrix values correspond to expected returns under a Nash equilibrium after training.
Section 5.4.3 uses stage-game Nash equilibria of Q_phi as policy targets, but no proof links the critic's policy-evaluation targets to equilibrium values.
ad hoc to paper Bootstrapping the critic from expectations over current policies, instead of from the equilibrium value, preserves the validity of the equilibrium targets.
Section 5.4.2 defines y_t using pi_theta_B and pi_theta_R, which are the policies being updated; this is the central unproved premise.
domain assumption Softmax MLP policies are expressive enough to represent the equilibrium and provide sufficient exploration.
Section 5.2.1 defines policies as softmax MLPs, with no analysis of capacity or exploration coverage.
standard math A Nash equilibrium of each stage game can be computed exactly and efficiently for the constructed Q-matrix.
Section 3.2 mentions the Lemke-Howson algorithm, but the implementation details are not given.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Nash Q-Network for Multi-Agent Cybersecurity Simulation." pith.science (2026). https://pith.science/paper/K4UE4HSJ

@misc{pith2026250900678,
  author       = {Pith},
  title        = {Pith review of: Nash Q-Network for Multi-Agent Cybersecurity Simulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K4UE4HSJ}},
  note         = {Machine review of arXiv:2509.00678}
}

read the original abstract

Cybersecurity defense involves interactions between adversarial parties (namely defenders and hackers), making multi-agent reinforcement learning (MARL) an ideal approach for modeling and learning strategies for these scenarios. This paper addresses one of the key challenges to MARL, the complexity of simultaneous training of agents in nontrivial environments, and presents a novel policy-based Nash Q-learning to directly converge onto a steady equilibrium. We demonstrate the successful implementation of this algorithm in a notable complex cyber defense simulation treated as a two-player zero-sum Markov game setting. We propose the Nash Q-Network, which aims to learn Nash-optimal strategies that translate to robust defenses in cybersecurity settings. Our approach incorporates aspects of proximal policy optimization (PPO), deep Q-network (DQN), and the Nash-Q algorithm, addressing common challenges like non-stationarity and instability in multi-agent learning. The training process employs distributed data collection and carefully designed neural architectures for both agents and critics.

Figures

Figures reproduced from arXiv: 2509.00678 by the authors.

**Figure 2.** Figure (a) shows steady improvement and convergence in cumulative reward over epochs, [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

24 extracted references · 17 canonical work pages

[1]

Mastering the game of go with deep neural networks and tree search,

D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctotet al., “Mastering the game of go with deep neural networks and tree search,”nature, vol. 529, no. 7587, pp. 484–489, 2016. 14

work page 2016
[2]

Grandmaster level in starcraft ii using multi-agent reinforcement learning,

O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Mathieu, A. Dudzik, J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgievet al., “Grandmaster level in starcraft ii using multi-agent reinforcement learning,”nature, vol. 575, no. 7782, pp. 350–354, 2019

2019
[3]

Nash q-learning for general-sum stochastic games,

J. Hu and M. P. Wellman, “Nash q-learning for general-sum stochastic games,”Journal of Machine Learning Research, vol. 4, pp. 1039–1069, 2003

work page 2003
[4]

Cyborg: A gym for the development of autonomous cyber agents,

M. Standen, M. Lucas, D. Bowman, T. J. Richer, J. Kim, and D. Marriott, “Cyborg: A gym for the development of autonomous cyber agents,”arXiv preprint arXiv:2108.09118, 2021

arXiv 2021
[5]

Markov games as a framework for multi-agent reinforcement learning,

M. L. Littman, “Markov games as a framework for multi-agent reinforcement learning,” in Machine learning proceedings 1994. Elsevier, 1994, pp. 157–163

work page 1994
[6]

Markov perfect equilibrium in a repeated principal-agent rela- tionship,

D. Minehart and J. Umanski, “Markov perfect equilibrium in a repeated principal-agent rela- tionship,” Journal of Economic Theory, vol. 64, no. 2, pp. 322–340, 1994

work page 1994
[7]

A survey of game theory as applied to network security,

S. Roy, C. Ellis, S. Shiva, D. Dasgupta, V. Shandilya, and Q. Wu, “A survey of game theory as applied to network security,” in2010 43rd Hawaii international conference on system sciences. IEEE, 2010, pp. 1–10

work page 2010
[8]

The complexity of computing a nash equilibrium,

C. Daskalakis, P. W. Goldberg, and C. H. Papadimitriou, “The complexity of computing a nash equilibrium,” Communications of the ACM, vol. 52, no. 2, pp. 89–97, 2009

2009

Show all 24 references

[9]

A comprehensive survey of multiagent reinforce- ment learning,

L. Busoniu, R. Babuska, and B. De Schutter, “A comprehensive survey of multiagent reinforce- ment learning,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 38, no. 2, pp. 156–172, 2008

2008
[10]

A deep learning-based multi-agent system for intrusion detection,

F. Louati and F. B. Ktata, “A deep learning-based multi-agent system for intrusion detection,” SN Applied Sciences, vol. 2, no. 4, p. 675, 2020

2020
[11]

Deep reinforcement learning for adaptive cyber defense in network security,

A. A. Hammad, S. R. Ahmed, M. K. Abdul-Hussein, M. R. Ahmed, D. A. Majeed, and S. Al- gburi, “Deep reinforcement learning for adaptive cyber defense in network security,” inPro- ceedings of the Cognitive Models and Artificial Intelligence Conference, 2024, pp. 292–297

2024
[12]

Reinforcement learning for efficient network penetration testing,

M. C. Ghanem and T. M. Chen, “Reinforcement learning for efficient network penetration testing,” Information, vol. 11, no. 1, p. 6, 2019

2019
[13]

Combining deep reinforcement learning and search for imperfect-information games,

N. Brown, A. Bakhtin, A. Lerer, and Q. Gong, “Combining deep reinforcement learning and search for imperfect-information games,”Advances in neural information processing systems, vol. 33, pp. 17057–17069, 2020

2020
[14]

Safe multi-agent reinforcement learning with convergence to generalized nash equilibrium,

Z. Li and N. Azizan, “Safe multi-agent reinforcement learning with convergence to generalized nash equilibrium,” arXiv preprint arXiv:2411.15036, 2024

2024 arXiv
[15]

A comprehensive systematic literature review on intrusion detection systems,

M. Ozkan-Okay, R. Samet, Ö. Aslan, and D. Gupta, “A comprehensive systematic literature review on intrusion detection systems,”IEEE Access, vol. 9, pp. 157727–157760, 2021

2021
[16]

Correlated q-learning,

A. Greenwald, K. Hall, R. Serranoet al., “Correlated q-learning,” inICML, vol. 3, 2003, pp. 242–249. 15

2003
[17]

Alpcan and T

T. Alpcan and T. Başar,Network security: A decision and game-theoretic approach. Cam- bridge University Press, 2010

2010
[18]

Deep q-learning for nash equilibria: Nash-dqn,

P. Casgrain, B. Ning, and S. Jaimungal, “Deep q-learning for nash equilibria: Nash-dqn,” Applied Mathematical Finance, vol. 29, no. 1, pp. 62–78, 2022

2022
[19]

Equilibrium points of bimatrix games,

C. E. Lemke and J. T. Howson, Jr, “Equilibrium points of bimatrix games,”Journal of the Society for industrial and Applied Mathematics, vol. 12, no. 2, pp. 413–423, 1964

1964
[20]

Ray: A distributed framework for emerging{AI} applications,

P. Moritz, R. Nishihara, S. Wang, A. Tumanov, R. Liaw, E. Liang, M. Elibol, Z. Yang, W. Paul, M. I. Jordanet al., “Ray: A distributed framework for emerging{AI} applications,” in 13th USENIX symposium on operating systems design and implementation (OSDI 18), 2018, pp. 561–577

2018
[21]

Adam: A method for stochastic optimization,

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014

2014 arXiv
[22]

On autonomous agents in a cyber defence environment,

M. Kiely, D. Bowman, M. Standen, and C. Moir, “On autonomous agents in a cyber defence environment,” arXiv preprint arXiv:2309.07388, 2023

2023 arXiv
[23]

Autonomous network defence using reinforcement learning,

M. Foley, C. Hicks, K. Highnam, and V. Mavroudis, “Autonomous network defence using reinforcement learning,” in Proceedings of the 2022 ACM on Asia Conference on Computer and Communications Security, 2022, pp. 1252–1254

2022
[24]

Learning to communicate in multi-agent reinforcement learning for au- tonomous cyber defence,

F. Contractoret al., “Learning to communicate in multi-agent reinforcement learning for au- tonomous cyber defence,” 2024. 16

2024

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

Mastering the game of go with deep neural networks and tree search,

D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctotet al., “Mastering the game of go with deep neural networks and tree search,”nature, vol. 529, no. 7587, pp. 484–489, 2016. 14

work page 2016

[2] [2]

Grandmaster level in starcraft ii using multi-agent reinforcement learning,

O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Mathieu, A. Dudzik, J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgievet al., “Grandmaster level in starcraft ii using multi-agent reinforcement learning,”nature, vol. 575, no. 7782, pp. 350–354, 2019

2019

[3] [3]

Nash q-learning for general-sum stochastic games,

J. Hu and M. P. Wellman, “Nash q-learning for general-sum stochastic games,”Journal of Machine Learning Research, vol. 4, pp. 1039–1069, 2003

work page 2003

[4] [4]

Cyborg: A gym for the development of autonomous cyber agents,

M. Standen, M. Lucas, D. Bowman, T. J. Richer, J. Kim, and D. Marriott, “Cyborg: A gym for the development of autonomous cyber agents,”arXiv preprint arXiv:2108.09118, 2021

arXiv 2021

[5] [5]

Markov games as a framework for multi-agent reinforcement learning,

M. L. Littman, “Markov games as a framework for multi-agent reinforcement learning,” in Machine learning proceedings 1994. Elsevier, 1994, pp. 157–163

work page 1994

[6] [6]

Markov perfect equilibrium in a repeated principal-agent rela- tionship,

D. Minehart and J. Umanski, “Markov perfect equilibrium in a repeated principal-agent rela- tionship,” Journal of Economic Theory, vol. 64, no. 2, pp. 322–340, 1994

work page 1994

[7] [7]

A survey of game theory as applied to network security,

S. Roy, C. Ellis, S. Shiva, D. Dasgupta, V. Shandilya, and Q. Wu, “A survey of game theory as applied to network security,” in2010 43rd Hawaii international conference on system sciences. IEEE, 2010, pp. 1–10

work page 2010

[8] [8]

The complexity of computing a nash equilibrium,

C. Daskalakis, P. W. Goldberg, and C. H. Papadimitriou, “The complexity of computing a nash equilibrium,” Communications of the ACM, vol. 52, no. 2, pp. 89–97, 2009

2009