REVIEW 4 major objections 6 minor 28 references
Evolutionary and Coevolutionary Multi-Agent Design Choices and Dynamics
T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Grammar-evolved code-like cyber agents achieve the highest team reward, while coevolution flattens the peaks and valleys that one-sided training produces.
desk verdict A useful but under-powered comparative study whose central coevolutionary dynamics claim is compromised by non-comparable fitness curves. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism pair is (1) a context-free grammar whose non-terminals expand into Python-like statement blocks for action and target selection, decoded from an integer vector by grammatical evolution—the grammar itself fixes the syntax while evolution searches the conditions, actions, and target heuristics; and (2) the competitive coevolutionary algorithm, two evolutionary algorithms coupled only at fitness evaluation, where each individual's score comes from all-vs-all competitions against the other population. For the LLM variant, the mutation operator sends the current code block plus grammar context to the LLM and expects a mutated block, operating on the code rather than the genome. The fixed-size action-selection matrix, with one probability per state–action pair, is the contrasting mechanism that the grammar representation must beat.
What would settle it
Re-train one-sided agents against a substantially stronger static adversary (for example, a previously evolved red team with high reward) and check whether their peak fitness still exceeds the coevolutionary envelope; if it does not, the paper's central dynamic claim fails. A second check: run coevolution for several times more generations and measure whether the flattened reward envelope persists, ruling out that the effect is only a transient of limited-horizon training.
Extended reading notes
Core claim
On its own terms, the paper claims that representation choices dominate algorithm choices in this domain: controllers expressed as Python-like code via grammatical evolution, using grammars that expose actions, targets, and observation counts, achieve the best team training performance across all tested designs, and the LLM-supported GE variant is competitive but highly sensitive to prompt and model choice. It further claims that coevolution, coupling two evolutionary algorithms at the fitness evaluation step, blunts the performance highs and lows of both red and blue teams while inducing fluctuations in their rewards, whereas one-sided evolution against a fixed finite-state-machine adversary produces higher and more sustained peaks. The paper also reports that discrete action-probability optimization outperforms continuous optimization for the matrix representation, and that richer grammars can improve performance when they supply the right terminals.
Load-bearing premise
The hidden premise is that the hand-tuned finite-state machine used as the fixed adversary is a representative non-evolving opponent; if that baseline were much stronger, the observed peak advantage of one-sided training could shrink or reverse.
Editorial extensions
If this is right
- In this scenario, switching the controller representation from a fixed action-probability matrix to grammar-generated code is the highest-leverage design decision for training reward.
- Coevolving the attacker and defender reduces the best-achievable reward of both sides; a defender that expects adaptation should train against a fixed strong adversary to maximize peak performance, at the cost of flexibility against novel attack strategies.
- The LLM-based mutation operator's output quality is dominated by prompt engineering and model choice rather than by the evolutionary search itself.
- Adding observation functions to the grammar (connections, file counts, root access levels) improves both best and mean fitness, showing that expressiveness helps when the right terminals are available.
Reading between the lines
- If the fixed finite-state-machine adversary in Appendix A.2 is weaker than a well-trained defender would be, then the one-sided-versus-coevolution comparison is biased toward one-sided training; re-running the comparison with a stronger scripted opponent would test whether the damping result is robust.
- The oscillation the paper sees under coevolution hints at intransitive strategy cycles rather than simple convergence; checking whether fitness time series cycle under a fixed evaluation after training would separate cycling from genuine capability loss.
- A practical extension is a hybrid mutation operator that validates each LLM-produced mutation against the grammar (and against syntax) before accepting it, which the paper's prompt-sensitivity results suggest would recover much of the GE-LLM performance loss.
- Because the grammar abstracts controller logic away from the fixed network layout, the grammar-based controllers may transfer across CybORG network instantiations better than matrix controllers; a zero-shot transfer test across randomizations would quantify that.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper compares two controller representations (FSM-based action-selection matrices optimized by GA/ES, and code-like grammars optimized by GE/GE-LLM) for red and blue teams in a modified CybORG CCC4 scenario. It reports one-sided training against a fixed FSM opponent and two-sided coevolution, and claims that grammatical representations achieve the highest training fitness, that the GE-LLM operator is prompt- and model-sensitive, and that coevolution consistently dampens performance highs and lows while inducing fluctuations, whereas one-sided evolution reaches higher and more sustained peaks.
Significance. The paper addresses a relevant design space for evolutionary multi-agent cyber defense, extending CCC4 to train red agents and providing grammars and FSM baselines in the appendix. Strengths include the explicit statement of limitations, the reporting of LLM mutation validity and token statistics, and the inclusion of the experimental setup parameters. However, the central comparative claims about coevolution dynamics rest on fitness values measured under different evaluation protocols, and the headline performance claim is based on training reward only. With two repetitions, six trials, and no variance reporting, the empirical base is thin. If properly supported, the result would be a useful empirical data point for the evolutionary computation and cyber-security communities, but the current evidence does not fully support the abstract's general conclusions.
major comments (4)
- [§5.2, Fig. 4] The abstract's central claim that coevolution reduces the performance highs and lows and induces fluctuations is not supported by the displayed comparison, because the two sets of curves are not measured on a common scale. One-sided fitness values are computed against the fixed FSM adversary (Appendix A.2), whereas the paper itself states that 'the best fitness for coevolving agents is relative to the adversaries that it was evaluated against.' A decrease or oscillation in the coevolutionary curve can therefore be caused by the opponent improving, not by the evaluated population degrading. To make the comparison valid, all designs should be re-evaluated at each generation against the same fixed reference opponent (or a common evaluation protocol), and the peak/fluctuation comparison should be repeated on those commensurable fitness values.
- [§5.2, §5.3] The claim that the grammatical evolution versions 'can achieve the best team performance' is supported only by training fitness; Section 5.3 explicitly lists 'no out-of-sample evaluation of evolved solutions' as a limitation. Because the environment is stochastic and the host configurations are randomized, training reward is not necessarily team performance in the intended deployment sense. The authors should either add hold-out evaluation on new scenario seeds or unseen configurations, or restrict the claim to training fitness.
- [Table 2, §5.2] The robustness of the dynamics comparison is unclear with six trials and two repetitions, and the paper omits all variance information, stating 'for visual clarity we do not display the fitness variance among the independent trials.' The repeated qualitative statement that coevolution 'consistently' blunts highs and lows is a visual impression rather than a quantified result. Please report per-run summary statistics (e.g., mean and variance of peak, trough, and fluctuation metrics) or show individual trial curves, with a significance test or effect-size measure where possible.
- [§5.2.4] The final prompts used for the LLM-supported mutation operator are not provided, despite Section 5.2.4 showing that the operator's behavior changed qualitatively with prompt wording and that the authors had to add a cybersecurity-expert role instruction to obtain 'high quality mutations.' Since GE-LLM is one of the tested algorithms, this omission makes the GE-LLM comparison unrepeatable and the reported prompt sensitivity claim unverifiable. The exact prompts (or a complete prompt template) should be included in an appendix.
minor comments (6)
- [Appendix A.2] There is a typo in the red FSM state description: 'Usesr with discovery' should be 'User with discovery.'
- [Table 1] The first data row in Table 1 begins with 'Us,' which appears to be a leftover author tag rather than a meaningful environment or method entry; please clean this up.
- [§5.2.1] The statement that 'the GE-LLM-C has lower fitness for both red and blue, still order of magnitude lower than FSM' is confusing given the earlier statement that GE fitness is an order of magnitude higher than FSM; please disambiguate which comparison is intended.
- [Figure 2, Appendix A.1] Several grammar productions are missing closing quotation marks on the 'return' and 'target_heuristic' lines; this makes the grammar fragment harder to parse and should be corrected.
- [§5.1.1] The text says 'one controller is used for all the re/blue agents'; this should be 'red/blue agents.'
- [§5.2.2, Figure 5] The figure captions for the target-selection and extended-observation results (e.g., Fig. 6a) say 'GE-Red' without explaining which side is being plotted; please clarify the side and the comparison in each caption.
Circularity Check
No circular derivation: the paper is an empirical comparison, not a derivation, with only a minor non-load-bearing self-citation; the acknowledged relative-fitness caveat is a methodological concern, not a definitional circularity.
full rationale
The central claims (GE grammars achieve the best training fitness; coevolution dampens reward extremes relative to one-sided training) are direct observations of logged CybORG simulation rewards, not quantities derived from fitted parameters or from each other. No equation is involved, and no fitted input is renamed as a prediction. The only inline self-citation, [13] (Hemberg, Moskal and O'Reilly, 'Evolving code with a large language model'), appears in Section 4.2.1 to support the statement that 'LLM based operators of this type are often problem environment dependent'; it is peripheral and not load-bearing for any of the paper's comparative conclusions. The paper explicitly acknowledges the main caveat in Section 5.2: 'the best fitness for coevolving agents is relative to the adversaries that it was evaluated against,' and it lists 'no out-of-sample evaluation' as a limitation in Section 5.3. This means the one-sided versus coevolution comparison is not fully commensurable in an absolute sense, but the paper does not use that definition to construct its conclusions; it simply reports the observed curves. Consequently, there is no circular step where an output reduces by construction to an input. The score of 2 reflects the presence of a minor self-citation and the acknowledged protocol mismatch, neither of which makes the derivation circular.
Assumptions & free parameters
free parameters (5)
- Population size, trials, generations, steps, repetitions =
10, 6, >=20, 75, 2
- Mutation and crossover probability =
0.5
- FSM transition probabilities for fixed adversaries =
see matrices in Appendix A.2
- LLM prompts =
progressively refined through 4 versions
- Grammar design choices =
baseline plus 5 variants
assumptions (4)
- domain assumption CybORG's CAGE Challenge 4 simulator is an accurate-enough model of cyber operations for agent training conclusions.
- domain assumption The code produced by the grammars is executable and semantically meaningful in the CybORG environment.
- domain assumption Mean expected utility is an appropriate solution concept for comparing coevolving populations.
- ad hoc to paper Six trials and two repetitions suffice to reveal stable dynamics.
Cite this review
Pith. "Pith review of Evolutionary and Coevolutionary Multi-Agent Design Choices and Dynamics." pith.science (2026). https://pith.science/paper/T3YLXMZC
@misc{pith2026250705534,
author = {Pith},
title = {Pith review of: Evolutionary and Coevolutionary Multi-Agent Design Choices and Dynamics},
year = {2026},
howpublished = {\url{https://pith.science/paper/T3YLXMZC}},
note = {Machine review of arXiv:2507.05534}
}
read the original abstract
We investigate two representation alternatives for the controllers of teams of cyber agents. We combine these controller representations with different evolutionary algorithms, one of which introduces a novel LLM-supported mutation operator. Using a cyber security scenario, we evaluate agent learning when one side is trained to compete against a side that does not evolve and when two sides coevolve with each other. This allows us to quantify the relative merits and tradeoffs of representation and algorithm combinations in terms of team performance. Our versions of grammatical evolution algorithms using grammars that allow a controller to be expressed in code-like logic can achieve the best team performance. The scenario also allows us to compare the performance impact and dynamics of coevolution versus evolution under different combinations. Across the algorithms and representations, we observe that coevolution reduces the performance highs and lows of both sides while it induces fluctuations on both sides. In contrast, when only one-side is optimized, performance peaks are higher and is more sustained than when both sides are optimized with coevolution.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[13]
Erik Hemberg, Stephen Moskal, and Una-May O’Reilly. 2024. Evolving code with a large language model. Genetic Programming and Evolvable Machines 25, 2 (2024), 21
work page 2024
-
[1]
2022. Cyber Operations Research Gym. https://github.com/cage-challenge/ CybORG. Created by Maxwell Standen, David Bowman, Olivia Naish, Ben Edwards, James Drane, Claire Owens, KC Cowan, Wayne Gould, Mitchell Kiely, Son Hoang, Toby Richer, Martin Lucas, Richard Van Tassel, Phillip Vu, Natalie Konschnik, Joshua Collyer, Calum Fairchild, Thomas Harding
work page 2022
-
[2]
L. M. Antonio and C. A. C. Coello. 2018. Coevolutionary Multi-objective Evolu- tionary Algorithms: A Survey of the State-of-the-Art. IEEE Transactions on Evo- lutionary Computation (2018), 1–16. https://doi.org/10.1109/TEVC.2017.2767023
arXiv 2018
-
[3]
Thomas Back. 1996. Evolutionary algorithms in theory and practice: evolution strategies, evolutionary programming, genetic algorithms. Oxford university press
work page 1996
-
[4]
Leonardo Lucio Custode, Chiara Camilla Migliore Rambaldi, Marco Roveri, and Giovanni Iacca. 2024. Comparing large language models and grammatical evolu- tion for code generation. In Proceedings of the Genetic and Evolutionary Computa- tion Conference Companion. 1830–1837
work page 2024
-
[5]
Paul R Ehrlich and Peter H Raven. 1964. Butterflies and plants: a study in coevolution. Evolution 18, 4 (1964), 586–608
work page 1964
-
[6]
Mario Alejandro Hevia Fajardo, Erik Hemberg, Jamal Toutouh, Una-May O’Reilly, and P. Lehre. 2024. A Self-adaptive Coevolutionary Algorithm. Proceedings of the Genetic and Evolutionary Computation Conference (2024). https://api. semanticscholar.org/CorpusID:271065215
work page 2024
-
[7]
Diksha Goel, Max Ward, Aneta Neumann, Frank Neumann, Hung Nguyen, and Mingyu Guo. 2024. Hardening Active Directory Graphs via Evolutionary Diver- sity Optimization based Policies. ACM Transactions on Evolutionary Learning (2024)
work page 2024
Show all 28 references
-
[8]
Goldberg
David E. Goldberg. 1989. Genetic Algorithms in Search, Optimization and Machine Learning (1st ed.). Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA
1989
-
[9]
TTCP CAGE Working Group. 2023. TTCP CAGE Challenge 4. https://github. com/cage-challenge/cage-challenge-4
2023
-
[10]
Kim Hammar, Neil Dhir, and Rolf Stadler. 2024. Optimal Defender Strategies for CAGE-2 using Causal Modeling and Tree Search. arXiv preprint arXiv:2407.11070 (2024)
2024 arXiv
-
[11]
Harris and Daniel R
Sean N. Harris and Daniel R. Tauritz. 2021. Competitive coevolution for defense and security: Elo-based similar-strength opponent sampling. Proceedings of the Genetic and Evolutionary Computation Conference Companion (2021). https: //api.semanticscholar.org/CorpusID:235770343
2021
-
[12]
Kade Heckel. 2023. Neuroevolution for Autonomous Cyber Defense. Proceedings of the Companion Conference on Genetic and Evolutionary Computation (2023). https://api.semanticscholar.org/CorpusID:260119489
2023
-
[14]
Erik Hemberg, Jacob Rosen, Geoff Warner, Sanith Wijesinghe, and Una-May O’Reilly. 2016. Detecting tax evasion: a co-evolutionary approach. Artificial Intelligence and Law 24 (2016), 149–182
2016
-
[15]
Stephen Kelly and Malcolm I. Heywood. 2017. Emergent Tangled Graph Repre- sentations for Atari Game Playing Agents. In European Conference on Genetic Programming. https://api.semanticscholar.org/CorpusID:26568610
2017
-
[16]
Krzysztof Krawiec and Malcolm Heywood. 2016. Solving Complex Problems with Coevolutionary Algorithms. InProceedings of the 2016 on Genetic and Evolutionary Computation Conference Companion. ACM, 687–713
2016
-
[17]
Irina Maliukov, Gera Weiss, Oded Margalit, and Achiya Elyasaf. 2024. Evolving Assembly Code in an Adversarial Environment. ArXiv abs/2403.19489 (2024). https://api.semanticscholar.org/CorpusID:268732824
2024 arXiv
-
[18]
Melanie Mitchell. 2006. Coevolutionary learning with spatially distributed popu- lations. Computational intelligence: principles and practice 400 (2006)
2006
-
[19]
Michael O’Neill and Conor Ryan. 2001. Grammatical evolution.IEEE Transactions on Evolutionary Computation 5, 4 (2001), 349–358
2001
-
[20]
Una-May O’Reilly, Jamal Toutouh, Marcos Pertierra, Daniel Prado Sanchez, Den- nis Garcia, Anthony Erb Luogo, Jonathan Kelly, and Erik Hemberg. 2020. Adver- sarial genetic programming for cyber security: A rising application domain where GP matters. Genetic Programming and Evol...
2020
-
[21]
Paul Wiegand, and Edwin D
Elena Popovici, Anthony Bucci, R. Paul Wiegand, and Edwin D. De Jong. 2012. Coevolutionary Principles. Springer Berlin Heidelberg, Berlin, Heidelberg, 987– 1033
2012
-
[22]
Christopher D Rosin and Richard K Belew. 1997. New methods for competitive coevolution. Evolutionary Computation 5, 1 (1997), 1–29
1997
-
[23]
Tauritz, and Alexander D
George Rush, Daniel R. Tauritz, and Alexander D. Kent. 2015. Coevolutionary Agent-based Network Defense Lightweight Event System (CANDLES). Proceed- ings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation (2015). https://api.sema...
2015
-
[24]
Alexander Shashkov, Erik Hemberg, Miguel Tulla, and Una-May O’Reilly. 2023. Adversarial agent-learning for cybersecurity: a comparison of algorithms. The Knowledge Engineering Review 38 (2023). https://api.semanticscholar.org/ CorpusID:257354029
2023
-
[25]
Karl Sims. 1994. Evolving 3D morphology and behavior by competition. Artificial life 1, 4 (1994), 353–372
1994
-
[26]
Smith, Ayse Nur Zincir-Heywood, Malcolm I
Robert J. Smith, Ayse Nur Zincir-Heywood, Malcolm I. Heywood, and John T. Jacobs. 2016. Initiating a Moving Target Network Defense with a Real-time Neuro-evolutionary Detector. Proceedings of the 2016 on Genetic and Evolutionary Computation Conference Companion (2016). https:/...
2016
-
[27]
Richer, Junae Kim, and Damian A
Maxwell Standen, Martin Lucas, David Bowman, Toby J. Richer, Junae Kim, and Damian A. Marriott. 2021. CybORG: A Gym for the Development of Autonomous Cyber Agents. ArXiv abs/2108.09118 (2021). https://api.semanticscholar.org/ CorpusID:237259783
2021 arXiv
-
[28]
def select_action_and_target(observation, name):
Alexander Wei, David A. Bierbrauer, Emily A. Nack, John Pavlik, and Nathaniel Bastian. 2024. Offline Reinforcement Learning for Autonomous Cyber Defense Agents. 2024 Winter Simulation Conference (WSC) (2024), 1978–1989. https: //api.semanticscholar.org/CorpusID:275773130 10 A ...
2024
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.