REVIEW 2 major objections 5 minor 1 cited by
No Press Diplomacy: Modeling Multi-Agent Gameplay
T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A neural policy trained on 150,000 human games beats hand-crafted rule-based bots in the No Press variant of Diplomacy, the paper reports.
desk verdict A solid benchmark paper: the first large-scale neural result in No Press Diplomacy, and the claim that DipNet beats rule-based bots holds up despite a few evaluation caveats. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is DipNet: a graph convolutional network over the 81-province map with FiLM conditioning on player power and season, an LSTM decoder that generates unit orders one at a time in a top-left to bottom-right ordering with a masked softmax over valid orders, and a second input stream encoding previous-phase orders. The graph convolution aggregates neighbor information over 16 layers so the network can track long-range unit coordination; FiLM injects whose turn it is and what season; the sequential masked decoder ensures only legal orders are produced. This object carries the argument because the claimed performance and the support-coordination analysis both depend on it: ablations that remove FiLM, the board state, or the location-aware decoder drop support accuracy, especially for the 16th unit in the order sequence, and reduce effective cross-power support.
What would settle it
Re-run the mixed tournament with starting powers forced to be balanced across agents and recompute TrueSkill; if Albert reaches or passes DipNet's score under that protocol, the claimed state-of-the-art performance is an artifact of the original unbalanced draw.
Extended reading notes
Core claim
The paper's central claim is that a fully end-to-end neural policy can achieve state-of-the-art No Press Diplomacy performance without an explicit communication channel: DipNet, trained by supervised learning from expert human trajectories and then fine-tuned by self-play reinforcement learning, beats popular rule-based bots, including Albert. In the 1-vs-6 tournament, SL DipNet wins 28.9% of games against six Albert copies and is defeated in 23.1% of games, with a TrueSkill of 28.1 against Albert's 24.5; RL DipNet scores 27.4. The paper also claims that the architecture's support-order prediction accuracy and its effective cross-power support ratio show it can coordinate units and cooperate with other powers, and that this cooperation learned from human data partially deteriorates during self-play.
Load-bearing premise
The ranking rests on the tournament's TrueSkill scores ignoring Diplomacy's starting-power imbalance, so the comparison is only trustworthy if every agent drew each of the seven powers equally often across the 1,378 games.
Editorial extensions
If this is right
- If the TrueSkill ranking holds, DipNet is the first data-driven agent to beat the strongest rule-based No Press bot, making Diplomacy a viable target for learning-based multi-agent research rather than purely hand-crafted strategies.
- The 150,000-game human dataset and the released game engine give other agents a shared evaluation platform, so future No Press results can be compared against the same rule-based bots and the same rating procedure.
- The support-order ablation shows the full architecture keeps coordination accuracy at the 16th decoded unit (32.2% vs 25.9% for the board-state-only variant), indicating that the map encoder and decoder jointly sustain multi-unit tactics.
- The coalition analysis shows SL DipNet has an effective cross-power support ratio of 10.2% versus 5.3% for RL DipNet, so the paper's self-play recipe improves overall strength without preserving the human-learned cooperative behavior.
Reading between the lines
- An implication the authors leave implicit is that the power-imbalance caveat could be probed directly: a balanced-power rematch between DipNet and Albert, conditioned on starting country, would show whether the claimed edge is robust or driven by favorable draws.
- A natural extension is to use DipNet's previous-order stream as a controlled channel: feeding the agent games with randomized or current-phase orders would test whether cross-power support is genuinely coordinated signaling or a side effect of the board-state encoding.
- The deterioration of effective cooperation under self-play suggests an experiment the authors point toward but do not run: retrain the RL agent with an opponent-aware or inequity-averse objective and compare its effective cross-power support ratio to the SL baseline; if the ratio recovers, the known failure mode of naive policy gradients in social dilemmas would be confirmed in Diplomacy's richer
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper introduces DipNet, a neural-network policy for the No Press variant of Diplomacy. The model is trained by supervised learning on a newly collected dataset of 156,468 human games, then fine-tuned with A2C self-play. The central claim is that both the supervised (SL) and reinforcement-learned (RL) versions beat popular rule-based bots, with TrueSkill scores of 28.1 and 27.4 respectively, compared with 24.5 for the strongest baseline, Albert. The paper also presents architecture ablations, support-order prediction accuracy, and a coalition analysis based on cross-power support ratios, and it releases the dataset, game engine, and code.
Significance. The work is a significant empirical contribution: it is, to my knowledge, the first large-scale data-driven agent for No Press Diplomacy, and the reported TrueSkill gap over rule-based bots is a meaningful demonstration that learned policies can outperform hand-crafted bots in a complex deterministic social dilemma. The 156k-game dataset, the DAIDE-compatible engine, and the public code are valuable community assets. The ablation study and the coalition analysis are useful, and the observation that self-play training reduces effective cross-power support is an interesting and non-obvious finding. The evaluation caveats discussed below are important but do not invalidate the contribution.
major comments (2)
- [Appendix A; Section 6.2] The headline claim that DipNet 'beats popular rule-based bots' rests on TrueSkill ratings computed without accounting for starting-power imbalance, as the authors acknowledge in the final paragraph of Appendix A, and reported without confidence intervals or per-power breakdowns. Given that human win rates range from 3.6% for Italy to 7.2% for Turkey (Table 1), the aggregate TrueSkill gap could in principle be driven by a subset of starting powers. Random power assignment balances in expectation, so this is not a claim of overt bias, but it limits the support for the claim as stated. Please add a power-stratified analysis (for example, per-power TrueSkill or win rate conditional on assigned power) and either confidence intervals or a significance test for the SL-vs-Albert and RL-vs-Albert gaps.
- [Section 6.1] The supervised test set is defined as the last 5% of games sorted by game id in alphabetical order. If game ids are assigned chronologically or correlate with map variant or player population, this split may not be representative and could yield optimistic accuracy numbers. Please use a random split, or alternatively demonstrate that the last-5% subset has the same distribution over powers, game lengths, and game outcomes as the full dataset, and report standard errors or confidence intervals for the accuracy numbers.
minor comments (5)
- [Section 4.3] The phrase 'attention based on the location' is misleading because the decoder described in Eq. (2) concatenates the current location's embedding rather than computing attention over locations; please rephrase to match the actual mechanism.
- [Table 4] Please define in the caption whether 'Win%' means solo victories only, and clarify the relationship between the 1-vs-6 tournament and the TrueSkill tournament; the 'TrueSkill A-B' column appears to be the global rating difference rather than a rating computed from the adjacent 1-vs-6 games, and this should be stated explicitly.
- [Appendix B; Section 1] There are typos in which 'affect' should be 'effect'; please correct them throughout.
- [Table 1] The 'Total' row mixes win/draw/defeated percentages with survival-rate columns, which is confusing; please split the row or add a note explaining the columns.
- [Equation (1)] Please specify the encoding of the conditioning variables p and s (e.g., one-hot vectors) and the dimensions of the FiLM linear layer f_l, since the vector [p;s] is not defined in the text.
Circularity Check
No significant circularity: DipNet's central claim is an empirical tournament comparison against independent rule-based baselines, with no fitted parameter defining the target metric.
full rationale
The paper's central claim is that the SL and RL versions of DipNet 'demonstrate state-of-the-art No Press performance by beating popular rule-based bots' (Abstract; Section 6.2). This claim is supported by TrueSkill ratings computed from a tournament in which DipNet variants and the baselines (Random, GreedyBot, Dumbbot, Albert) are all evaluated under the same protocol (Appendix A). No parameter is fitted to the evaluation metric: the SL model is trained on held-out human trajectories, the RL model is trained by self-play with hand-set reward coefficients, and the TrueSkill rating is an external ranking computed from game outcomes. The ablation studies compare genuine architectural variants, and the coalition analysis uses independently defined metrics. The only self-citations in the paper are to FiLM conditioning layers (references [29] and [31], with overlapping authorship), but FiLM is used as a general architectural tool and is not load-bearing for the central empirical result. Appendix A explicitly acknowledges a limitation: 'we do not consider the existing power imbalance in the game, e.g., winning as Austria is harder than winning as France.' This is a validity concern about the ranking protocol, not a circularity: it does not make DipNet's victory definitional, and the manuscript itself flags the issue as future work. Because the central derivation is self-contained and the evaluation is against independent baselines, no circular step is present.
Assumptions & free parameters
free parameters (4)
- Local reward coefficient =
+1/-1 per supply center gained/lost
- Terminal reward scale =
34 points for solo win, proportional by supply centers for draws
- n-step return horizon =
15
- Number of GCN layers =
16
assumptions (4)
- domain assumption Orders from the last movement phase are sufficient to infer current relationships between powers.
- domain assumption The webDiplomacy dataset is a valid source of expert-level No Press play.
- domain assumption TrueSkill ratings computed from 1,378 games with ranks by elimination order and supply centers are a reliable agent ranking.
- standard math Standard stochastic gradient training and backpropagation are reliable for optimizing the architecture.
Cite this review
Pith. "Pith review of No Press Diplomacy: Modeling Multi-Agent Gameplay." pith.science (2026). https://pith.science/paper/4FFAXM4S
@misc{pith2026190902128,
author = {Pith},
title = {Pith review of: No Press Diplomacy: Modeling Multi-Agent Gameplay},
year = {2026},
howpublished = {\url{https://pith.science/paper/4FFAXM4S}},
note = {Machine review of arXiv:1909.02128}
}
read the original abstract
Diplomacy is a seven-player non-stochastic, non-cooperative game, where agents acquire resources through a mix of teamwork and betrayal. Reliance on trust and coordination makes Diplomacy the first non-cooperative multi-agent benchmark for complex sequential social dilemmas in a rich environment. In this work, we focus on training an agent that learns to play the No Press version of Diplomacy where there is no dedicated communication channel between players. We present DipNet, a neural-network-based policy model for No Press Diplomacy. The model was trained on a new dataset of more than 150,000 human games. Our model is trained by supervised learning (SL) from expert trajectories, which is then used to initialize a reinforcement learning (RL) agent trained through self-play. Both the SL and RL agents demonstrate state-of-the-art No Press performance by beating popular rule-based bots.
Figures
Forward citations
Cited by 1 Pith paper
-
Cognitive Dark Matter: Measuring What AI Misses
AI's patchy performance stems from under-measured hidden cognitive processes, called cognitive dark matter, which could be made visible by collecting process-tracing and neural-behavioral data at scale.
Reference graph
Works this paper leans on
-
[1]
Multi-agent reinforcement learning in sequential social dilemmas
Joel Z Leibo, Vinicius Zambaldi, Marc Lanctot, Janusz Marecki, and Thore Graepel. Multi-agent reinforcement learning in sequential social dilemmas. In Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems, pages 464–473. International Foundation for Autonomous Agents and Multiagent Systems, 2017
work page 2017
-
[2]
The hanabi challenge: A new frontier for ai research
Nolan Bard, Jakob N Foerster, Sarath Chandar, Neil Burch, Marc Lanctot, H Francis Song, Emilio Parisotto, Vincent Dumoulin, Subhodeep Moitra, Edward Hughes, et al. The hanabi challenge: A new frontier for ai research. arXiv preprint arXiv:1902.00506, 2019
arXiv 1902
-
[3]
Superhuman ai for heads-up no-limit poker: Libratus beats top professionals
Noam Brown and Tuomas Sandholm. Superhuman ai for heads-up no-limit poker: Libratus beats top professionals. Science, 359(6374):418–424, 2018
work page 2018
-
[4]
Deepstack: Expert-level artificial intelligence in heads-up no-limit poker
Matej Moravˇcík, Martin Schmid, Neil Burch, Viliam Lis`y, Dustin Morrill, Nolan Bard, Trevor Davis, Kevin Waugh, Michael Johanson, and Michael Bowling. Deepstack: Expert-level artificial intelligence in heads-up no-limit poker. Science, 356(6337):508–513, 2017
work page 2017
-
[5]
Openai five
OpenAI. Openai five. https://blog.openai.com/openai-five/, 2018
2018
-
[6]
Oriol Vinyals, Igor Babuschkin, Junyoung Chung, Michael Mathieu, Max Jaderberg, Wo- jciech M. Czarnecki, Andrew Dudzik, Aja Huang, Petko Georgiev, Richard Powell, Timo Ewalds, Dan Horgan, Manuel Kroiss, Ivo Danihelka, John Agapiou, Junhyuk Oh, Valentin Dalibard, David Choi, Laurent Sifre, Yury Sulsky, Sasha Vezhnevets, James Molloy, Trevor Cai, David Budd...
work page 2019
-
[7]
Dp w1995a: Communication in no-press diplomacy
Simon Szykman. Dp w1995a: Communication in no-press diplomacy. http://uk.diplom. org/pouch/Zine/W1995A/Szykman/Syntax.html, 1995. Accessed: 2019-05-01
work page 1995
-
[8]
Daide - diplomacy artificial intelligence development environment
David Norman. Daide - diplomacy artificial intelligence development environment. http: //www.daide.org.uk/, 2013. Accessed: 2019-05-01. 9
work page 2013
Show all 37 references
-
[9]
Diplomacy ai - albert
Jason van Hal. Diplomacy ai - albert. https://sites.google.com/site/diplomacyai/,
-
[10]
TrueskillTM: a bayesian skill rating system
Ralf Herbrich, Tom Minka, and Thore Graepel. TrueskillTM: a bayesian skill rating system. In Advances in neural information processing systems, pages 569–576, 2007
2007
-
[11]
A player rating system for diplomacy
Tony Nichols. A player rating system for diplomacy. http://www.stabbeurfou.org/docs/ articles/en/DP_S1998R_Diplomacys_New_Rating_System.html, 1998. Accessed: 2019-05-01
1998
-
[12]
Ghost-ratings explained
WebDiplomacy. Ghost-ratings explained. https://sites.google.com/view/ webdipinfo/ghost-ratings/ghost-ratings-explained , 2019. Accessed: 2019-05-01
2019
-
[13]
Site scoring system
super dipsy. Site scoring system. https://www.playdiplomacy.com/forum/viewtopic. php?f=565&t=34913, 2013. Accessed: 2019-05-01
2013
-
[14]
Asynchronous methods for deep reinforcement learning
V olodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lilli- crap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In International conference on machine learning, pages 1928–1937, 2016
1928
-
[15]
Mastering the game of go with deep neural networks and tree search
David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driess- che, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search. Nature, 529(7587):484, 2016
2016
-
[16]
Mastering the game of go without human knowledge
David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. Nature, 550(7676):354, 2017
2017
-
[17]
Human-level performance in first-person multiplayer games with population-based deep rein- forcement learning
Max Jaderberg, Wojciech M Czarnecki, Iain Dunning, Luke Marris, Guy Lever, Antonio Garcia Castaneda, Charles Beattie, Neil C Rabinowitz, Ari S Morcos, Avraham Ruderman, et al. Human-level performance in first-person multiplayer games with population-based deep rein- forcement l...
2018 arXiv
-
[18]
Dipblue: A diplomacy agent with strategic and trust reasoning
André Ferreira, Henrique Lopes Cardoso, and Luis Paulo Reis. Dipblue: A diplomacy agent with strategic and trust reasoning. In ICAART 2015-7th International Conference on Agents and Artificial Intelligence, Proceedings, 2015
2015
-
[19]
Dipgame: A testbed for multiagent systems
Angela Fabregues, David Navarro, Alejandro Serrano, and Carles Sierra. Dipgame: A testbed for multiagent systems. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1-Volume 1, pages 1619–1620. International Foundation for Au...
2010
-
[20]
Negotiations over large agreement spaces, 2015
Dave Jonge and Jordi González Sabaté. Negotiations over large agreement spaces, 2015
2015
-
[21]
Learning a game strategy using pattern-weights and self-play
Ari Shapiro, Gil Fuchs, and Robert Levinson. Learning a game strategy using pattern-weights and self-play. In International Conference on Computers and Games, pages 42–60. Springer, 2002
2002
-
[22]
The evolution of cooperation
Robert Axelrod and William D Hamilton. The evolution of cooperation. Science, 211(4489):1390–1396, 1981
1981
-
[23]
Learning to communicate with deep multi-agent reinforcement learning
Jakob Foerster, Ioannis Alexandros Assael, Nando de Freitas, and Shimon Whiteson. Learning to communicate with deep multi-agent reinforcement learning. In Advances in Neural Information Processing Systems, pages 2137–2145, 2016
2016
-
[24]
Intrinsic social motivation via causal influence in multi-agent rl
Natasha Jaques, Angeliki Lazaridou, Edward Hughes, Caglar Gulcehre, Pedro A Ortega, DJ Strouse, Joel Z Leibo, and Nando de Freitas. Intrinsic social motivation via causal influence in multi-agent rl. arXiv preprint arXiv:1810.08647, 2018
-
[25]
Inequity aversion improves cooperation in intertemporal social dilemmas
Edward Hughes, Joel Z Leibo, Matthew Phillips, Karl Tuyls, Edgar Dueñez-Guzman, Anto- nio García Castañeda, Iain Dunning, Tina Zhu, Kevin McKee, Raphael Koster, et al. Inequity aversion improves cooperation in intertemporal social dilemmas. In Advances in Neural Infor- mation ...
2018
-
[26]
Consequentialist conditional cooperation in social dilemmas with imperfect information
Alexander Peysakhovich and Adam Lerer. Consequentialist conditional cooperation in social dilemmas with imperfect information. arXiv preprint arXiv:1710.06975, 2017
2017 arXiv
-
[27]
Behavioural game theory: thinking, learning and teaching
Colin F Camerer, Teck-Hua Ho, and Juin Kuan Chong. Behavioural game theory: thinking, learning and teaching. In Advances in Understanding Strategic Behaviour , pages 120–180. Springer, 2004. 10
2004
-
[28]
Semi-supervised classification with graph convolutional networks
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016
2016 arXiv
-
[29]
Film: Visual reasoning with a general conditioning layer
Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018
2018
-
[30]
Weight normalization: A simple reparameterization to accelerate training of deep neural networks
Tim Salimans and Durk P Kingma. Weight normalization: A simple reparameterization to accelerate training of deep neural networks. In Advances in Neural Information Processing Systems, pages 901–909, 2016
2016
-
[31]
Feature-wise transformations
Vincent Dumoulin, Ethan Perez, Nathan Schucher, Florian Strub, Harm de Vries, Aaron Courville, and Yoshua Bengio. Feature-wise transformations. Distill, 3(7):e11, 2018
2018
-
[32]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016
2016
-
[33]
Daide - clients
David Norman. Daide - clients. http://www.daide.org.uk/clients.html, 2013. Ac- cessed: 2019-05-01
2013
-
[34]
Emergent communication through negotiation
Kris Cao, Angeliki Lazaridou, Marc Lanctot, Joel Z Leibo, Karl Tuyls, and Stephen Clark. Emergent communication through negotiation. arXiv preprint arXiv:1804.03980, 2018
2018 arXiv
-
[35]
Strategic information transmission
Vincent P Crawford and Joel Sobel. Strategic information transmission. Econometrica: Journal of the Econometric Society, pages 1431–1451, 1982
1982
-
[36]
Learning with opponent-learning awareness
Jakob Foerster, Richard Y Chen, Maruan Al-Shedivat, Shimon Whiteson, Pieter Abbeel, and Igor Mordatch. Learning with opponent-learning awareness. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems , pages 122–130. International Foun...
2018
-
[2013]
Accessed: 2019-05-01
2019
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.