pith. sign in

arxiv: 1907.03840 · v1 · pith:PH4WDFWBnew · submitted 2019-07-08 · 💻 cs.AI · cs.NE

Diverse Agents for Ad-Hoc Cooperation in Hanabi

Rodrigo Canaan , Julian Togelius , Andy Nealen , Stefan Menzel This is my paper

Pith reviewed 2026-05-25 00:51 UTC · model grok-4.3

classification 💻 cs.AI cs.NE
keywords Hanabiad-hoc cooperationQuality Diversityagent populationsmulti-agent systemscooperative gamesadaptive agents
0
0 comments X

The pith

Quality Diversity algorithms generate populations of agents for testing ad-hoc cooperation in Hanabi.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to show that Quality Diversity algorithms provide a systematic way to produce varied agent populations that can serve as test partners when evaluating how well new agents cooperate with unknown players in Hanabi. This matters because the game requires each player to infer the intentions of others from limited shared information, yet researchers currently lack agreed-upon methods for creating suitable pools of such partners. An initial generator based on the idea is described, along with possible metrics for judging different population generators and how the resulting agents could support the development of more flexible cooperative strategies.

Core claim

Quality Diversity algorithms are a promising class of algorithms to generate populations for ad-hoc cooperation evaluation in Hanabi. An initial implementation of an agent generator based on this idea is presented, metrics for comparing such generators are discussed, and the generator is positioned as a tool to help build adaptive agents for the game.

What carries the argument

Quality Diversity algorithms, which search for solutions that combine high performance with behavioral diversity across a population.

If this is right

  • The generated populations can function as standardized test sets for measuring how well new agents adapt to unknown partners.
  • Explicit metrics can be defined to compare the quality and diversity of populations produced by different generator methods.
  • Adaptive agents trained or tested against these populations may achieve better performance in scenarios with novel teammates.
  • The approach supplies a concrete starting point for replacing ad-hoc choices of test agents with reproducible generation procedures.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same generator style could be applied to other partially observable cooperative games that reward accurate modeling of partners.
  • Populations produced this way might reveal which behavioral traits most help or hinder cooperation when agents have no shared history.
  • If the populations cover a wide range of strategies, they could serve as a benchmark set for measuring progress toward general ad-hoc cooperation methods.

Load-bearing premise

The agent populations created by the proposed generator will prove useful for evaluating and building agents that must cooperate with previously unseen partners.

What would settle it

An experiment that measures ad-hoc cooperation scores of new agents when partnered with populations from the Quality Diversity generator versus populations drawn from existing hand-designed or random methods.

Figures

Figures reproduced from arXiv: 1907.03840 by Andy Nealen, Julian Togelius, Rodrigo Canaan, Stefan Menzel.

Figure 1
Figure 1. Figure 1: Main results of the MAP-Elites experiment after generating 1 million individuals and reevaluating the elite at each niche for 1000 games each. Columns [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Average pairwise performance of each agent when paired with all 326 valid agents in the pool [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Number of pairings for which each agent in the map is the best partner for the other agent. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Average score of each pair of agent versus the Manhattan Distance [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
read the original abstract

In complex scenarios where a model of other actors is necessary to predict and interpret their actions, it is often desirable that the model works well with a wide variety of previously unknown actors. Hanabi is a card game that brings the problem of modeling other players to the forefront, but there is no agreement on how to best generate a pool of agents to use as partners in ad-hoc cooperation evaluation. This paper proposes Quality Diversity algorithms as a promising class of algorithms to generate populations for this purpose and shows an initial implementation of an agent generator based on this idea. We also discuss what metrics can be used to compare such generators, and how the proposed generator could be leveraged to help build adaptive agents for the game.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 0 minor

Summary. The paper proposes Quality Diversity (QD) algorithms as a promising approach to generate diverse populations of agents for ad-hoc cooperation evaluation in Hanabi. It mentions an initial implementation of a QD-based agent generator, discusses metrics for comparing generators, and suggests leveraging such populations to build adaptive agents.

Significance. If empirically validated, a QD generator that produces agent populations with measurable improvements in diversity and utility for ad-hoc teamwork could address a key methodological gap in Hanabi research and multi-agent systems more broadly. The absence of any quantitative results on population properties or downstream ad-hoc performance means the contribution remains a proposal whose significance cannot yet be assessed.

major comments (2)
  1. [Abstract] Abstract: the central claim that QD algorithms are 'promising' for generating useful populations rests on an 'initial implementation' whose output quality, diversity metrics, or impact on ad-hoc cooperation scores are never quantified or compared to baselines.
  2. [Abstract] Abstract: without any reported results on agent performance distributions, population coverage, or ad-hoc evaluation improvements, it is impossible to evaluate whether the proposed generator satisfies the requirement that it 'produce agent populations useful for evaluating and building adaptive agents'.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments. Our manuscript is a proposal paper that introduces Quality Diversity algorithms as a method for generating diverse agent populations for ad-hoc cooperation in Hanabi, presents an initial implementation to demonstrate feasibility, and discusses relevant metrics and downstream uses. We address the major comments point by point below.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claim that QD algorithms are 'promising' for generating useful populations rests on an 'initial implementation' whose output quality, diversity metrics, or impact on ad-hoc cooperation scores are never quantified or compared to baselines.

    Authors: The assessment of QD algorithms as promising draws from their documented properties in the broader QD literature (diversity maintenance and quality optimization), which directly address the need for varied partners in ad-hoc evaluation. The initial implementation serves only to show that a QD-based generator can be constructed for Hanabi; it is not presented as empirical evidence supporting the claim. We can revise the abstract to make this distinction explicit and to frame the contribution more clearly as a proposal. revision: partial

  2. Referee: [Abstract] Abstract: without any reported results on agent performance distributions, population coverage, or ad-hoc evaluation improvements, it is impossible to evaluate whether the proposed generator satisfies the requirement that it 'produce agent populations useful for evaluating and building adaptive agents'.

    Authors: We agree that quantitative evaluation of the generated population would be required to demonstrate utility for ad-hoc teamwork. The manuscript instead focuses on defining the problem, proposing the QD approach, outlining possible metrics, and sketching how such populations could support adaptive agent construction. No performance numbers are reported because the work is positioned as an initial proposal rather than a completed empirical study. We can add clarifying language in the abstract and conclusion to emphasize the preliminary status of the implementation. revision: yes

Circularity Check

0 steps flagged

No circularity: proposal paper with no derivations or fitted quantities

full rationale

The paper proposes Quality Diversity algorithms for generating Hanabi agent populations and discusses metrics, but presents no equations, derivations, predictions, or fitted parameters. The abstract and described content contain only conceptual claims and an 'initial implementation' mention without quantitative results or self-referential reductions. No load-bearing steps match any enumerated circularity patterns; the work is self-contained as a forward-looking suggestion rather than a closed derivation chain.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract contains no technical details on parameters, axioms or entities.

pith-pipeline@v0.9.0 · 5644 in / 1075 out tokens · 25317 ms · 2026-05-25T00:51:31.246588+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution

    cs.CL 2023-09 unverdicted novelty 8.0

    Promptbreeder evolves both task prompts and the mutation prompts that improve them using LLMs, outperforming Chain-of-Thought and Plan-and-Solve on arithmetic and commonsense reasoning benchmarks.

Reference graph

Works this paper leans on

27 extracted references · 27 canonical work pages · cited by 1 Pith paper · 2 internal anchors

  1. [1]

    Chinook the world man- machine checkers champion,

    J. Schaeffer, R. Lake, P. Lu, and M. Bryant, “Chinook the world man- machine checkers champion,” AI Magazine, vol. 17, no. 1, p. 21, 1996

    work page 1996

  2. [2]

    Deep blue,

    M. Campbell, A. J. Hoane Jr, and F.-h. Hsu, “Deep blue,” Artificial intelligence, vol. 134, no. 1-2, pp. 57–83, 2002

    work page 2002

  3. [3]

    Mastering the game of go with deep neural networks and tree search,

    D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V . Panneershelvam, M. Lanctot et al., “Mastering the game of go with deep neural networks and tree search,” nature, vol. 529, no. 7587, pp. 484–489, 2016

    work page 2016

  4. [4]

    Towards game-based metrics for computational co-creativity,

    R. Canaan, S. Menzel, J. Togelius, and A. Nealen, “Towards game-based metrics for computational co-creativity,” in 2018 IEEE Conference on Computational Intelligence and Games (CIG) . IEEE, 2018, pp. 1–8

    work page 2018

  5. [5]

    Does the chimpanzee have a theory of mind?

    D. Premack and G. Woodruff, “Does the chimpanzee have a theory of mind?” Behavioral and brain sciences, vol. 1, no. 4, pp. 515–526, 1978

    work page 1978

  6. [6]

    Ad hoc autonomous agent teams: Collaboration without pre-coordination,

    P. Stone, G. A. Kaminka, S. Kraus, and J. S. Rosenschein, “Ad hoc autonomous agent teams: Collaboration without pre-coordination,” in Twenty-Fourth AAAI Conference on Artificial Intelligence , 2010

    work page 2010

  7. [7]

    Autonomous agents modelling other agents: A comprehensive survey and open problems,

    S. V . Albrecht and P. Stone, “Autonomous agents modelling other agents: A comprehensive survey and open problems,” Artificial Intelligence, vol. 258, pp. 66–95, 2018

    work page 2018

  8. [8]

    Bayesian action decoder for deep multi- agent reinforcement learning,

    J. N. Foerster, F. Song, E. Hughes, N. Burch, I. Dunning, S. Whiteson, M. Botvinick, and M. Bowling, “Bayesian action decoder for deep multi- agent reinforcement learning,” arXiv preprint arXiv:1811.01458 , 2018

    work page arXiv 2018

  9. [9]

    Illuminating search spaces by mapping elites

    J.-B. Mouret and J. Clune, “Illuminating search spaces by mapping elites,” arXiv preprint arXiv:1504.04909 , 2015

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  10. [10]

    Solving hanabi: Estimating hands by opponent’s actions in cooperative game with incomplete information

    H. Osawa, “Solving hanabi: Estimating hands by opponent’s actions in cooperative game with incomplete information.” in AAAI workshop: Computer Poker and Imperfect Information , 2015, pp. 37–43

    work page 2015

  11. [11]

    Aspects of the cooperative card game hanabi,

    M. J. van den Bergh, A. Hommelberg, W. A. Kosters, and F. M. Spieksma, “Aspects of the cooperative card game hanabi,” in Benelux Conference on Artificial Intelligence . Springer, 2016, pp. 93–105

    work page 2016

  12. [12]

    An intentional ai for hanabi,

    M. Eger, C. Martens, and M. A. C ´ordoba, “An intentional ai for hanabi,” in Computational Intelligence and Games (CIG), 2017 IEEE Conference on. IEEE, 2017, pp. 68–75

    work page 2017

  13. [13]

    Evaluating and modelling hanabi-playing agents,

    J. Walton-Rivers, P. R. Williams, R. Bartle, D. Perez-Liebana, and S. M. Lucas, “Evaluating and modelling hanabi-playing agents,” in Evolutionary Computation (CEC), 2017 IEEE Congress on . IEEE, 2017, pp. 1382–1389

    work page 2017

  14. [14]

    A survey of monte carlo tree search methods,

    C. B. Browne, E. Powley, D. Whitehouse, S. M. Lucas, P. I. Cowling, P. Rohlfshagen, S. Tavener, D. Perez, S. Samothrakis, and S. Colton, “A survey of monte carlo tree search methods,” IEEE Transactions on Computational Intelligence and AI in games , vol. 4, no. 1, pp. 1–43, 2012

    work page 2012

  15. [15]

    Re-determinizing Information Set Monte Carlo Tree Search in Hanabi

    J. Goodman, “Re-determinizing information set monte carlo tree search in hanabi,” arXiv preprint arXiv:1902.06075 , 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1902

  16. [16]

    Hanabi competition results,

    J. Walton-Rivers, “Hanabi competition results,” https://community. fossgalaxy.com/t/hanabi-competition-results/154, 2018, access: 05/14/2018

    work page 2018

  17. [17]

    Evolving agents for the hanabi 2018 cig competition,

    R. Canaan, H. Shen, R. Torrado, J. Togelius, A. Nealen, and S. Menzel, “Evolving agents for the hanabi 2018 cig competition,” in 2018 IEEE Conference on Computational Intelligence and Games (CIG) . IEEE, 2018, pp. 1–8

    work page 2018

  18. [18]

    The hanabi chal- lenge: A new frontier for ai research,

    N. Bard, J. N. Foerster, S. Chandar, N. Burch, M. Lanctot, H. F. Song, E. Parisotto, V . Dumoulin, S. Moitra, E. Hugheset al., “The hanabi chal- lenge: A new frontier for ai research,” arXiv preprint arXiv:1902.00506, 2019

    work page arXiv 1902

  19. [19]

    State of the art hanabi bots + simulation framework in rust,

    J. Wu, “State of the art hanabi bots + simulation framework in rust,” https://github.com/WuTheFWasThat/hanabi.rs, 2016, access: 05/14/2018

    work page 2016

  20. [20]

    How to make the perfect fireworks display: Two strategies for hanabi,

    C. Cox, J. De Silva, P. Deorsey, F. H. Kenter, T. Retter, and J. Tobin, “How to make the perfect fireworks display: Two strategies for hanabi,” Mathematics Magazine, vol. 88, no. 5, pp. 323–336, 2015

    work page 2015

  21. [21]

    Playing hanabi near-optimally,

    B. Bouzy, “Playing hanabi near-optimally,” in Advances in Computer Games. Springer, 2017, pp. 51–62

    work page 2017

  22. [22]

    Quality diversity: A new frontier for evolutionary computation,

    J. K. Pugh, L. B. Soros, and K. O. Stanley, “Quality diversity: A new frontier for evolutionary computation,” Frontiers in Robotics and AI , vol. 3, p. 40, 2016

    work page 2016

  23. [23]

    Abandoning objectives: Evolution through the search for novelty alone,

    J. Lehman and K. O. Stanley, “Abandoning objectives: Evolution through the search for novelty alone,” Evolutionary computation, vol. 19, no. 2, pp. 189–223, 2011

    work page 2011

  24. [24]

    Multi-objective optimization,

    K. Deb, “Multi-objective optimization,” in Search methodologies . Springer, 2014, pp. 403–449

    work page 2014

  25. [25]

    Robots that can adapt like animals,

    A. Cully, J. Clune, D. Tarapore, and J.-B. Mouret, “Robots that can adapt like animals,” Nature, vol. 521, no. 7553, p. 503, 2015

    work page 2015

  26. [26]

    Error detecting and error correcting codes,

    R. W. Hamming, “Error detecting and error correcting codes,” The Bell system technical journal , vol. 29, no. 2, pp. 147–160, 1950

    work page 1950

  27. [27]

    Re-evaluating evaluation,

    D. Balduzzi, K. Tuyls, J. Perolat, and T. Graepel, “Re-evaluating evaluation,” in Advances in Neural Information Processing Systems , 2018, pp. 3268–3279

    work page 2018