Pith. sign in

REVIEW 3 major objections 5 minor 63 references

Augmenting the action space with conventions to improve multi-agent cooperation in Hanabi

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By encoding human Hanabi conventions as optional cooperative actions, multi-agent Rainbow reaches higher converged scores for 3-5 players and learns to cooperate with new partners.

desk verdict Genuinely new convention formalism for MARL, but the performance numbers are provisional until the baseline is retrained under the same protocol. read the letter →

arxiv 2412.06333 v3 pith:PAJIZGHB submitted 2024-12-09 cs.MA cs.AIcs.LG

F. Bredell , H.A. Engelbrecht , J.C. Schoeman This is my paper · ORCID
classification cs.MAcs.AIcs.LG
keywords multi-agentreinforcementlearningHanabiconventionsactionspaceaugmentationself-playcross-playRainbowcooperativeagents
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the bottleneck in multi-agent Hanabi is not only network architecture, and that borrowing the conventions human players use can substitute for much of the complexity and training data. The authors formalise a convention as a sequence of cooperative actions with entry, continuation, and termination conditions, append these convention-steps to the agent's primitive action space, and let each agent opt in or opt out based on its own observation. On the standard Hanabi benchmark, convention-augmented multi-agent Rainbow reaches self-play scores of 20.65/25 (2P), 20.32 (3P), 20.09 (4P), and 19.05 (5P) against baseline scores of 20.64, 18.71, 18.00, and 15.26, with substantially faster training. Cross-play with independently trained partners jumps from 2.91/25 to 17.02/25 in two players and reaches 18.60, 18.56, and 17.69 in three, four, and five players. If correct, the paper establishes a cheap, transferable way to inject shared domain knowledge into partially observable multi-agent learning.

What carries the argument

The central object is the artificial convention: a tuple $\langle m_k, \lambda^1_k, \pi^1_k, \ldots, \lambda^{m_k}_k, \pi^{m_k}_k \rangle$ with a step count, per-step observability conditions, and per-step policies that map a selected convention-step to an environment action. The machinery works through action masking and a translation layer: the conditions $\lambda$ define which convention-steps are legal in the current observation, the agent selects among those steps with its usual value network, and the translation layer executes $\pi$ on the observation to produce the actual Hanabi move. This is what carries the argument, because it converts implicit communication into ordinary discrete decisions without adding a communication channel and without altering the Dec-POMDP (decentralised partially observable Markov decision process) the agents face.

What would settle it

Run convention-augmented Rainbow in five-player Hanabi while replacing all convention policies with random legal actions under the same action mask; if scores stay near 19/25 the gain comes from the larger action space rather than from convention semantics, and if scores fall back toward the 15.26 baseline the semantics are doing the work. A second check is to remove the record of other players' recent actions from the observation vector; the paper's opt-in mechanism predicts that the self-play and cross-play gains should largely disappear.

Watch

Extended reading notes

Core claim

The paper's discovery is that human Hanabi conventions can be lifted into an action-space construct without changing the environment. A convention is formalised as $c_k = \langle m_k, \lambda^1_k, \pi^1_k, \ldots, \lambda^{m_k}_k, \pi^{m_k}_k \rangle$, where each $\lambda$ is a condition on the agent's observation and each $\pi$ is a policy that turns the chosen convention-step into a legal Hanabi action. Once these convention-steps are appended to the primitive actions, the Q-network only sees a larger masked action space; the extra translation layer is what gives the agent a shared, public interpretation of why an action was taken. The authors report that this shared public layer is what makes the difference: self-play converges faster, converged scores rise for three-to five-player Hanabi, and independently trained agents from the same convention list can still cooperate, which ordinary self-play Rainbow agents cannot.

Load-bearing premise

The load-bearing premise is that every agent's observation carries enough information to tell which convention is currently active and which convention-steps are legal; if that observability fails, or if one of the hand-written trigger conditions fires at the wrong moment, agents will opt into steps whose translated game actions are wrong and the cooperation gain disappears.

Editorial extensions

If this is right

  • In self-play, convention-augmented Rainbow beats baseline Rainbow at 3, 4, and 5 players (20.32 vs 18.71, 20.09 vs 18.00, and 19.05 vs 15.26), and trains about five times faster at five players, needing under 30 million steps rather than billions.
  • In cross-play, agents trained independently on the same convention list score 17.02/25 at two players against 2.91 for baseline Rainbow, and 18.60, 18.56, and 17.69 at three, four, and five players, so shared conventions are enough to produce zero-shot coordination without the other-play training objective.
  • Because the convention layer sits between the observation and the action output, the paper argues the augmentation transfers to any MARL method, not just Rainbow.
  • The remaining bottleneck identified in the paper is memory: agents select convention-steps about 70% of the time but fail to learn some conventions like The Chop (the rule to discard the oldest unmarked card), which the authors attribute to the feed-forward network's lack of memory.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The mechanism predicts that an ablation replacing convention policies with random legal actions under the same mask would collapse scores toward the baseline, while mere extra action capacity would not; this would pin the gain to convention semantics.
  • The two-player pattern (no self-play gain, largest cross-play drop) points to the multi-step Prompt and Finesse conventions, which require a third player's hand to be visible, as the engine of the improvement; adding recurrent memory should recover memory-dependent conventions such as The Chop and push two-player scores higher.
  • If the approach transfers beyond Hanabi, the natural targets are turn-based partially observable domains with an existing human convention list and observable partner actions, since the mechanism requires exactly those two ingredients.
  • A direct extension the authors flag is convention discovery: letting agents propose new convention-steps during training and then testing whether discovered conventions generalise across runs would show whether the hand-authored list is special or merely a good prior.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes augmenting a MARL agent's action space with "conventions": hand-authored, multi-step, cooperative actions that agents can initiate and subscribe to, each defined by conditions and policies that translate convention-steps into environment actions. The authors formalize conventions as an extension of the Dec-POMDP action space, integrate them into independent Q-learning and Rainbow, and evaluate on Small Hanabi and full 2-5 player Hanabi. They report improved self-play scores and faster training for Rainbow with conventions compared to the published Rainbow baseline of Bard et al., as well as improved cross-play scores across player counts. The paper argues that conventions enable implicit coordination without additional communication, and that the approach is a step toward convention discovery.

Significance. If the central claim is upheld, the method is a practical, low-cost way to inject domain knowledge into MARL: it does not change the reward, observation, or core algorithm, only the action space and two network layers, and the code is publicly available. The formalization of conventions as opt-in multi-step actions is clearly presented and distinguishes the approach from options. The empirical claims are currently stronger than the evidence: the self-play comparison uses a published baseline that was not retrained under the same protocol, and the cross-play comparison selects the best of 10 pairings. These protocol choices affect the quantitative magnitude of the reported gains, so the significance of the result depends on the follow-up experiments requested below.

major comments (3)
  1. [Section 5.1, Table 1, Fig. 4] The central self-play claim is not supported by a controlled comparison. Table 1 and Fig. 4 compare Rainbow with conventions, reported as the best of 10 runs (Section 5.1), against the published Rainbow numbers of Bard et al. [37], which were not retrained in the same codebase, with the same hyperparameter search, the same evaluation episode count, or the same run-selection rule. The text's statement that Rainbow has little run-to-run variance (Section 5) is not a substitute for reporting the 10-run distribution. Because the 2P comparison is already statistically indistinguishable (p=0.97, Table C4), the 3P/4P/5P gaps of 1.6/2.1/3.8 points cannot be attributed to the action-space augmentation until a same-compute baseline is trained and evaluated under an identical protocol, and the best-of-10 selection is applied symmetrically or replaced by run-level distributions. The Small Hanabi control (Section 4.3) does not fill this gap, since it compares different action-space sizes and partly uses an in-house environment.
  2. [Section 5.1, Table 3] The cross-play claim is inflated by selection. The text states that the cross-play pairing was chosen as the combination of agents that performed the best on average from 10 separately trained agents, and Table 3 reports only that best pair. Zero-shot coordination is about performance with arbitrary unseen partners; selecting the best pair measures the upper tail, not the typical or expected cross-play score. The paper should report the full pairwise matrix, or a fixed random-pair protocol, for all runs, and compare with baselines evaluated under the same pairing rule. Without this, the claim that conventions "allow agents to cooperate with never-before-seen teammates" is not quantified.
  3. [Section 4.2 / Appendix A.2] The mechanism depends on the hand-coded conditions lambda_k being correctly computable from the agent's observation, and on the observation containing enough history for the multi-step conventions. Section 4.2 asserts that Hanabi includes other agents' actions in the observation, but the Prompt and Finesse conventions (Table A2, k=10 and 11) require an agent to recognize that a play hint was "incorrect" and to distinguish a prompt from a finesse, which is an inference over previous actions and current hand knowledge. Section 5.2 simultaneously states that Rainbow has no memory of past observations. The paper should verify the convention layer explicitly: report how often each lambda_k fires, how often the resulting action is legal, and ideally compare against a variant in which the convention policies are executed by a hand-coded oracle. If the conditions misfire, convention-steps translate into wrong environment actions, and the reported gains would not transfer to other observation encodings or environments.
minor comments (5)
  1. [Section 1] The phrase "does not fair well" should be "does not fare well".
  2. [Table B3] Rainbow's training exploration epsilon_t is listed as 0.00 while the decay description says exploration starts at 1 and ends at epsilon_t; this is inconsistent and should be clarified.
  3. [Section 5.2 and Conclusion] Section 5.2 states that conventions require 30 million training steps, while the Conclusion says the five-player cross-play agents were trained on 50 million steps; the training budgets for Fig. 4 and Table 1 are not stated. Please add a table of training steps for each player count and use consistent numbers.
  4. [Fig. 3] The text uses "7,6/10" with a European decimal comma while the rest of the paper uses decimal points; please standardize the notation.
  5. [Section 5.1] The term "multi-agent Rainbow" is used in the introduction, but the experiments use a shared-policy independent Rainbow with a shared Q-function; this distinction should be stated clearly when the experimental setup is introduced.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the convention-augmented action space is a hand-coded intervention whose reported gains are empirical training outcomes, not quantities derived from the method's own definitions.

full rationale

The paper's central claim is that augmenting Rainbow's action space with hand-coded human conventions improves self-play and cross-play performance in Hanabi. This is an empirical claim validated by training curves and evaluation scores, not a quantity derived by construction from the formalism. Section 3.1 defines a convention c_k as a tuple of conditions and policies, and Algorithm 2 shows how the agent selects a convention-step and maps it to an environment action; nothing in these definitions forces the reported scores of 20.65 (2P), 20.32 (3P), 20.09 (4P), and 19.05 (5P). The conventions encode expert knowledge, which is the intended mechanism of the proposed method, and the paper explicitly frames this as incorporating domain knowledge rather than as a prediction from theory. No parameter is fitted to the target scores, and no load-bearing result is imported from the authors' own prior work; the baseline numbers come from Bard et al. [37], an external source. The most serious methodological concern is that the baseline Rainbow results are taken from published numbers while the convention-augmented results are best-of-10 selections, which could inflate the reported uplift, but that is a benchmarking-fairness and correctness risk, not circularity under the definitions used here. The 2-player self-play comparison is statistically indistinguishable (p=0.9676, Table C4), which further indicates that the method does not trivially guarantee improvement by construction. The paper therefore contains no self-definitional reduction, no fitted-input-as-prediction, and no self-citation chain that forces the conclusion.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The paper introduces no new physical entities. Its free parameters are hyperparameters and a post hoc convention subset. The load-bearing axioms are the observability of other agents' actions and the validity of H-Group conventions as expert prior knowledge.

free parameters (2)
  • Rainbow hyperparameters (Table B3) = alpha=2.5e-5, gamma=0.99, replay=50000, batch=32, target update=500, atoms=51, n=1
    Selected by parameter sweeps; the paper claims they match the Bard et al. baseline, but sweeps are a form of fitting that can inflate reported performance.
  • Small Hanabi convention subset = kept conventions 0-3, removed conventions 4-7
    Post hoc simplification claimed to have minor impact; this selection is a design choice that affects the augmented-action results in Fig. 3.
assumptions (6)
  • domain assumption Game rules of Hanabi as implemented in the DeepMind environment
    Result depends on the standard Hanabi rules and scoring; the environment is the testbed for the claims.
  • domain assumption Agents can observe other players' most recent actions in their observation vector
    Section 4.2 states this; convention detection and subscription depend on it.
  • domain assumption H-Group conventions are valid, near-optimal Hanabi strategies
    Appendix A.2 adopts these as the source of convention policies; if they were poor strategies, the performance claim would collapse.
  • domain assumption Shared-policy training is a fair proxy for independent agents
    Section 5.1 uses one shared Q-function; the authors argue there is no state sharing, but this is a modeling choice.
  • standard math Welch's t-test assumptions hold for the 1000-episode evaluation samples
    Used for significance claims; assumes independent samples and approximate normality of mean scores.
  • ad hoc to paper Convention policies can be represented as a deterministic mapping from observation to game action
    The paper defines each convention step with a fixed policy pi; if stochasticity or asymmetric information broke this mapping, the action translation layer would fail.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Augmenting the action space with conventions to improve multi-agent cooperation in Hanabi." pith.science (2026). https://pith.science/paper/PAJIZGHB

@misc{pith2026241206333,
  author       = {Pith},
  title        = {Pith review of: Augmenting the action space with conventions to improve multi-agent cooperation in Hanabi},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PAJIZGHB}},
  note         = {Machine review of arXiv:2412.06333}
}
read the original abstract

The card game Hanabi is considered a strong medium for the testing and development of multi-agent reinforcement learning (MARL) algorithms, due to its cooperative nature, partial observability, limited communication and remarkable complexity. Previous research efforts have explored the capabilities of MARL algorithms within Hanabi, focusing largely on advanced architecture design and algorithmic manipulations to achieve state-of-the-art performance for various number of cooperators. However, this often leads to complex solution strategies with high computational cost and requiring large amounts of training data. For humans to solve the Hanabi game effectively, they require the use of conventions, which often allows for a means to implicitly convey ideas or knowledge based on a predefined, and mutually agreed upon, set of "rules" or principles. Multi-agent problems containing partial observability, especially when limited communication is present, can benefit greatly from the use of implicit knowledge sharing. In this paper, we propose a novel approach to augmenting an agent's action space using conventions, which act as a sequence of special cooperative actions that span over and include multiple time steps and multiple agents, requiring agents to actively opt in for it to reach fruition. These conventions are based on existing human conventions, and result in a significant improvement on the performance of existing techniques for self-play and cross-play for various number of cooperators within Hanabi.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

63 extracted references · 42 canonical work pages

  1. [37]

    Artificial Intelligence 280, 103216 (2020) https://doi.org/10

    Bard, N., Foerster, J.N., Chandar, S., Burch, N., Lanctot, M., Song, H.F., Parisotto, E., Dumoulin, V., Moitra, S., Hughes, E., Dunning, I., Mourad, S., Larochelle, H., Bellemare, M.G., Bowling, M.: The Hanabi challenge: A new fron- tier for AI research. Artificial Intelligence 280, 103216 (2020) https://doi.org/10. 1016/j.artint.2019.103216

    arXiv 2020

  2. [1]

    (eds.) Multi-agent Reinforcement Learning: An Overview, pp

    Bu¸ soniu, L., Babuˇ ska, R., De Schutter, B.: In: Srinivasan, D., Jain, L.C. (eds.) Multi-agent Reinforcement Learning: An Overview, pp. 183–221. Springer, Berlin, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14435-6 7

    work page doi:10.1007/978-3-642-14435-6 2010

  3. [2]

    IEEE Transactions on Industrial Informatics 9(1), 427–438 (2013) https://doi.org/10.1109/TII.2012.2219061

    Cao, Y., Yu, W., Ren, W., Chen, G.: An overview of recent progress in the study of distributed multi-agent coordination. IEEE Transactions on Industrial Informatics 9(1), 427–438 (2013) https://doi.org/10.1109/TII.2012.2219061

    work page arXiv 2013

  4. [3]

    arXiv preprint arXiv:1709.06011 (2017) https://doi.org/10

    H¨ uttenrauch, M.,ˇSoˇ si´ c, A., Neumann, G.: Guided deep reinforcement learning for swarm systems. arXiv preprint arXiv:1709.06011 (2017) https://doi.org/10. 48550/arXiv.1709.06011

    arXiv doi:10.48550/arxiv.1709.06011 2017

  5. [4]

    In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pp

    Branavan, S.R., Chen, H., Zettlemoyer, L., Barzilay, R.: Reinforcement learning 34 for mapping instructions to actions. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pp. 82–90 (2009)

    work page 2009

  6. [5]

    IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 38(2), 156–172 (2008) https://doi.org/10

    Busoniu, L., Babuska, R., De Schutter, B.: A comprehensive survey of multiagent reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 38(2), 156–172 (2008) https://doi.org/10. 1109/TSMCC.2007.913919

    arXiv 2008

  7. [6]

    PloS one 12(4), 0172395 (2017) https://doi.org/10.1371/journal.pone

    Tampuu, A., Matiisen, T., Kodelja, D., Kuzovkin, I., Korjus, K., Aru, J., Aru, J., Vicente, R.: Multiagent cooperation and competition with deep reinforcement learning. PloS one 12(4), 0172395 (2017) https://doi.org/10.1371/journal.pone. 0172395

    doi:10.1371/journal.pone 2017

  8. [7]

    AAAI/IAAI 1998(746-752), 2 (1998)

    Claus, C., Boutilier, C.: The dynamics of reinforcement learning in cooperative multiagent systems. AAAI/IAAI 1998(746-752), 2 (1998)

    work page 1998

Show all 63 references
  1. [8]

    The Knowledge Engineering Review 27(1), 1–31 (2012) https://doi.org/10.1017/ S0269888912000057

    Matignon, L., Laurent, G.J., Le Fort-Piat, N.: Independent reinforcement learn- ers in cooperative Markov games: a survey regarding coordination problems. The Knowledge Engineering Review 27(1), 1–31 (2012) https://doi.org/10.1017/ S0269888912000057

    2012

  2. [9]

    In: AAAI Fall Symposia, vol

    Hausknecht, M.J., Stone, P.: Deep recurrent Q-Learning for partially observable MDPs. In: AAAI Fall Symposia, vol. 45, p. 141 (2015)

    2015

  3. [10]

    arXiv preprint arXiv:1706.05296 (2017) https://doi.org/10.48550/arXiv.1706.05296

    Sunehag, P., Lever, G., Gruslys, A., Czarnecki, W.M., Zambaldi, V., Jaderberg, M., Lanctot, M., Sonnerat, N., Leibo, J.Z., Tuyls, K., Graepel, T.: Value- decomposition networks for cooperative multi-agent learning. arXiv preprint arXiv:1706.05296 (2017) https://doi.org/10.4855...

    2017 doi arXiv

  4. [11]

    Journal of Machine Learning Research 21(178), 1–51 (2020)

    Rashid, T., Samvelyan, M., Witt, C.S., Farquhar, G., Foerster, J., Whiteson, S.: Monotonic value function factorisation for deep multi-agent reinforcement learning. Journal of Machine Learning Research 21(178), 1–51 (2020)

    2020

  5. [12]

    In: Advances in Neural Information Processing Systems, vol

    Foerster, J., Assael, I.A., Freitas, N., Whiteson, S.: Learning to communicate with deep multi-agent reinforcement learning. In: Advances in Neural Information Processing Systems, vol. 29 (2016). https://proceedings.neurips.cc/paper files/ paper/2016/file/c7635bfd99248a2cdef82...

    2016

  6. [13]

    In: International Conference on Learning Representations (2020)

    Hu, H., Foerster, J.N.: Simplified action decoder for deep multi-agent rein- forcement learning. In: International Conference on Learning Representations (2020)

    2020

  7. [14]

    In: Proceedings of the AAAI Conference on Artificial Intelligence, vol

    Lerer, A., Hu, H., Foerster, J., Brown, N.: Improving policies via search in coop- erative partially observable games. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 7187–7194 (2020). https://doi.org/10.1609/ aaai.v34i05.6208 35

    2020

  8. [15]

    In: Icml, vol

    Ng, A.Y., Harada, D., Russell, S.: Policy invariance under reward transformations: Theory and application to reward shaping. In: Icml, vol. 99, pp. 278–287 (1999)

    1999

  9. [16]

    In: ICML, vol

    Randløv, J., Alstrøm, P.: Learning to drive a bicycle using reinforcement learning and shaping. In: ICML, vol. 98, pp. 463–471 (1998)

    1998

  10. [17]

    In: Tenth International Conference on Autonomous Agents and Multi-Agent Systems, pp

    Devlin, S., Kudenko, D.: Theoretical considerations of potential-based reward shaping for multi-agent systems. In: Tenth International Conference on Autonomous Agents and Multi-Agent Systems, pp. 225–232 (2011). ACM

    2011

  11. [18]

    arXiv preprint arXiv:1611.05397 (2016) https://doi.org/10.48550/arXiv.1611

    Jaderberg, M., Mnih, V., Czarnecki, W.M., Schaul, T., Leibo, J.Z., Silver, D., Kavukcuoglu, K.: Reinforcement learning with unsupervised auxiliary tasks. arXiv preprint arXiv:1611.05397 (2016) https://doi.org/10.48550/arXiv.1611. 05397

    2016 doi arXiv

  12. [19]

    arXiv preprint arXiv:1611.03673 (2016) https://doi.org/10

    Mirowski, P., Pascanu, R., Viola, F., Soyer, H., Ballard, A.J., Banino, A., Denil, M., Goroshin, R., Sifre, L., Kavukcuoglu, K., et al.: Learning to navigate in com- plex environments. arXiv preprint arXiv:1611.03673 (2016) https://doi.org/10. 48550/arXiv.1611.03673

    2016 doi arXiv

  13. [20]

    Artificial Intelligence 112(1), 181–211 (1999) https://doi.org/10.1016/S0004-3702(99)00052-1

    Sutton, R.S., Precup, D., Singh, S.: Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence 112(1), 181–211 (1999) https://doi.org/10.1016/S0004-3702(99)00052-1

    1999 doi

  14. [21]

    Nature 529(7587), 484–489 (2016)

    Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al.: Mastering the game of Go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)

    2016

  15. [22]

    Communications of the ACM 38(3), 58–68 (1995)

    Tesauro, G.: Temporal difference learning and TD-Gammon. Communications of the ACM 38(3), 58–68 (1995)

    1995

  16. [23]

    arXiv preprint arXiv:1912.06680 (2019) https://doi.org/10.48550/arXiv.1912.06680

    Berner, C., Brockman, G., Chan, B., Cheung, V., Debiak, P., Dennison, C., Farhi, D., Fischer, Q., Hashme, S., Hesse, C., J´ ozefowicz, R., Gray, S., Olsson, C., Pachocki, J., Petrov, M., Pinto, H.P.d.O., Raiman, J., Salimans, T., Schlatter, J., Schneider, J., Sidor, S., Sutske...

    2019 doi arXiv

  17. [24]

    Premack, D., Woodruff, G.: Does the chimpanzee have a theory of mind? Behavioral and Brain Science 1, 515–526 (1978)

    1978

  18. [25]

    Blackwell Publishers Ltd, Hoboken, New Jersey (2008)

    Lewis, D.: Convention: A Philosophical Study, 1st edn. Blackwell Publishers Ltd, Hoboken, New Jersey (2008)

    2008

  19. [26]

    In: International Conference on Machine Learning, pp

    Foerster, J., Song, F., Hughes, E., Burch, N., Dunning, I., Whiteson, S., Botvinick, M., Bowling, M.: Bayesian action decoder for deep multi-agent reinforcement 36 learning. In: International Conference on Machine Learning, pp. 1942–1951 (2019). PMLR

    2019

  20. [27]

    arXiv preprint arXiv:1712.01815 (2017) https: //doi.org/10.48550/arXiv.1712.01815

    Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., Lillicrap, T., Simonyan, K., Hassabis, D.: Mastering Chess and Shogi by Self-Play with a General Rein- forcement Learning Algorithm. arXiv preprint a...

    2017 doi arXiv

  21. [28]

    In: International Conference on Machine Learning, pp

    Hu, H., Lerer, A., Peysakhovich, A., Foerster, J.: ”Other-Play” for zero-shot coordination. In: International Conference on Machine Learning, pp. 4399–4410 (2020). PMLR

    2020

  22. [29]

    In: Thirty-second AAAI Conference on Artificial Intelligence (2018)

    Hessel, M., Modayil, J., Van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., Horgan, D., Piot, B., Azar, M., Silver, D.: Rainbow: Combining improvements in deep reinforcement learning. In: Thirty-second AAAI Conference on Artificial Intelligence (2018). https://doi.org/10....

    2018 doi

  23. [30]

    Energy 238, 121958 (2022) https://doi.org/10.1016/j.energy.2021.121958

    Harrold, D.J.B., Cao, J., Fan, Z.: Data-driven battery operation for energy arbi- trage using rainbow deep reinforcement learning. Energy 238, 121958 (2022) https://doi.org/10.1016/j.energy.2021.121958

    2022

  24. [31]

    International Journal of Electrical Power & Energy Systems 119, 105928 (2020) https://doi.org/10.1016/j.ijepes.2020

    Yang, J., Yang, M., Wang, M., Du, P., Yu, Y.: A deep reinforcement learning method for managing wind farm uncertainties through energy storage system control and external reserve purchasing. International Journal of Electrical Power & Energy Systems 119, 105928 (2020) https://...

    2020 doi

  25. [32]

    Sustainability 14(3), 1884 (2022) https://doi.org/10.3390/su14031884

    Wang, R., Chen, Z., Xing, Q., Zhang, Z., Zhang, T.: A modified rainbow-based deep reinforcement learning method for optimal scheduling of charging station. Sustainability 14(3), 1884 (2022) https://doi.org/10.3390/su14031884

    2022 doi

  26. [33]

    Applied Energy 318, 119151 (2022) https://doi.org/10.1016/j.apenergy.2022.119151

    Harrold, D.J.B., Cao, J., Fan, Z.: Renewable energy integration and microgrid energy trading using multi-agent deep reinforcement learning. Applied Energy 318, 119151 (2022) https://doi.org/10.1016/j.apenergy.2022.119151

    2022

  27. [34]

    arXiv preprint arXiv:1906.10124 (2019) https: //doi.org/10.48550/ARXIV.1906.10124

    Zhao, Y., Borovikov, I., Rupert, J., Somers, C., Beirami, A.: On multi-agent learning in team sports games. arXiv preprint arXiv:1906.10124 (2019) https: //doi.org/10.48550/ARXIV.1906.10124

    2019 doi arXiv

  28. [35]

    In: International Conference on Machine Learning, pp

    Ceron, J.S.O., Castro, P.S.: Revisiting rainbow: Promoting more insightful and inclusive deep reinforcement learning research. In: International Conference on Machine Learning, pp. 1373–1383 (2021). PMLR

    2021

  29. [36]

    arXiv preprint arXiv:1908.04683 (2019) https://doi.org/10.48550/arXiv.1908.04683 37

    Toromanoff, M., Wirbel, E., Moutarde, F.: Is deep reinforcement learning really superhuman on atari? leveling the playing field. arXiv preprint arXiv:1908.04683 (2019) https://doi.org/10.48550/arXiv.1908.04683 37

    2019 doi arXiv

  30. [38]

    Mathematics Magazine 88(5), 323–336 (2015)

    Cox, C., De Silva, J., Deorsey, P., Kenter, F.H., Retter, T., Tobin, J.: How to make the perfect fireworks display: Two strategies for hanabi. Mathematics Magazine 88(5), 323–336 (2015)

    2015

  31. [39]

    [Online; accessed 17 July 2024]

    Wu, J.: Github - wuthefwasthat/hanabi.rs: Hanabi simulation in rust. [Online; accessed 17 July 2024]. https://github.com/WuTheFWasThat/hanabi.rs

    2024

  32. [40]

    Science 365(6456), 885–890 (2019) https://doi.org/10.1126/science.aay2400

    Brown, N., Sandholm, T.: Superhuman ai for multiplayer poker. Science 365(6456), 885–890 (2019) https://doi.org/10.1126/science.aay2400

    2019 doi

  33. [41]

    Advances in neural information processing systems 35, 24611–24624 (2022)

    Yu, C., Velu, A., Vinitsky, E., Gao, J., Wang, Y., Bayen, A., WU, Y.: The sur- prising effectiveness of ppo in cooperative multi-agent games. Advances in neural information processing systems 35, 24611–24624 (2022)

    2022

  34. [42]

    Advances in neural information processing systems 30 (2017)

    Lowe, R., Wu, Y.I., Tamar, A., Harb, J., Pieter Abbeel, O., Mordatch, I.: Multi- agent actor-critic for mixed cooperative-competitive environments. Advances in neural information processing systems 30 (2017)

    2017

  35. [43]

    Nature 575(7782), 350–354 (2019)

    Vinyals, O., Babuschkin, I., Czarnecki, W.M., Mathieu, M., Dudzik, A., Chung, J., Choi, D.H., Powell, R., Ewalds, T., Georgiev, P., et al.: Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature 575(7782), 350–354 (2019)

    2019

  36. [44]

    In: Proceedings of the AAAI Conference on Artificial Intelligence, vol

    Kurach, K., Raichuk, A., Stanczyk, P., Zajac, M., Bachem, O., Espeholt, L., Riquelme, C., Vincent, D., Michalski, M., Bousquet, O., et al.: Google research football: A novel reinforcement learning environment. In: Proceedings of the AAAI Conference on Artificial Intelligence, ...

    2020 doi

  37. [45]

    In: International Conference on Learning Representation (2021)

    Wang, J., Ren, Z., Liu, T., Yu, Y., Zhang, C.: Qplex: Duplexdueling multi-agent q-learning. In: International Conference on Learning Representation (2021)

    2021

  38. [46]

    In: International Conference on Learning Representation (2021)

    Wang, T., Gupta, T., Mahajan, A., Peng, B., Whiteson, S., Zhang, C.: Rode: Learning roles to decompose multi-agent tasks. In: International Conference on Learning Representation (2021)

    2021

  39. [47]

    Journal of Artificial Intelligence Research 64, 817–859 (2019) https://doi.org/10.1613/jair.1.11418

    Amato, C., Konidaris, G., Kaelbling, L.P., How, J.P.: Modeling and planning with macro-actions in decentralized pomdps. Journal of Artificial Intelligence Research 64, 817–859 (2019) https://doi.org/10.1613/jair.1.11418

    2019 doi

  40. [48]

    Advances in Neural Information Processing Systems 35, 30406–30418 (2022)

    Chen, J., Chen, J., Lan, T., Aggarwal, V.: Scalable multi-agent covering option 38 discovery based on kronecker graphs. Advances in Neural Information Processing Systems 35, 30406–30418 (2022)

    2022

  41. [49]

    Advances in Neural Information Processing Systems 34, 14502–14515 (2021)

    Strouse, D., McKee, K., Botvinick, M., Hughes, E., Everett, R.: Collaborating with humans without human data. Advances in Neural Information Processing Systems 34, 14502–14515 (2021)

    2021

  42. [50]

    Advances in neural information processing systems 32 (2019)

    Carroll, M., Shah, R., Ho, M.K., Griffiths, T., Seshia, S., Abbeel, P., Dragan, A.: On the utility of learning about humans for human-ai coordination. Advances in neural information processing systems 32 (2019)

    2019

  43. [51]

    1, 2nd edn

    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction vol. 1, 2nd edn. The MIT Press, Cambridge, Massachusetts (2018)

    2018

  44. [52]

    1, 1st edn

    Oliehoek, F.A., Amato, C.: A Concise Introduction to Decentralized POMDPs vol. 1, 1st edn. Springer, Cham, Switzerland (2016)

    2016

  45. [53]

    cooperative agents

    Tan, M.: Multi-agent reinforcement learning: Independent vs. cooperative agents. In: Proceedings of the Tenth International Conference on Machine Learning, pp. 330–337 (1993)

    1993

  46. [54]

    Nature 518(7540), 529–533 (2015)

    Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)

    2015

  47. [55]

    Machine learning 8(3), 293–321 (1992) https://doi.org/10

    Lin, L.J.: Self-improving reactive agents based on reinforcement learning, plan- ning and teaching. Machine learning 8(3), 293–321 (1992) https://doi.org/10. 1007/BF00992699

    1992

  48. [56]

    arXiv preprint arXiv:1412.6980 (2017) https://doi.org/10.48550/ARXIV.1412.6980

    Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2017) https://doi.org/10.48550/ARXIV.1412.6980

    2017 doi arXiv

  49. [57]

    In: International Conference on Machine Learning, pp

    Machado, M.C., Bellemare, M.G., Bowling, M.: A laplacian framework for option discovery in reinforcement learning. In: International Conference on Machine Learning, pp. 2295–2304 (2017). PMLR

    2017

  50. [58]

    Nature 588(7839), 604–609 (2020)

    Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T.,et al.: Mastering atari, go, chess and shogi by planning with a learned model. Nature 588(7839), 604–609 (2020)

    2020

  51. [59]

    IEEE communications surveys & tutorials 21(4), 3133–3174 (2019) https://doi.org/10.1109/COMST.2019.2916583

    Luong, N.C., Hoang, D.T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., Kim, D.I.: Applications of deep reinforcement learning in communications and net- working: A survey. IEEE communications surveys & tutorials 21(4), 3133–3174 (2019) https://doi.org/10.1109/COMST.2019.2916583

    2019

  52. [60]

    IEEE Communications surveys & tutorials 21(3), 2224–2287 39 (2019) https://doi.org/10.1109/COMST.2019.2904897

    Zhang, C., Patras, P., Haddadi, H.: Deep learning in mobile and wireless net- working: A survey. IEEE Communications surveys & tutorials 21(3), 2224–2287 39 (2019) https://doi.org/10.1109/COMST.2019.2904897

    2019

  53. [61]

    The Annals of Statistics 48(4), 1875–1897 (2020)

    Schmidt-Hieber, J.: Nonparametric regression using deep neural networks with ReLU activation function. The Annals of Statistics 48(4), 1875–1897 (2020)

    2020

  54. [62]

    Neurocomputing 190, 82–94 (2016) https://doi.org/10

    Kraemer, L., Banerjee, B.: Multi-agent reinforcement learning as a rehearsal for decentralized planning. Neurocomputing 190, 82–94 (2016) https://doi.org/10. 1016/j.neucom.2016.01.031

    2016

  55. [63]

    Biometrika 34(1-2), 28–35 (1947) https://doi.org/ 10.1093/biomet/34.1-2.28 40

    Welch, B.L.: The generalization of ‘student’s’problem when several different pop- ulation varlances are involved. Biometrika 34(1-2), 28–35 (1947) https://doi.org/ 10.1093/biomet/34.1-2.28 40

    1947 doi

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.