Pith. sign in

REVIEW 4 major objections 7 minor 40 references

Learning to Cooperate with Humans using Generative Agents

T0 review · 4 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Training an AI teammate against a generative model of partner behavior, rather than against a fixed simulated population or cloned human proxy, improves zero-shot coordination with real humans in Overcooked.

desk verdict GAMMA is a genuinely useful technique for training cooperative agents with generative partner sampling, backed by a real human evaluation, but the 'consistent improvement' claim outruns the statistics on the human-data track. read the letter →

arxiv 2411.13934 v1 pith:RPS5YBUF submitted 2024-11-21 cs.LG cs.AIcs.MA

Yancheng Liang , Daphne Chen , Abhishek Gupta , Simon S. Du , Natasha Jaques This is my paper · ORCID
classification cs.LGcs.AIcs.MA
keywords zero-shotcoordinationhuman-AIcooperationgenerativeagentmodelingvariationalautoencodermulti-agentreinforcementlearningOvercookedpartnerhumanevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

To make AI agents that cooperate with strangers, standard training either clones a small set of human demonstrations into a fake partner or assembles a discrete population of simulated policies. This paper argues that both miss the diversity of real human partners, and proposes GAMMA: learn a variational autoencoder over coordination trajectories, then sample partner policies from its latent space to train a Cooperator with reinforcement learning. In a user study with 80 novel human players on two hard Overcooked layouts, GAMMA variants outperform their corresponding baselines in both objective team score and subjective ratings, and a human-adaptive variant that centers latent sampling on a small human dataset reaches the best scores. The mechanism works whether the generative model is trained on simulated populations or on human data, because interpolation in latent space yields partners that cover strategies the discrete samples miss.

What carries the argument

The carrying object is a conditional variational autoencoder over joint trajectories. The encoder compresses a partner's behavioral style into a low-dimensional latent code, and the decoder acts as a partner policy conditioned on that code and on the interaction history. At Cooperator training time, latent codes are drawn from the prior or from a human-adaptive Gaussian each episode, and the decoder is unrolled as the partner, so the Cooperator faces a continuum of partners rather than a finite set. The latent-space interpolation is what the paper relies on to cover human strategy space beyond the training data.

What would settle it

Train GAMMA on a simulated population that provably excludes one strategy known to be common among humans, such as tomato-soup play on Multi-strategy Counter, then recruit human participants who prefer that strategy; if the GAMMA-trained Cooperator does not beat the corresponding non-generative baseline when paired with those humans, the latent-coverage claim is falsified. The paper's own CoMeDi+GAMMA result on Multi-strategy Counter is an existence proof of this failure mode.

Watch

Extended reading notes

Core claim

GAMMA's central discovery is that a generative model of partner behavior, not the individual policies, is the right object to train against for zero-shot coordination. The paper formalizes human partners as draws from a latent strategy space, and trains a VAE whose encoder maps a trajectory to a latent code and whose decoder produces the partner's action distribution given that code and history. Sampling latent codes from the prior yields unlimited partner policies; sampling from a human-centered Gaussian, estimated by encoding a small human dataset, biases the Cooperator's training toward human-like partners. In real-human evaluation, GAMMA improves the FCP, CoMeDi, and MEP baselines, and GAMMA-HA surpasses the strongest baseline by 60% on Counter Circuit and 43% on Multi-strategy Counter. The paper also reports that this fails when the underlying training population lacks a human strategy, as with CoMeDi on the tomato-soup layout.

Load-bearing premise

The load-bearing premise is that the VAE's latent space, when sampled from its prior, generates partner policies that include the strategies real humans will use; if a strategy is absent from the training trajectories, sampling cannot invent it.

Editorial extensions

If this is right

  • A single generative partner model can replace large discrete populations; trained Cooperators see a continuum of partner styles and can transfer to novel humans without additional data collection.
  • Small human datasets become useful not by replacing simulation but by shifting the latent prior; even tens of trajectories can improve coordination.
  • The approach gives a common framework for comparing simulated populations and human-data baselines: embed each into the same latent space and compare their coverage.
  • Performance inherits the diversity of the training data; improving population-generation methods amplifies GAMMA's gains, while poor populations limit them.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the latent-coverage hypothesis is right, the same GAMMA machinery could be used as a diagnostic: project any candidate population of simulated partners into the latent space and measure how much of the human-data latent distribution it covers, turning a coverage assumption into a checkable quantity.
  • A natural next step is a stabilized version of the z-conditioned Cooperator that infers the human's latent code online and conditions its policy on it; the paper reports that preliminary attempts were unstable, so this could add online adaptation to the method's offline-trained robustness.
  • The approach should transfer beyond two-player games and beyond Overcooked, but the paper notes the quadratic growth in trajectories with more agents, so sampling techniques that scale the latent space to teams are the obvious testbed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes GAMMA, a VAE-based generative model of partner behavior for zero-shot coordination in Overcooked. The method trains a Cooperator by sampling latent partner policies from a VAE trained on either simulated populations (FCP, CoMeDi, MEP) or human data, with a Human-Adaptive sampling variant that uses a small amount of human data to shift the latent prior and fine-tune the model. The evaluation includes a human study with 80 participants on two complex layouts and reports that GAMMA improves performance over baselines and that GAMMA-HA is the best method.

Significance. If the results hold, the paper would make a useful contribution by introducing generative partner modeling as a way to diversify training populations for human-AI coordination, by comparing two lines of research (simulated populations vs. human-data BC) under a common framework, and by releasing code and a demo. The human evaluation is a strength, as are the Holm-Bonferroni corrected tests and the introduction of a more complex layout. However, the central "consistent improvement" claim is not fully supported by the reported statistics, and the proposed mechanism (latent-space span) is not directly validated. The paper has the potential to be accepted after the claims are calibrated and the missing evidence is supplied.

major comments (4)
  1. [Abstract, §6.3, Tables 3 and 4] The central claim that "GAMMA consistently improves performance" is not supported by all pairwise comparisons in the human evaluation. On Multi-strategy Counter, PPO+BC+GAMMA scores 77.53 ± 2.00, below the PPO+BC baseline of 85.26 ± 2.28, and the one-sided test in Table 4 gives p = 7.48 × 10⁻². On Counter Circuit, MEP+GAMMA does not significantly beat MEP (p = 0.639). In addition, the key comparison GAMMA-HA > PPO+BC on Multi-strategy Counter is not significant (p = 0.426). Since the abstract and Section 6.3 use "consistently" and "significant advantages" to summarize the entire study, the authors should either qualify these statements to the comparisons that are actually significant or provide additional data to close these gaps. This is load-bearing because the human evaluation is the primary evidence for the method's benefit.
  2. [§4.1 and §6.2.1] The proof of concept for GAMMA rests on the claim in Section 4.1 that the generative model "can generate behaviors that go far beyond the training data" and that "human behavior is more likely to lie within the span of strategies generated by the generative model." The CoMeDi+GAMMA result on Multi-strategy Counter is direct counter-evidence: because the CoMeDi population contains no tomato-related strategies, "GAMMA also fails to learn any strategies involving tomatoes," and the CoMeDi+GAMMA score (34.72) is only slightly above CoMeDi (27.11). The paper acknowledges this as "garbage in, garbage out," but the acknowledgement does not resolve the tension with the span hypothesis. To keep the claim, the authors need a direct, quantitative test that latent sampling produces novel and human-relevant behaviors outside the training support (e.g., action-level novel ingredient use, diversity or coverage metrics on held-out human data). Otherwise, the contribution should be reframed as interpolation or smoothing over the existing population, which is a weaker claim than the one made in Section 4.1 and Figure 1.
  3. [§5 and Table 4] The human study is described at a level of detail that prevents the reader from verifying the statistical analysis. The text states that 80 users were recruited and that each played several rounds in randomized order, but it does not report the number of participants assigned to each layout, the number of rounds per participant, or the handling of participants who failed the attention check. Without this information, the p-values in Table 4 cannot be linked to a specific sample size, and the use of Holm-Bonferroni correction is not enough to assess the reliability of the tests. Please report the full protocol, per-comparison N, and effect sizes or confidence intervals.
  4. [Appendix C vs Appendix I.1] There is a direct factual inconsistency in the size of the human dataset for Multi-strategy Counter. Appendix C states "we collect 38 trajectories" for Multi-strategy Counter, while Appendix I.1 states that "the number of human coordination trajectories available for training (N = 11) is significantly less than the first layout (N = 37)." Since the "small amount of human data" is a central premise of the Human-Adaptive sampling claims, the actual dataset size must be stated consistently, and the final results (Table 3) must be tied to that dataset.
minor comments (7)
  1. [§5] The sentence "each of the 9P Pgame rounds" contains a typo; please clarify the intended number of rounds.
  2. [§6.3] The sentence "surpassing the most competitive baselines by 60% and 43%" does not correspond to any pairwise percentage in Table 3; please specify the reference baseline and the calculation.
  3. [§5 and Figure 5] The caption phrase "use the Standard Error of the Mean (SE) for statistical significance (p < 0.05)" is not a valid significance criterion; overlapping SE bars do not imply non-significance. Please report proper confidence intervals alongside the tests.
  4. [§5 and Appendix I] The two fine-tuning variants are both called "DFT" in Section 5 ("decoder-only (DFT) and full fine-tuning (DFT)"); please rename the second variant to FFT consistently.
  5. [Table 4] The column header "Multi-(p-value)" is incomplete; use "Multi-strategy Counter (p-value)".
  6. [Acknowledgments] The acknowledgment section contains a duplicated sentence: "DC acknowledges the support of AI@UW gift award and NSF RI: 2212310."
  7. [Appendix I, Table 5] Table 5 is labeled "outdated" but remains in the paper; it should be removed or moved to a clearly marked appendix with an explanation of what changed.

Circularity Check

0 steps flagged · score 1.0 of 10

No material circularity: the central claim is validated against held-out human players, and the paper's own 'garbage in, garbage out' failure case shows the result is not forced by construction.

full rationale

No material circularity found. GAMMA's derivation chain is: (i) train a VAE on partner trajectories from simulated populations or human data; (ii) sample latent z from a prior, or from a human-centered Gaussian, and use the decoder as a partner policy; (iii) train a Cooperator by PPO against these sampled partners; and (iv) test against held-out human participants. The human-adaptive sampling step (Eq. 3, Section 4.3) fits the latent mean to a human training dataset, but the evaluation recruits 'new participants that were not in the training data' (Section 6.2), so the headline result is an external benchmark rather than a refit of the input. No load-bearing self-citation chain appears: the Grupen et al. citation concerns a background claim about self-play conventions, and it does not justify the method's validity. The paper's own failure case, CoMeDi+GAMMA on Multi-strategy Counter (Section 6.2.1), shows the output is not forced by construction: when tomato strategies are absent from the source population, GAMMA 'also fails to learn any strategies involving tomatoes.' This is an empirical data-coverage limitation, explicitly acknowledged in Section 7, not a circular reduction. The only mildly self-referential element is Figure 1, where the 'coverage' of generative agents is visualized in the VAE's own latent space by encoding decoded episodes; this supports motivation but is not the basis for the central human-evaluation claim, which is tested externally. The method can fail when its training data lacks diversity, which confirms that its success is not guaranteed by definition or by a fitted parameter renamed as a prediction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central claim rests on the latent-variable model of human partners, the assumption that VAE sampling produces useful training partners, and the proxies used for evaluation (BC human proxy, Prolific participants). The free parameters are hand-chosen VAE hyperparameters (latent dim, KL target, identity covariance for the human prior) that affect results. No new physical entities are introduced; the latent variable z is a modeling construct.

free parameters (3)
  • KL target value for VAE training = 7 (Forced Coordination/CoMeDi), 32 (default)
    Hand-chosen target KL divergence per layout and population (Appendix B); Section I.2 reports using a larger KL penalty to fix FFT instability, showing the value affects results.
  • Human latent prior covariance = identity matrix I
    Human-adaptive sampling sets ph(z) = N(zbar, I) in Eq. 3, with covariance chosen rather than estimated from human data.
  • Latent dimension = 16
    Latent variable dimension chosen as a hyperparameter (Table 2); it controls the expressiveness of partner diversity.
assumptions (4)
  • domain assumption Human partner policies are generated by conditioning on a latent variable z, with a population distribution D(Z) over z (Section 3).
    The latent MDP formulation assumes human behavior is explained by a single latent style variable; if human variation is not low-dimensional, the model is misspecified.
  • domain assumption A VAE trained with the ELBO on joint trajectories yields decoders that behave as coherent partner policies when z is sampled from the prior (Sections 4.1, 4.2).
    The paper hypothesizes that interpolation and generalization of the generative model expands coverage; this is unproven and fails when training data is low diversity (Section 6.2.1).
  • domain assumption The held-out BC human proxy is a valid automatic evaluation partner for H1 (Section 5).
    The authors use this proxy to compare simulated-data methods but acknowledge it can be exploited by PPO-BC, so it may not predict real-human performance.
  • standard math PPO optimizes the Cooperator over a batch of MDPs sampled from the latent distribution (Eq. 2).
    Replaces the expectation over z with a finite sample approximation; standard RL practice, not proven.
invented entities (1)
  • Latent variable z (partner style/strategy embedding)
    purpose: Encodes the partner's strategy, intention, or skill; sampling z generates varied partners for training the Cooperator.
    Introduced as a modeling construct in Section 3; no direct measurement of z is provided. The success of the method is the only indirect evidence for its usefulness.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning to Cooperate with Humans using Generative Agents." pith.science (2026). https://pith.science/paper/RPS5YBUF

@misc{pith2026241113934,
  author       = {Pith},
  title        = {Pith review of: Learning to Cooperate with Humans using Generative Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RPS5YBUF}},
  note         = {Machine review of arXiv:2411.13934}
}
read the original abstract

Training agents that can coordinate zero-shot with humans is a key mission in multi-agent reinforcement learning (MARL). Current algorithms focus on training simulated human partner policies which are then used to train a Cooperator agent. The simulated human is produced either through behavior cloning over a dataset of human cooperation behavior, or by using MARL to create a population of simulated agents. However, these approaches often struggle to produce a Cooperator that can coordinate well with real humans, since the simulated humans fail to cover the diverse strategies and styles employed by people in the real world. We show \emph{learning a generative model of human partners} can effectively address this issue. Our model learns a latent variable representation of the human that can be regarded as encoding the human's unique strategy, intention, experience, or style. This generative model can be flexibly trained from any (human or neural policy) agent interaction data. By sampling from the latent space, we can use the generative model to produce different partners to train Cooperator agents. We evaluate our method -- \textbf{G}enerative \textbf{A}gent \textbf{M}odeling for \textbf{M}ulti-agent \textbf{A}daptation (GAMMA) -- on Overcooked, a challenging cooperative cooking game that has become a standard benchmark for zero-shot coordination. We conduct an evaluation with real human teammates, and the results show that GAMMA consistently improves performance, whether the generative model is trained on simulated populations or human datasets. Further, we propose a method for posterior sampling from the generative model that is biased towards the human data, enabling us to efficiently improve performance with only a small amount of expensive human interaction data.

Figures

Figures reproduced from arXiv: 2411.13934 by the authors.

Figure 1
Figure 1. We show the latent space covered by different methods. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Overview of the method for GAMMA. The generative model learns a latent distribution over partner strategies from either simulated or human data. Sampling partners from the generative model enables training a robust Cooperator that can coordinate with a variety of different humans. ity [8, 21] of humans. The agent might fail to coordinate well when it does not properly infer the preferences or intentions of humans. O… view at source ↗
Figure 3
Figure 3. The first five layouts Cramped Room, Asymmetric Advantages, Coordination Ring, Forced Coordination, Counter Circuit are originally proposed in Carroll et al. [1]. We create an additional Multi-strategy Counter layout. In this new layout, humans can additionally choose between making onion vs. tomato soup, which makes coordination significantly more challenging. H3: State-of-the-art. Can we obtain better performance … view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Evaluation of different methods using a human proxy model. Rewards are normalized by [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Performance of different agents when played with real humans. Error bars [4] use the Standard Error of the Mean (SE) for statistical significance (p < 0.05). Methods trained on human data are shown in green. Whether training with simulated or human data, GAMMA shows co…
Figure 6
Figure 6. Figure 6: Human ratings for different agents. Individuals were asked to respond to the following question: "Overall, I felt that the agent’s ability to coordinate with me was: {Very poor, Poor, Neutral, Good, Very good}". FCP + GAMMA, PPO + BC + GAMMA, and GAMMA-HA consistently …
Figure 7
Figure 7. Figure 7: In our user study, we ensure that participants pass an attention check by reviewing the [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 7
Figure 7. Figure 7: Percent of participants who agree with the statement “I have experience playing the game [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Human performance improves with the number of trials, indicating that the humans learn, [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Human ratings for different agents. Individuals were asked to respond to the following question: "The agent adapted to me when making decisions: {Strongly Disagree, Disagree, Neutral, Agree, Strongly Agree}". FCP + GAMMA and GAMMA-HA-DFT consistently receive higher rat…
Figure 10
Figure 10. Figure 10: Human ratings for different agents. Individuals were asked to respond to the following question: "The agent’s actions were human-like: {Strongly Disagree, Disagree, Neutral, Agree, Strongly Agree}". FCP + GAMMA, CoMeDi + GAMMA, and GAMMA-HA-DFT consistently receive ra…
Figure 11
Figure 11. Figure 11: Human ratings for different agents. Individuals were asked to respond to the following question: "The agent’s behavior was frustrating: {Strongly Disagree, Disagree, Neutral, Agree, Strongly Agree}". FCP + GAMMA and GAMMA-HA consistently receive ratings for less frust…
Figure 12
Figure 12. Figure 12: Learning curves for methods using simulated data across six layouts. Error bars are the [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: Learning curves for methods using human data. When evaluated with a held-out human [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: With larger KL Divergence penalty coefficient ( [PITH_FULL_IMAGE:figures/full_fig_p020_14.png]
Figure 15
Figure 15. Figure 15: Performance of z-conditioned Cooperator. The z-conditioned Cooperator reaches a higher reward in the Multi-strategy Counter. The performance decreases after the peak since the z-conditioned policy overfits the encoder. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_15.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 30 canonical work pages

  1. [1]

    Carroll, R

    M. Carroll, R. Shah, M. K. Ho, T. L. Griffiths, S. A. Seshia, P. Abbeel, and A. D. Dragan. On the utility of learning about humans for human-ai coordination. CoRR, abs/1910.05789, 2019. URL http://arxiv.org/abs/1910.05789

    arXiv 1910

  2. [2]

    Charakorn, P

    R. Charakorn, P. Manoonpong, and N. Dilokthanakul. Generating diverse cooperative agents by learning incompatible policies. In The Eleventh International Conference on Learning Representations, 2022

    work page 2022

  3. [3]

    B. Cui, A. Lupu, S. Sokota, H. Hu, D. J. Wu, and J. N. Foerster. Adversarial diversity in hanabi. In The Eleventh International Conference on Learning Representations, 2022

    work page 2022

  4. [4]

    Cumming, F

    G. Cumming, F. Fidler, and D. L. Vaux. Error bars in experimental biology. The Journal of cell biology, 177(1):7–11, 2007

    work page 2007

  5. [5]

    Grover, M

    A. Grover, M. Al-Shedivat, J. Gupta, Y . Burda, and H. Edwards. Learning policy representations in multiagent systems. In International conference on machine learning, pages 1802–1811. PMLR, 2018

    work page 2018

  6. [6]

    Grupen, N

    N. Grupen, N. Jaques, B. Kim, and S. Omidshafiei. Concept-based understanding of emergent multi-agent behavior. In Deep Reinforcement Learning Workshop NeurIPS 2022, 2022

    work page 2022

  7. [7]

    J. Henrich. The secret of our success: How culture is driving human evolution, domesticating our species, and making us smarter. In The secret of our success. princeton University press, 2015

    work page 2015

  8. [8]

    S. J. Hoch and G. F. Loewenstein. Time-inconsistent preferences and consumer self-control. Journal of consumer research, 17(4):492–507, 1991

    work page 1991

Show all 40 references
  1. [9]

    J. Hong, A. Dragan, and S. Levine. Learning to influence human behavior with offline rein- forcement learning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023

    2023

  2. [10]

    Hu and J

    H. Hu and J. N. Foerster. Simplified action decoder for deep multi-agent reinforcement learning. In International Conference on Learning Representations, 2019

    2019

  3. [11]

    other-play

    H. Hu, A. Lerer, A. Peysakhovich, and J. Foerster. “other-play” for zero-shot coordination. In International Conference on Machine Learning, pages 4399–4410. PMLR, 2020. 11

    2020

  4. [12]

    D. P. Kingma and M. Welling. Auto-encoding variational bayes. In Y . Bengio and Y . LeCun, editors, 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. URL http://arxiv.org/ abs/1312.6114

    2014 arXiv

  5. [13]

    J. Kwon, Y . Efroni, C. Caramanis, and S. Mannor. Rl for latent mdps: Regret guarantees and a lower bound. Advances in Neural Information Processing Systems, 34:24523–24534, 2021

    2021

  6. [14]

    Y . Li, J. Song, and S. Ermon. Infogail: Interpretable imitation learning from visual demonstra- tions. Advances in neural information processing systems, 30, 2017

    2017

  7. [15]

    Y . Li, S. Zhang, J. Sun, Y . Du, Y . Wen, X. Wang, and W. Pan. Cooperative open-ended learning framework for zero-shot coordination. In International Conference on Machine Learning, pages 20470–20484. PMLR, 2023

    2023

  8. [16]

    R. Likert. A technique for the measurement of attitudes. Archives of Psychology, 140:1–55, 1932

    1932

  9. [17]

    M. L. Littman. Markov games as a framework for multi-agent reinforcement learning. In Machine learning proceedings 1994, pages 157–163. Elsevier, 1994

    1994

  10. [18]

    X. Lou, J. Guo, J. Zhang, J. Wang, K. Huang, and Y . Du. Pecan: Leveraging policy ensemble for context-aware zero-shot human-ai coordination. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems, pages 679–688, 2023

    2023

  11. [19]

    A. Lupu, B. Cui, H. Hu, and J. Foerster. Trajectory diversity for zero-shot coordination. In International conference on machine learning, pages 7204–7213. PMLR, 2021

    2021

  12. [20]

    Papoudakis, F

    G. Papoudakis, F. Christianos, and S. Albrecht. Agent modelling under partial observability for deep reinforcement learning. Advances in Neural Information Processing Systems , 34: 19210–19222, 2021

    2021

  13. [21]

    J. W. Pratt. Risk aversion in the small and in the large. In Uncertainty in economics, pages 59–79. Elsevier, 1978

    1978

  14. [22]

    J. K. Pugh, L. B. Soros, and K. O. Stanley. Quality diversity: A new frontier for evolutionary computation. Frontiers in Robotics and AI, 3:40, 2016

    2016

  15. [23]

    Rashid, M

    T. Rashid, M. Samvelyan, C. S. De Witt, G. Farquhar, J. Foerster, and S. Whiteson. Monotonic value function factorisation for deep multi-agent reinforcement learning.The Journal of Machine Learning Research, 21(1):7234–7284, 2020

    2020

  16. [24]

    Sarkar, A

    B. Sarkar, A. Shih, and D. Sadigh. Diverse conventions for human-ai collaboration. In Thirty- seventh Conference on Neural Information Processing Systems, 2023

    2023

  17. [25]

    Schulman, F

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  18. [26]

    M. Shum, M. Kleiman-Weiner, M. L. Littman, and J. B. Tenenbaum. Theory of minds: Understanding behavior in groups through inverse planning. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 6163–6170, 2019

    2019

  19. [27]

    Silver, J

    D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton, et al. Mastering the game of go without human knowledge. nature, 550 (7676):354–359, 2017

    2017

  20. [28]

    Stone, G

    P. Stone, G. Kaminka, S. Kraus, and J. Rosenschein. Ad hoc autonomous agent teams: Col- laboration without pre-coordination. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 24, pages 1504–1509, 2010

    2010

  21. [29]

    Strouse, K

    D. Strouse, K. McKee, M. Botvinick, E. Hughes, and R. Everett. Collaborating with humans without human data. Advances in Neural Information Processing Systems, 34:14502–14515, 2021. 12

    2021

  22. [30]

    Z. Tang, C. Yu, B. Chen, H. Xu, X. Wang, F. Fang, S. S. Du, Y . Wang, and Y . Wu. Discovering diverse multi-agent strategic behavior via reward randomization. In International Conference on Learning Representations, 2020

    2020

  23. [31]

    Tjanaka, M

    B. Tjanaka, M. C. Fontaine, J. Togelius, and S. Nikolaidis. Approximating gradients for differentiable quality diversity in reinforcement learning. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 1102–1111, 2022

    2022

  24. [32]

    Tomasello

    M. Tomasello. Why we cooperate. MIT press, 2009

    2009

  25. [33]

    Vinyals, I

    O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Mathieu, A. Dudzik, J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgiev, et al. Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature, 575(7782):350–354, 2019

    2019

  26. [34]

    C. Wang, C. Pérez-D’Arpino, D. Xu, L. Fei-Fei, K. Liu, and S. Savarese. Co-gail: Learning diverse strategies for human-robot collaboration. In Conference on Robot Learning , pages 1279–1290. PMLR, 2022

    2022

  27. [35]

    S. Wu, J. Yao, H. Fu, Y . Tian, C. Qian, Y . Yang, Q. Fu, and Y . Wei. Quality-similar diversity via population based reinforcement learning. In The Eleventh International Conference on Learning Representations, 2022

    2022

  28. [36]

    S. A. Wu, R. E. Wang, J. A. Evans, J. B. Tenenbaum, D. C. Parkes, and M. Kleiman-Weiner. Too many cooks: Bayesian inference for coordinating multi-agent collaboration. Topics in Cognitive Science, 13(2):414–432, 2021

    2021

  29. [37]

    C. Yu, J. Gao, W. Liu, B. Xu, H. Tang, J. Yang, Y . Wang, and Y . Wu. Learning zero-shot coop- eration with humans, assuming humans are biased. In The Eleventh International Conference on Learning Representations, 2022

    2022

  30. [38]

    C. Yu, A. Velu, E. Vinitsky, J. Gao, Y . Wang, A. Bayen, and Y . Wu. The surprising effectiveness of ppo in cooperative multi-agent games. Advances in Neural Information Processing Systems, 35:24611–24624, 2022

    2022

  31. [39]

    M. Zhao, R. Simmons, and H. Admoni. Coordination with humans via strategy matching. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 9116–9123. IEEE, 2022

    2022

  32. [40]

    16 joint human-human trajectories for Cramped Room environment, 17 for Asymmetric Advantages, 16 for Coordination Ring, 12 for Forced Coordination, and 15 for Counter Circuit

    R. Zhao, J. Song, Y . Yuan, H. Hu, Y . Gao, Y . Wu, Z. Sun, and W. Yang. Maximum entropy population-based training for zero-shot human-ai coordination. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 6145–6153, 2023. 13 A Reproducibility Our ...

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.