Pith. sign in

REVIEW 4 major objections 4 minor 40 references

Efficient Generation of Diverse Cooperative Agents with World Models

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that cross-play minimization for generating diverse cooperative partners can run on simulated trajectories from a learned world model, cutting environment steps by about three times at eight agents while preserving…

desk verdict A promising idea for faster XPM population generation, but the headline efficiency gain is confounded by swapping the base RL algorithm, so the 3x claim needs matched-algorithm ablations before it holds. read the letter →

arxiv 2506.07450 v1 pith:HUBTPYZX submitted 2025-06-09 cs.AI

Yi Loo , Akshunn Trivedi , Malika Meghjani This is my paper
classification cs.AI
keywords zero-shotcoordinationcross-playminimizationworldmodelsmodel-basedreinforcementlearningdiversepopulationgenerationOvercookedAImulti-agentself-play
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes that cross-play minimization, the standard way to train a population of agents with incompatible cooperative conventions, can be run on trajectories simulated by a learned world model instead of on extra environment rollouts. It introduces XPM-Sim, an objective that trains a partner agent only on simulated self-play and cross-play trajectories, and argues that starting simulated self-play from reachable cross-play states removes the need for the mixed-play trajectories used by CoMeDi to prevent handshake and self-sabotage behavior. It then wraps this objective in XPM-WM, a Dreamer-style world model learned together with the first agent and fine-tuned while later agents are trained. On Overcooked AI, the paper reports that XPM-WM reaches populations of eight agents with about three times fewer environment steps than CoMeDi while producing ego agents that score comparably against holdout partners and 40 human users. The point of the claim is that diverse-partner generation, a major cost in zero-shot coordination, can be amortized across agents rather than paid from scratch every time.

What carries the argument

The central object is the XPM-Sim objective, a training loss that maximizes expected simulated self-play return from both self-play and cross-play reachable states and minimizes expected simulated cross-play return against the partner with the highest estimated cross-play return. The identity that carries the argument is Proposition 3.2: a simulated self-play trajectory started from a cross-play state is equivalent to a mixed-play trajectory for preventing handshake and self-sabotage, which is what lets the method drop mixed-play sampling entirely. The supporting machinery is a world model in the Dreamer style, a recurrent latent-state model that predicts compact representations, rewards, and continuation signals, adapted to two-player observations with player-specific latent partitions and event-based vectorized reward prediction. That world model supplies the simulated trajectories used by XPM-Sim, provides a pretrained representation that bootstraps each subsequent partner, and is fine-tuned with a KL term so its latent distribution does not drift away from the frozen previous partners.

What would settle it

Measure, on a held-out set of cross-play pairs in Overcooked, the agreement between XPM-Sim's simulated return estimates and true environment returns starting from the same states; if the ranking of partners by simulated return disagrees with the real ranking often enough to change which partner is selected as the maximum cross-play partner, then the objective is optimizing model error rather than convention divergence and the sample-efficiency claim collapses.

Watch

Extended reading notes

Core claim

The central claim is that the extra trajectory types required by prior cross-play minimization are not actually necessary if the training objective is expressed on simulated trajectories. XPM-Sim replaces the LIPO and CoMeDi objectives with a single trade-off: maximize simulated self-play return, minimize simulated cross-play return against the current best partner, and let self-play trajectories start from both self-play and cross-play reachable states. Proposition 3.2 asserts that a simulated self-play rollout begun at a cross-play state is equivalent, for mitigating handshake and self-sabotage behavior, to a mixed-play trajectory, so the mixed-play term can be dropped. With a learned world model, the method first trains an initial agent and a Dreamer-based model of the two-player environment with event-based vectorized reward prediction, then sequentially trains partners against frozen predecessors while fine-tuning the model. The reported evidence is that across five Overcooked layouts, populations of eight agents require at least three times fewer environment steps than CoMeDi, that ego agents trained on XPM-WM populations match baseline ego agents on holdout partners and human partners, and that in the MPPMR toy environment XPM-Sim shows less observed self-sabotage than both LIPO and CoMeDi.

Load-bearing premise

The learned or hand-specified dynamics model F must predict transitions and rewards accurately enough that a return ranked in simulation matches the return the policy would obtain in the real environment.

Editorial extensions

If this is right

  • Because XPM-Sim needs only simulated trajectories, partner training no longer has to resample cross-play or mixed-play trajectories at every update, and the best cross-play partner can be chosen from replay buffers, removing a per-agent step cost that grows with population size.
  • Mixed-play trajectories can be dropped without bringing back self-sabotage: starting simulated self-play from reachable cross-play states gives the same protection, and the MPPMR experiment supports this.
  • Since one world model is fine-tuned across all agents, later partners start from a strong prior instead of from scratch, so per-agent training steps drop and populations scale to more agents at roughly linear rather than compounding cost.
  • An ego agent trained with an XPM-WM population performs comparably to LIPO and CoMeDi populations on holdout agents and human users, so the efficiency gain does not come at the cost of population diversity.
  • The method is demonstrated on fully observable two-player tasks, so its stated benefits are limited to that setting rather than to partially observable or larger-team coordination problems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If simulated-return estimation is the bottleneck, the same XPM-Sim trick could be applied to other population objectives, such as trajectory-diversity or maximum-entropy objectives, whenever a decent dynamics model exists; the paper demonstrates it only for cross-play minimization.
  • The effectiveness of starting self-play from cross-play states suggests a testable principle: any population method that fears handshake conventions can seed its positive training from the other agent's reachable states rather than adding a separate trajectory type, which could be evaluated directly by ablating the starting-state distribution in CoMeDi without any world model.
  • The vectorized reward result suggests that XPM-WM's gain is partly carried by structured reward prediction, so in environments where scalar rewards are the only signal, the world model would likely need a different inductive bias to learn meaningful policies.
  • The paper itself limits the demonstration to fully observable two-player tasks and states that partial observability and more than two players are not addressed, so the 3x sample-efficiency result should not be read beyond that setting.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes XPM-Sim, an objective for cross-play minimization that trains agents on simulated self-play and cross-play trajectories from a dynamics model, and XPM-WM, a concrete instantiation that learns a Dreamer-style RSSM world model to generate those trajectories. The authors claim that starting simulated SP trajectories from cross-play reachable states removes the need for mixed-play trajectories, that the MPPMR toy experiment confirms reduced self-sabotage, and that XPM-WM is at least three times more environment-step efficient than LIPO and CoMeDi when scaled to eight agents on Overcooked AI, while producing populations of comparable diversity for training HiPT ego agents, as measured on holdout partners and 40 human subjects.

Significance. If the efficiency and diversity claims were cleanly established, this would be a useful contribution to zero-shot coordination, where diverse partner generation is a major computational bottleneck. The paper has clear strengths: the MPPMR experiment in Section 3.1 directly measures self-sabotage, Algorithm 1 gives a complete training procedure, the evaluation covers five Overcooked layouts, a holdout partner population, and an IRB-approved human study, and the authors report the main hyperparameters. However, the headline efficiency comparison is confounded by a change of base RL algorithm, and the diversity evidence is noisy and lacks formal statistical support. These issues prevent the central quantitative claims from being considered established at this stage.

major comments (4)
  1. [Section 5.1, Figure 3, Appendix D.1] The claimed "at least 3 times more sample efficient" result is confounded by the training-algorithm change. LIPO and CoMeDi are implemented with MAPPO, while XPM-WM uses a Dreamer RSSM world model with REINFORCE and lambda-target returns, and every subsequent agent is bootstrapped from the same pretrained world model. The reported difference therefore bundles at least three changes: the XPM-Sim objective, the base learner (on-policy MAPPO vs. model-based/off-policy REINFORCE), and the shared pretrained representation. Without an ablation that runs LIPO/CoMeDi on the XPM-WM backbone, or XPM-WM with MAPPO, the efficiency gain cannot be attributed to XPM-Sim or XPM-WM. This is load-bearing because sample efficiency is the paper's central quantitative contribution.
  2. [Appendix A, Proposition 3.2] The proof of Proposition 3.2 does not establish the stated claim that simulated SP trajectories from cross-play starting states are equivalent to mixed-play trajectories "in terms of mitigating handshake and self-sabotage behavior." It only shows that the states visited by the simulated SP rollout from sXP are contained in the SP-reachable set from sXP; it does not show that the policy-optimization dynamics are equivalent to CoMeDi's mixed-play objective, nor does it give a mechanism by which this construction prevents self-sabotage. The MPPMR experiment in Section 3.1 provides some empirical support, but the proposition as stated is not proven by the argument in Appendix A.
  3. [Section 5.2, Table 2, Appendix H.3] The diversity-parity claim rests on differences that are within noise of the reported standard errors. The average holdout score is 5.80 ± 3.45 for E-XPM-WM versus 5.59 ± 3.31 for E-LIPO and 4.94 ± 3.64 for E-CoMeDi, and the per-layout tables show similarly wide overlapping intervals. No significance or equivalence test is reported, so the assertion that XPM-WM populations are "comparable in diversity" is not statistically established. Given that diversity is the other central claim, a non-inferiority analysis or a more precise evaluation is needed.
  4. [Section 3, Equation (3), Appendix G.1] The XPM-Sim objective assumes that the dynamics model F accurately approximates the transition and reward functions, as stated in Appendix A. Appendix G.1 shows that without hand-specified vectorized event rewards, the learned world model fails to learn meaningful policies. This indicates that the method's practical success depends on a task-specific reward decomposition that is not part of the XPM-Sim formulation. The paper should present this as an explicit scope condition rather than suggesting in Section 4 that XPM-WM is directly applicable to "more complex cooperative environments."
minor comments (4)
  1. [Algorithm 1, line 12] The algorithm text says "Estimate cross-play returns and compute cross-play objective with ˆτ SP W"; this should almost certainly read "ˆτ XP W" for consistency with the surrounding lines.
  2. [Section 5.2, Results paragraph] The phrase "WPM-XP" appears to be a typo for "XPM-WM."
  3. [Section 2.1 and Equation (7)] The symbol γ is used for the reward discount factor in the Dec-MDP definition and again as the KL weight in the world-model loss in Equation (7). This notational clash should be resolved.
  4. [Appendix H.1] The text says "we plot the training curves ... in Figure" without a figure number; the reference is incomplete.

Circularity Check

1 steps flagged · score 2.0 of 10

No significant circularity in the main empirical claims; one Proposition is a definitional reformulation of Mixed-Play, but the headline efficiency and diversity results rest on external real-environment evaluation.

  1. self definitional [Section 3 (Proposition 3.2 and Equation 3); Appendix A]
    "We propose that we are able to remove the need of Mixed play objective by simply expanding the starting states of simulated SP trajectories to include reachable XP states. Proposition 3.2. Simulated SP trajectories, τ̂ πSP F (sxp) generated via dynamics model F from any cross-play state, sxp ∈ SXP is equivalent to generating Mixed-Play trajectory in terms of mitigating handshake and self-sabotage behavior."

    By Section 2.2's own definition, an MP trajectory is an XP trajectory in which 'the trajectory before t* is discarded and the remaining trajectory segment is trained with the self-play objective' — i.e., an SP trajectory starting from an XP state. Equation (3) then defines dJSP to include starts from {S_SP, S_XP}, so the 'removal' of Mixed-Play is already present in the definition of XPM-Sim. The Appendix A proof only unpacks these definitions plus the assumption F ≈ T,R; it does not derive the equivalence from independent premises. The proposition is therefore true by construction (self-definitional) rather than a first-principles result. This does not invalidate the empirical self-sabotage comparison in Table 1, which tests the intended behavior.

full rationale

The paper's central quantitative claims are external: real-environment SP/XP returns (Table 9 and Figure 3), holdout-partner evaluations (Table 2), and a human user study (Figure 4). The world model is trained on real trajectories and is continuously fine-tuned with real trajectories; diversity is measured in the true environment, not by the world model's own predictions. The 3x sample-efficiency claim is an empirical comparison of environment steps, not a fitted quantity derived from the model. The main experimental weakness is an algorithm mismatch — LIPO/CoMeDi use MAPPO while XPM-WM uses a Dreamer/REINFORCE backbone — but that is a correctness/confound concern, not circularity. No load-bearing self-citation or imported uniqueness theorem appears; the citation to the authors' HiPT [16] is as an evaluation ego-agent, not as evidence that XPM-WM's populations are diverse. The only definitional reduction is Proposition 3.2, which is explicitly accompanied by an empirical self-sabotage test. Overall circularity is therefore low.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on one main assumption: that the dynamics model (known in XPM-Sim, learned in XPM-WM) is accurate enough for simulated self- and cross-play returns to proxy real-environment returns. No invented entities; the player-specific latent split and event-based reward head are architectural choices within an established world-model family. Free parameters include the XPM weights lambda_XP (tuned per method and per layout), the simulation horizon, and the hand-chosen event-reward scaling constants. The KL-stabilization term in Equation 7 and the bootstrapping benefit are additional design assumptions without dedicated ablations.

free parameters (4)
  • lambda_XP (XPM-Sim / XPM-WM) = 0.5 (MPPMR), 0.25 (Overcooked), 0.1 (Counter Circuit)
    Hand-chosen trade-off between SP return and XP minimization; tuned per method and per layout, with no sensitivity analysis, which can disadvantage baselines.
  • lambda_XP and lambda_MP (baselines) = lambda_XP 0.25 (LIPO), 0.5 (CoMeDi); lambda_MP 0.25 (CoMeDi)
    Baseline hyperparameters governing diversity strength; chosen differently from XPM-WM, complicating direct comparison.
  • Simulation horizon H' = 30 (MPPMR), 15 (Overcooked)
    Length of imagined rollouts; affects sample efficiency and model-error accumulation, chosen by hand.
  • Event reward scaling weights = 1, 1, 1, 3, 12 (Table 4)
    Hand-chosen scalar multipliers converting predicted event vectors into scalar rewards; Appendix G.1 shows the vectorized reward design is critical for the world model to learn at all.
assumptions (5)
  • domain assumption The dynamics model F accurately approximates the true transition and reward functions of the Dec-MDP.
    Invoked in the proof of Proposition 3.2 (Appendix A) and in the XPM-Sim objective (Equation 3). If false, simulated XP minimization may reflect model error instead of convention incompatibility.
  • standard math The Dec-MDP formulation (Section 2.1) and the ZSC objective against an unseen policy set adequately model zero-shot coordination.
    Assumed throughout; follows the ZSC literature [11] and the broader definition adopted in Section 2.1.
  • domain assumption SP trajectories replayed from cross-play reachable states are behaviorally equivalent to Mixed-Play trajectories for preventing handshakes and self-sabotage in Overcooked, not just in MPPMR.
    Section 3 and Proposition 3.2; only directly tested in the MPPMR toy environment (Table 1), then assumed to transfer.
  • ad hoc to paper The KL-divergence penalty in Equation 7 keeps the world model's latent distribution close enough to previous agents to produce consistent XP trajectories.
    A design choice proposed to prevent latent drift; not derived from first principles and no ablation isolates its contribution.
  • domain assumption Bootstrapping each subsequent agent from the pre-trained world model transfers environment knowledge without collapsing the population onto one convention.
    Central to the sample-efficiency claim (Section 4); the shared-encoder ablation (Appendix G.2) shows naive representation sharing can hurt, so the benefit depends on the specific world-model design.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Efficient Generation of Diverse Cooperative Agents with World Models." pith.science (2026). https://pith.science/paper/HUBTPYZX

@misc{pith2026250607450,
  author       = {Pith},
  title        = {Pith review of: Efficient Generation of Diverse Cooperative Agents with World Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HUBTPYZX}},
  note         = {Machine review of arXiv:2506.07450}
}
read the original abstract

A major bottleneck in the training process for Zero-Shot Coordination (ZSC) agents is the generation of partner agents that are diverse in collaborative conventions. Current Cross-play Minimization (XPM) methods for population generation can be very computationally expensive and sample inefficient as the training objective requires sampling multiple types of trajectories. Each partner agent in the population is also trained from scratch, despite all of the partners in the population learning policies of the same coordination task. In this work, we propose that simulated trajectories from the dynamics model of an environment can drastically speed up the training process for XPM methods. We introduce XPM-WM, a framework for generating simulated trajectories for XPM via a learned World Model (WM). We show XPM with simulated trajectories removes the need to sample multiple trajectories. In addition, we show our proposed method can effectively generate partners with diverse conventions that match the performance of previous methods in terms of SP population training reward as well as training partners for ZSC agents. Our method is thus, significantly more sample efficient and scalable to a larger number of partners.

Figures

Figures reproduced from arXiv: 2506.07450 by the authors.

Figure 1
Figure 1. Diagram of the training process of XPM-Sim (right) compared to CoMeDi. Our key insight [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Diagram of the training process for XPM-WM. For the first phase, only the self-play (in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Scaling by number of agents. We show the training cost in terms of environmental steps for [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: A comparison of the user study scores for HiPT ego agent trained with XPM-WM, CoMeDi [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: The initial state of Multi Particle Point Mass Rendezvous Environment. The 2 players [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: The Five Overcooked layouts From left to right: Cramped Room, Asymmetric Advantages, Coordination Ring, Forced Coordination and Counter Circuit. The Blue and Green hat chefs indicate the 2 different starting positions for each layout. D Implementation Details D.1 Train…
Figure 7
Figure 7. Figure 7: Trajectories of 2 sample agents under cross-play in the MPPMR environment. [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: A comparison of the training curves for XPM-WM with and without the use to vectorized [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: A comparison of the training curves for agents 2-8 for XPM-Sim with and without the [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]
Figure 10
Figure 10. Figure 10: A comparison of the training wall clock time for XPM-WM with CoMeDi and LIPO for [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: A comparison of the training wall clock time for XPM-WM with CoMeDi and LIPO for [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]
Figure 12
Figure 12. Figure 12: Training Curves for all 5 Overcooked layouts. We plot the number of dishes served in [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

40 extracted references · 31 canonical work pages

  1. [1]

    A. Bar, G. Zhou, D. Tran, T. Darrell, and Y . LeCun. Navigation world models.arXiv preprint arXiv:2412.03572, 2024

    arXiv 2024

  2. [2]

    Bruce, M

    J. Bruce, M. D. Dennis, A. Edwards, J. Parker-Holder, Y . Shi, E. Hughes, M. Lai, A. Mavalankar, R. Steigerwald, C. Apps, et al. Genie: Generative interactive environments. In Forty-first International Conference on Machine Learning, 2024

    work page 2024

  3. [3]

    Carroll, R

    M. Carroll, R. Shah, M. K. Ho, T. Griffiths, S. Seshia, P. Abbeel, and A. Dragan. On the utility of learning about humans for human-ai coordination. Advances in neural information processing systems, 32, 2019

    work page 2019

  4. [4]

    Charakorn, P

    R. Charakorn, P. Manoonpong, and N. Dilokthanakul. Generating diverse cooperative agents by learning incompatible policies. In The Eleventh International Conference on Learning Representations, 2023

    work page 2023

  5. [5]

    B. Cui, A. Lupu, S. Sokota, H. Hu, D. J. Wu, and J. N. Foerster. Adversarial diversity in hanabi. In The Eleventh International Conference on Learning Representations, 2023

    work page 2023

  6. [6]

    Ha and J

    D. Ha and J. Schmidhuber. World models. arXiv preprint arXiv:1803.10122, 2018

    arXiv 2018

  7. [7]

    Hafner, T

    D. Hafner, T. Lillicrap, J. Ba, and M. Norouzi. Dream to control: Learning behaviors by latent imagination. In International Conference on Learning Representations, 2020

    2020

  8. [8]

    Hafner, T

    D. Hafner, T. P. Lillicrap, M. Norouzi, and J. Ba. Mastering atari with discrete world models. In International Conference on Learning Representations, 2021

    work page 2021

Show all 40 references
  1. [9]

    Hafner, J

    D. Hafner, J. Pasukonis, J. Ba, and T. Lillicrap. Mastering diverse domains through world models. arXiv preprint arXiv:2301.04104, 2023

    2023 arXiv

  2. [10]

    A. Hu, L. Russell, H. Yeo, Z. Murez, G. Fedoseev, A. Kendall, J. Shotton, and G. Corrado. Gaia-1: A generative world model for autonomous driving. arXiv preprint arXiv:2309.17080, 2023

    2023 arXiv

  3. [11]

    other-play

    H. Hu, A. Lerer, A. Peysakhovich, and J. Foerster. “other-play” for zero-shot coordination. In International Conference on Machine Learning, pages 4399–4410. PMLR, 2020

    2020

  4. [12]

    H. Hu, A. Lerer, B. Cui, L. Pineda, N. Brown, and J. Foerster. Off-belief learning. In International Conference on Machine Learning, pages 4369–4379. PMLR, 2021

    2021

  5. [13]

    D. P. Kingma, M. Welling, et al. Auto-encoding variational bayes, 2013

    2013

  6. [14]

    Kraemer and B

    L. Kraemer and B. Banerjee. Multi-agent reinforcement learning as a rehearsal for decentralized planning. Neurocomputing, 190:82–94, 2016

    2016

  7. [15]

    Y . Li, S. Zhang, J. Sun, W. Zhang, Y . Du, Y . Wen, X. Wang, and W. Pan. Tackling cooperative incompatibility for zero-shot human-ai coordination. Journal of Artificial Intelligence Research, 80:1139–1185, 2024

    2024

  8. [16]

    Y . Loo, C. Gong, and M. Meghjani. A hierarchical approach to population training for human-ai collaboration. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, pages 3011–3019, 2023

    2023

  9. [17]

    X. Lou, J. Guo, J. Zhang, J. Wang, K. Huang, and Y . Du. Pecan: Leveraging policy ensemble for context-aware zero-shot human-ai coordination. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems, pages 679–688, 2023. 10

    2023

  10. [18]

    R. Lowe, Y . Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch. Multi-agent actor-critic for mixed cooperative-competitive environments. Neural Information Processing Systems, 2017

    2017

  11. [19]

    Lucas and R

    K. Lucas and R. E. Allen. Any-play: An intrinsic augmentation for zero-shot coordination. In Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, pages 853–861, 2022

    2022

  12. [20]

    A. Lupu, B. Cui, H. Hu, and J. Foerster. Trajectory diversity for zero-shot coordination. In International Conference on Machine Learning, pages 7204–7213. PMLR, 2021

    2021

  13. [21]

    Prasanna, K

    S. Prasanna, K. Farid, R. Rajan, and A. Biedenkapp. Dreaming of many worlds: Learning contextual world models aids zero-shot generalization. Reinforcement Learning Journal, 1, 2024

    2024

  14. [22]

    Rahman, J

    M. Rahman, J. Cui, and P. Stone. Minimum coverage sets for training robust ad hoc teamwork agents. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17523–17530, 2024

    2024

  15. [23]

    Rashid, M

    T. Rashid, M. Samvelyan, C. S. De Witt, G. Farquhar, J. Foerster, and S. Whiteson. Monotonic value function factorisation for deep multi-agent reinforcement learning. Journal of Machine Learning Research, 21(178):1–51, 2020

    2020

  16. [24]

    Sarkar, A

    B. Sarkar, A. Shih, and D. Sadigh. Diverse conventions for human-ai collaboration. Advances in Neural Information Processing Systems, 36:23115–23139, 2023

    2023

  17. [25]

    Schulman, P

    J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438, 2015

    2015 arXiv

  18. [26]

    Stone, G

    P. Stone, G. Kaminka, S. Kraus, and J. Rosenschein. Ad hoc autonomous agent teams: Col- laboration without pre-coordination. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 24, pages 1504–1509, 2010

    2010

  19. [27]

    Strouse, K

    D. Strouse, K. McKee, M. Botvinick, E. Hughes, and R. Everett. Collaborating with humans without human data. Advances in Neural Information Processing Systems, 34:14502–14515, 2021

    2021

  20. [28]

    Valevski, Y

    D. Valevski, Y . Leviathan, M. Arar, and S. Fruchter. Diffusion models are real-time game engines, 2024. URL https://arxiv.org/abs/2408.14837

    2024 arXiv

  21. [29]

    Venugopal, S

    A. Venugopal, S. Milani, F. Fang, and B. Ravindran. Mabl: Bi-level latent-variable world model for sample-efficient multi-agent reinforcement learning. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems, pages 1865–1873, 2024

    2024

  22. [30]

    X. Wang, S. Zhang, W. Zhang, W. Dong, J. Chen, Y . Wen, and W. Zhang. Zsc-eval: An evaluation toolkit and benchmark for multi-agent zero-shot coordination. Advances in Neural Information Processing Systems, 37:47344–47377, 2024

    2024

  23. [31]

    X. Wang, Z. Zhu, G. Huang, X. Chen, J. Zhu, and J. Lu. Drivedreamer: Towards real-world- drive world models for autonomous driving. In European Conference on Computer Vision, pages 55–72. Springer, 2024

    2024

  24. [32]

    L. Weng. Policy gradient algorithms. lilianweng.github.io, 2018. URL https://lilianweng. github.io/posts/2018-04-08-policy-gradient/

    2018

  25. [33]

    P. Wu, A. Escontrela, D. Hafner, P. Abbeel, and K. Goldberg. Daydreamer: World models for physical robot learning. In Conference on robot learning, pages 2226–2240. PMLR, 2023

    2023

  26. [34]

    K. J. Young, A. Ramesh, L. Kirsch, and J. Schmidhuber. The benefits of model-based general- ization in reinforcement learning. In International Conference on Machine Learning, pages 40254–40276. PMLR, 2023

    2023

  27. [35]

    C. Yu, A. Velu, E. Vinitsky, J. Gao, Y . Wang, A. Bayen, and Y . Wu. The surprising effectiveness of ppo in cooperative multi-agent games. Advances in Neural Information Processing Systems, 35:24611–24624, 2022. 11

    2022

  28. [36]

    C. Yu, J. Gao, W. Liu, B. Xu, H. Tang, J. Yang, Y . Wang, and Y . Wu. Learning zero-shot coop- eration with humans, assuming humans are biased. In The Eleventh International Conference on Learning Representations, 2023

    2023

  29. [37]

    A Study on Human-AI Collaboration

    R. Zhao, J. Song, Y . Yuan, H. Hu, Y . Gao, Y . Wu, Z. Sun, and W. Yang. Maximum entropy population-based training for zero-shot human-ai coordination. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 6145–6153, 2023. 12 A Proof of Proposition...

    2023

  30. [38]

    Game Participation Complete all 23 games including the tutorials

  31. [39]

    Note that inactivity exceeding 15 seconds in any game / Consistent zero rewards in the game rounds due to inactivity from your side will make you ineligible for reimbursement

    Active Engagement Maintain active participation with the AI agent throughout each game by moving and interacting continuously, and trying to collaborate with AI agent. Note that inactivity exceeding 15 seconds in any game / Consistent zero rewards in the game rounds due to ina...

  32. [40]

    Create Game

    Player Identification Upon completion of all game rounds, you will be assigned a unique in-game Player ID. Accurately copy and submit this Player ID in the provided Microsoft Form to ensure proper credit for your participation. Please be aware that submission of incorrect or f...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.