Pith. sign in

REVIEW 4 major objections 6 minor 50 references

Robot teams can learn to coordinate at once with unknown partners, new environments, and team sizes that change mid-task.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 10:42 UTC pith:JEWKNHVM

load-bearing objection Solid multi-robot MARL systems paper: hypergraphic game + open-ended training, real dual-platform hardware, but partner-pool mismatch is the soft underbelly of the three-dimension claim. the 4 major comments →

arxiv 2607.04972 v1 pith:JEWKNHVM submitted 2026-07-06 cs.RO cs.AI

Multi-Robot Open Adaptive Teaming Across Unseen Environments, Partners, and Scales

Yang Li , Feng Xue , Fan Mo , Yunhao Liu , Jianhong Wang , Ying Wen , Qingrui Zhang , Shaoshuai Mou
show 1 more author
Wei Pan
This is my paper
classification cs.RO cs.AI
keywords open adaptive teamingmulti-robot collaborationcooperative pursuithypergraphic-form gamezero-shot coordinationmulti-dronemulti-quadrupedsim-to-real transfer
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Real multi-robot work rarely keeps the same teammates, the same map, and the same headcount. This paper names that joint requirement open adaptive multi-robot teaming and argues that closed training with fixed partners is the wrong default. It models cooperation as a hypergraphic-form game, so payoffs attach to whole coalitions rather than only pairs, and uses that structure to decide which partners to train against as team membership shifts inside an episode. The resulting algorithm, HOLA, grows partner and environment diversity over training instead of optimizing for one fixed lineup. On cooperative pursuit with drones and quadrupeds, the method beats the listed baselines on all three axes and runs on physical Crazyflie and Zsibot platforms without fine-tuning.

Core claim

The authors claim that open adaptive multi-robot teaming—simultaneous zero-shot coordination with unseen partners, novel environments, and variable team sizes including within-episode joins and leaves—can be solved by a hypergraphic-form game that scores multi-agent coalitions, combined with open-ended training that keeps expanding partner and environment diversity. Their algorithm HOLA, built on that formulation, outperforms standard multi-agent and population-based baselines across those three dimensions on multi-drone and multi-quadruped pursuit, and the learned policies transfer directly to hardware without retuning.

What carries the argument

Open hypergraphic-form game (and its preference hypergraph with hyper-preference centrality): a game-theoretic model in which hyperedges carry coalition utilities for variable team sizes; HOLA’s Oracle uses inverse Myerson-style values on that hypergraph to sample hard partners while environment and team-size distributions keep expanding.

Load-bearing premise

That training against partners chosen by inverse cooperative value on the learned preference hypergraph, plus sampling of environments and team sizes, is enough to generalize to truly open partners and mid-episode membership changes that were never part of that population.

What would settle it

Train HOLA as described, then evaluate mid-episode team changes against partners whose behaviors sit outside the training population and evaluation pool (Greedy, VICSEK, D3QN-G variants); if capture rate, collisions, and episode length collapse relative to in-pool open-team tests, the three-axis generalization claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper formalizes open adaptive multi-robot teaming as simultaneous generalization to unseen environments, unknown partners, and variable (including within-episode) team sizes. It introduces an Open Hypergraphic-Form Game (OH-Game / O-HyFoG) that models higher-order coalition utilities via hyperedges, derives an Open Preference Hypergraph and hyper-preference centrality η, and proposes HOLA: pre-training a diverse population (max-entropy), then iteratively expanding via a Grapher (hyperedge returns) and Oracle (approximate best-preferred agents) whose ϕ Solver samples partners by inverse Myerson/Shapley values on those returns (Eqs. 12–19). Evaluation is multi-robot cooperative pursuit on multi-drone and multi-quadruped platforms under fixed- and open-team protocols, against MAPPO, DACOOP-A, self-play, PBT, FCP, and MEP, with an ablation HOLA_R removing the ϕ Solver; policies are claimed to transfer zero-shot to Crazyflie and Zsibot L1 hardware.

Significance. Simultaneous three-axis open teaming is a genuine deployment bottleneck for multi-robot systems, and most prior work (ad hoc teamwork, ZSC, dynamic team size) treats axes in isolation or stays in discrete game benchmarks. A game-theoretic hypergraph formulation (explicitly not a GNN architecture) plus open-ended partner/environment expansion is a coherent design, and the multi-platform sim protocol with multi-seed error bars, heterogeneous teammate pool, and ϕ-solver ablation is stronger than typical robotics MARL papers. Direct hardware transfer without fine-tuning, if quantitatively substantiated, would be a clear contribution. Strengths to credit: external baselines and held-out partners/environments (low circularity), explicit ablation of the sampling module, and dual embodiment (aerial + legged).

major comments (4)
  1. Central claim vs. partner-distribution mismatch (§IV-B Oracle/ϕ Solver, Eqs. 12–19; §V-E, Table I; §VI-A/B). Training builds hyperedges and ϕ only over the co-evolving HOLA population; evaluation partners (Greedy, VICSEK, D3QN-G variants) are external rule/RL agents never present as vertices of the OH-Game. The three-dimension generalization claim therefore rests on an untested proxy assumption: that inverse-Myerson sampling inside the population induces policies that zero-shot coordinate with true open partners of different action representations and skill levels. HOLA_R only removes ϕ while still sampling from the same population, so it does not stress-test distribution mismatch. Please either (i) include evaluation partners (or behavioral clones) as held-out vertices during training, (ii) report a controlled mismatch experiment, or (iii) substantially qualify the open-partner claim an
  2. Hardware evidence does not match the abstract/conclusion strength (Abstract; §V-B; Fig. 4; §VI–VII). The manuscript asserts direct transfer to Crazyflie and Zsibot L1 without fine-tuning and “robust real-world coordination in novel environments with unseen teammates,” but §VI reports quantitative capture/collision/length results only in simulation. Fig. 4 is a setup photo; no table of real-world success rate, collision rate, episode length, or number of trials under open-team conditions is given. For a robotics journal claim of this weight, add quantitative hardware metrics under the same open-team protocol (or clearly demote the claim to qualitative demonstration).
  3. Environment-axis margins under simultaneous stress are thin (Table II; Fig. 5 vs. Fig. 6). In fixed-team novel environments, HOLA SUC is 44.00% vs. PBT 42.67% and SP 40.00%, with AST essentially tied with PBT. Open-team multi-drone results look stronger, but the simultaneous three-dimension claim is load-bearing and currently uneven across axes. Clarify statistical significance (e.g., paired tests over seeds/episodes), and discuss whether env gains are partly confounded by richer three-obstacle topology (as the text itself notes higher capture in “unseen” layouts).
  4. Tractability of the ϕ Solver (§IV-B, Eq. 12; Proposition 4.3; Algorithm 1). Inverse Myerson is written as an average over all permutations Π(V_j), which is factorial in population size, and the Grapher enumerates subsets for each cardinality. The paper does not state population sizes used, whether exact Shapley is replaced by sampling/Monte-Carlo, or wall-clock cost per generation. Without this, reproducibility and scalability claims for “open-ended” growth are incomplete. Specify the approximation (if any), |V|, |L|, and compute budget.
minor comments (6)
  1. Notation drift: OH-Game, O-HyFoG, Open Hypergraphic-Form Game, and “preference hypergraph OPG” are used interchangeably; pick one acronym set and stick to it (esp. Def. 4.1–4.2 and §IV-B).
  2. Fig. 5 caption refers to “HOLA R (marked as Our R)” while the text uses HOLA_R / HOLAR; align labels with the legend.
  3. Eq. (1)–(2) joint agent-action/type spaces use power-set notation that is easy to misread; a short example of a valid element of A_C would help.
  4. Within-episode membership change mechanism in evaluation (§V-C Open Team Protocol) is described at a high level (“partners join or leave”) but not operationalized (when, how many, sampling rule). Align with Algorithm 1’s P_team sampling.
  5. Related work on multi-robot pursuit and ZSC is solid; a brief pointer to recent continuous-control ad hoc / open-team robotics work (beyond Hanabi/Overcooked) would better situate the hardware claim.
  6. arXiv footer and journal header show placeholder dates/volume; clean for camera-ready.

Circularity Check

0 steps flagged

No load-bearing circularity: HOLA's claims are empirical measurements against external baselines, held-out partners/environments, and hardware; hyperedge weights and inverse-Myerson sampling are training machinery, not tautological rewrites of the reported success rates.

full rationale

Walk of the derivation chain finds no self-definitional loop, fitted-input-as-prediction, or uniqueness-by-self-citation that forces the central results. The OH-Game and open preference hypergraph (Defs. 4.1–4.2, Eq. 6) assign hyperedge weights as measured expected returns under joint policies (Eq. 9); hyper-preference centrality and inverse Myerson/Shapley partner sampling (Eqs. 12–19, Prop. 4.3) are then used as a training distribution inside the Oracle, not as a redefinition of evaluation metrics. Reported outcomes—capture/success rates, collision rates, episode lengths on fixed and open team protocols (Figs. 5–6, Tables I–II), and zero-shot hardware transfer on Crazyflie and Zsibot L1—are scored against external baselines (MAPPO, DACOOP-A, SP, PBT, FCP, MEP) and a heterogeneous teammate pool (Greedy, VICSEK, D3QN-G) that are not vertices of the training hypergraph. Self-citations to related ZSC/open-team work by overlapping authors supply background methods and population-training ideas; they do not substitute for the measured success rates or forbid alternatives via an imported uniqueness theorem. Partner-distribution mismatch between the co-evolving training population and the evaluation pool is a generalization risk, not circularity. Score 1 only for ordinary non-load-bearing self-citation background.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 3 invented entities

The central empirical claim rests on standard Dec-POMDP multi-agent RL assumptions, cooperative-game value concepts extended to hypergraphs, hand-chosen training distributions and thresholds, and several paper-specific constructs (OH-Game, OPG, η, HOLA modules). No machine-checked proofs; free parameters are algorithmic and experimental knobs rather than fitted physical constants.

free parameters (4)
  • entropy weight α in pre-training objective (Eq. 7)
    Balances reward vs policy entropy for population diversity; chosen as a training hyperparameter, not derived.
  • top-m rank threshold for approximate best-preferred agent (Eq. 20)
    Relaxes η=1 to rank(η)≤m; m is a design choice that gates when Oracle training stops.
  • constraint penalty weights λ_i (Eq. 4)
    Trade off task reward vs robot constraint violations; set by designers for platform feasibility.
  • team-size and environment sampling distributions P_team, P_env
    Control open-ended diversity during Grapher/Oracle updates; shape of these distributions is not learned from first principles.
axioms (4)
  • domain assumption Multi-robot decision making is adequately modeled as a Dec-POMDP with shared team reward plus additive constraint penalties.
    §III.A–B; standard in cooperative MARL but omits rich communication failure and continuous contact dynamics except via Φ penalties.
  • ad hoc to paper Coalition value of a hyperedge equals expected discounted return of that agent subset, and inverse Myerson/Shapley values on those weights correctly rank 'cooperative capacity' for partner sampling.
    Defs 4.1–4.2, Prop. 4.3, Eqs. 9–19; extends cooperative game theory to open hypergraphs without independent validation that this ranking matches true open-partner difficulty.
  • domain assumption Higher-order team interactions that matter for pursuit cannot be reduced to pairwise graphs and are well captured by variable-cardinality hyperedges.
    Introduction and §IV.A; motivates hypergraphs over pairwise models.
  • domain assumption Sim-trained continuous control policies with domain-style diversity transfer to Crazyflie/L1 hardware under motion-capture state and low-level controllers without fine-tuning.
    §V.B, §VI–VII; common sim-to-real hope, only qualitatively supported here.
invented entities (3)
  • Open Hypergraphic-Form Game (OH-Game / O-HyFoG) no independent evidence
    purpose: Game-theoretic model of multi-agent payoffs via variable-cardinality hyperedges for open team composition.
    Definition 4.1; paper-specific construct for strategic interactions, not a standard named game form with prior independent evidence.
  • Open Preference Hypergraph (OPG) and hyper-preference centrality η no independent evidence
    purpose: Extract preferred coalitions and scalar cooperative ability from OH-Game weights.
    Definition 4.2 and Eq. 6; invented scoring layer used to define best-preferred agents.
  • HOLA (Grapher + Oracle + ϕ Solver) no independent evidence
    purpose: Open-ended training algorithm that expands partners/environments and samples hard teammates via inverse Myerson weights.
    §IV.B and Algorithm 1; the proposed method entity.

pith-pipeline@v1.1.0-grok45 · 32759 in / 3695 out tokens · 36057 ms · 2026-07-11T10:42:27.748205+00:00 · methodology

0 comments
read the original abstract

Deploying robot teams in the real world requires simultaneous adaptation to unseen environments, unknown partners, and varying team sizes, yet existing approaches often address these challenges in isolation under the closed-world assumption of fixed teammates. We formalize this as open adaptive multi-robot teaming and propose a hypergraphic-form game formulation that captures team-level cooperative relationships beyond pairwise interactions, providing a principled foundation for coordination structure inference when team composition changes dynamically within episodes. Unlike graph neural network architectures, this is a game-theoretic construct for modeling strategic interactions and payoff structures among agents. Building on this formulation, we develop the Hypergraphic Open-ended Learning Algorithm (HOLA), which progressively expands partner and environment diversity during training rather than optimizing for fixed configurations. Evaluated on cooperative pursuit with multi-drone and multi-quadruped platforms, HOLA outperforms all baselines across all three adaptability dimensions. Learned policies transfer directly to physical hardware without fine-tuning, with successful deployments on Crazyflie and Zsibot L1 platforms confirming robust real-world coordination in novel environments with unseen teammates.

Figures

Figures reproduced from arXiv: 2607.04972 by Fan Mo, Feng Xue, Jianhong Wang, Qingrui Zhang, Shaoshuai Mou, Wei Pan, Yang Li, Ying Wen, Yunhao Liu.

Figure 1
Figure 1. Figure 1: Comparison of Traditional CTDE and Open Adaptive [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Schematic illustration of Open Hypergraphic-Form [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Hypergraphic Open-ended Learning Algorithm: Detailed illustration of a single generation within the open-ended [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Real-world deployment of multi-robot open adaptive [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison of task success rate (first column, higher is better), collision rate (second column, lower is better), and [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Open team coordination on multi-drone (top row) and multi-quadruped (bottom row) platforms. Metrics: capture rate [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

50 extracted references · 5 linked inside Pith

  1. [1]

    Search and pursuit-evasion in mobile robotics: A survey,

    T. H. Chung, G. A. Hollinger, and V . Isler, “Search and pursuit-evasion in mobile robotics: A survey,”Autonomous robots, vol. 31, pp. 299–316, 2011

    2011

  2. [2]

    DACOOP-A: Decentralized adaptive cooperative pursuit via attention,

    Z. Zhang, D. Zhang, Q. Zhang, W. Pan, and T. Hu, “DACOOP-A: Decentralized adaptive cooperative pursuit via attention,”IEEE Robotics and Automation Letters, vol. PP, pp. 1–8, 11 2023

    2023

  3. [3]

    Collaborative multi-robot search and rescue: Planning, coordination, perception, and active vision,

    J. P. Queralta, J. Taipalmaa, B. C. Pullinen, V . K. Sarker, T. N. Gia, H. Tenhunen, M. Gabbouj, J. Raitoharju, and T. Westerlund, “Collaborative multi-robot search and rescue: Planning, coordination, perception, and active vision,”Ieee Access, vol. 8, pp. 191617–191643, 2020

    2020

  4. [4]

    Multi- agent actor-critic for mixed cooperative-competitive environments,

    R. Lowe, Y . Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch, “Multi- agent actor-critic for mixed cooperative-competitive environments,” in Advances in Neural Information Processing Systems, vol. 30, 2017

    2017

  5. [5]

    QMIX: Monotonic value function factorisation for decen- tralised multi-agent reinforcement learning,

    T. Rashid, M. Samvelyan, C. Schroeder, G. Farquhar, J. Foerster, and S. Whiteson, “QMIX: Monotonic value function factorisation for decen- tralised multi-agent reinforcement learning,” inInternational Conference on Machine Learning, pp. 4295–4304, PMLR, 2018

    2018

  6. [6]

    The surprising effectiveness of PPO in cooperative multi-agent games,

    C. Yu, A. Velu, E. Vinitsky, J. Gao, Y . Wang, A. Bayen, and Y . Wu, “The surprising effectiveness of PPO in cooperative multi-agent games,” Advances in Neural Information Processing Systems, vol. 35, pp. 24611– 24624, 2022

    2022

  7. [7]

    “Other-Play

    H. Hu, A. Lerer, A. Peysakhovich, and J. Foerster, ““Other-Play” for zero-shot coordination,” inInternational Conference on Machine Learning, pp. 4399–4410, PMLR, 2020

    2020

  8. [8]

    Towards optimally decentralized multi-robot collision avoidance via deep rein- forcement learning,

    P. Long, T. Fan, X. Liao, W. Liu, H. Zhang, and J. Pan, “Towards optimally decentralized multi-robot collision avoidance via deep rein- forcement learning,” in2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 6252–6259, IEEE, 2018

    2018

  9. [9]

    Graph neural networks for decentralized multi-robot path planning,

    Q. Li, F. Gama, A. Ribeiro, and A. Prorok, “Graph neural networks for decentralized multi-robot path planning,” in2020 IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS), pp. 11785– 11792, IEEE, 2020

    2020

  10. [10]

    Coop- erative open-ended learning framework for zero-shot coordination,

    Y . Li, S. Zhang, J. Sun, Y . Du, Y . Wen, X. Wang, and W. Pan, “Coop- erative open-ended learning framework for zero-shot coordination,” in Proceedings of the 40th International Conference on Machine Learn- ing (ICML), vol. 202 ofProceedings of Machine Learning Research, pp. 20470–20484, PMLR, 2023

    2023

  11. [11]

    Ad hoc autonomous agent teams: Collaboration without pre-coordination,

    P. Stone, G. A. Kaminka, S. Kraus, and J. S. Rosenschein, “Ad hoc autonomous agent teams: Collaboration without pre-coordination,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 24, pp. 1504–1509, 2010

    2010

  12. [12]

    Open ad hoc teamwork with cooperative game theory,

    J. Wang, Y . Li, Y . Zhang, W. Pan, and S. Kaski, “Open ad hoc teamwork with cooperative game theory,” inProceedings of the 41st International Conference on Machine Learning (ICML), PMLR, 2024

    2024

  13. [13]

    Domain randomization for transferring deep neural networks from simulation to the real world,

    J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” inIEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 23–30, 2017

    2017

  14. [14]

    The surprising effectiveness of ppo in cooperative, multi-agent games,

    C. Yu, A. Velu, E. Vinitsky, J. Gao, Y . Wang, A. Bayen, and Y . Wu, “The surprising effectiveness of ppo in cooperative, multi-agent games,” 2022

    2022

  15. [15]

    Population based training of neural networks,

    M. Jaderberg, V . Dalibard, S. Osindero, W. M. Czarnecki, J. Don- ahue, A. Razavi, O. Vinyals, T. Green, I. Dunning, K. Simonyan, et al., “Population based training of neural networks,”arXiv preprint arXiv:1711.09846, 2017

    Pith/arXiv arXiv 2017

  16. [16]

    Collaborating with humans without human data,

    D. Strouse, K. McKee, M. Botvinick, E. Hughes, and R. Everett, “Collaborating with humans without human data,” inAdvances in Neural Information Processing Systems, vol. 34, pp. 14502–14515, 2021

    2021

  17. [17]

    Multi-robot system based on model of wolf hunting behavior to emulate wolf and elk interactions,

    J. D. Madden, R. C. Arkin, and D. R. MacNulty, “Multi-robot system based on model of wolf hunting behavior to emulate wolf and elk interactions,” in2010 IEEE International Conference on Robotics and Biomimetics, pp. 1043–1050, Dec. 2010

    2010

  18. [18]

    Wolf-pack (Canis lupus) hunting strategies emerge from simple rules in computa- tional simulations,

    C. Muro, R. Escobedo, L. Spector, and R. P. Coppinger, “Wolf-pack (Canis lupus) hunting strategies emerge from simple rules in computa- tional simulations,”Behavioural Processes, vol. 88, pp. 192–197, Nov. 2011

    2011

  19. [19]

    Collective Predation and Escape Strategies,

    L. Angelani, “Collective Predation and Escape Strategies,”Phys. Rev. Lett., vol. 109, no. 11, 2012

    2012

  20. [20]

    Group chasing tactics: how to catch a faster prey,

    M. Janosov, C. Vir ´agh, G. V ´as´arhelyi, and T. Vicsek, “Group chasing tactics: how to catch a faster prey,”New J. Phys., vol. 19, p. 053003, May 2017. Publisher: IOP Publishing

    2017

  21. [21]

    Intercepting Rogue Robots: An Algorithm for Capturing Multiple Evaders With Multiple Pursuers,

    A. Pierson, Z. Wang, and M. Schwager, “Intercepting Rogue Robots: An Algorithm for Capturing Multiple Evaders With Multiple Pursuers,” IEEE Robotics and Automation Letters, vol. 2, pp. 530–537, Apr. 2017. Conference Name: IEEE Robotics and Automation Letters

    2017

  22. [22]

    A survey of the pursuit–evasion problem in swarm intelligence,

    Z. Mu, J. Pan, Z. Zhou, J. Yu, and L. Cao, “A survey of the pursuit–evasion problem in swarm intelligence,”Front Inform Technol Electron Eng, vol. 24, pp. 1093–1116, Aug. 2023

    2023

  23. [23]

    A Geometric Approach for the Cooperative Two-Pursuer One-Evader Differential Game,

    E. Garcia, Z. E. Fuchs, D. Milutinovic, D. W. Casbeer, and M. Pachter, “A Geometric Approach for the Cooperative Two-Pursuer One-Evader Differential Game,”IFAC-PapersOnLine, vol. 50, pp. 15209–15214, July 2017

    2017

  24. [24]

    Cooperative Multiple Pursuers against a Single Evader,

    M. Kothari, J. G. Manathara, and I. Postlethwaite, “Cooperative Multiple Pursuers against a Single Evader,”J Intell Robot Syst, vol. 86, pp. 551– 567, June 2017

    2017

  25. [25]

    A Two-on-One Linear Pursuit–Evasion Game with Bounded Controls,

    S. Y . Hayoun and T. Shima, “A Two-on-One Linear Pursuit–Evasion Game with Bounded Controls,”J Optim Theory Appl, vol. 174, pp. 837– 857, Sept. 2017

    2017

  26. [26]

    Hysteretic Q-learning : an algorithm for Decentralized Reinforcement Learning in Cooperative Multi-Agent Teams,

    L. Matignon, G. J. Laurent, and N. Le Fort-Piat, “Hysteretic Q-learning : an algorithm for Decentralized Reinforcement Learning in Cooperative Multi-Agent Teams,” in2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 64–69, Oct. 2007. ISSN: 2153-0866

    2007

  27. [27]

    Robust Multi- Agent Reinforcement Learning via Minimax Deep Deterministic Policy Gradient,

    S. Li, Y . Wu, X. Cui, H. Dong, F. Fang, and S. Russell, “Robust Multi- Agent Reinforcement Learning via Minimax Deep Deterministic Policy Gradient,”Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 4213–4220, July 2019. Number: 01

    2019

  28. [28]

    Cascaded Attention: Adaptive and Gated Graph Attention Network for Multiagent Reinforcement Learning,

    S. Qi, X. Huang, P. Peng, X. Huang, J. Zhang, and X. Wang, “Cascaded Attention: Adaptive and Gated Graph Attention Network for Multiagent Reinforcement Learning,”IEEE Transactions on Neural Networks and Learning Systems, vol. 35, pp. 3769–3779, Mar. 2024. Conference Name: IEEE Transactions on Neural Networks and Learning Systems. JOURNAL OF LATEX CLASS FI...

    2024

  29. [29]

    Decentralized Multi-Agent Pursuit Using Deep Rein- forcement Learning,

    C. de Souza, R. Newbury, A. Cosgun, P. Castillo, B. Vidolov, and D. Kuli ´c, “Decentralized Multi-Agent Pursuit Using Deep Rein- forcement Learning,”IEEE Robotics and Automation Letters, vol. 6, pp. 4552–4559, July 2021. Conference Name: IEEE Robotics and Automation Letters

    2021

  30. [30]

    Multi-agent rein- forcement learning by the actor-critic model with an attention interface,

    L. Zhang, J. Li, Y . Zhu, H. Shi, and K.-S. Hwang, “Multi-agent rein- forcement learning by the actor-critic model with an attention interface,” Neurocomputing, vol. 471, pp. 275–284, Jan. 2022

    2022

  31. [31]

    Taskflex solver for multi-agent pursuit via automatic curriculum learning,

    J. Chen, G. Li, C. Yu, X. Yang, B. Xu, H. Yang, and Y . Wang, “Taskflex solver for multi-agent pursuit via automatic curriculum learning,”arXiv preprint arXiv:2312.12255, 2023

    Pith/arXiv arXiv 2023

  32. [32]

    Multi-uav pursuit-evasion with online planning in un- known environments by deep reinforcement learning,

    J. Chen, C. Yu, G. Li, W. Tang, X. Yang, B. Xu, H. Yang, and Y . Wang, “Multi-uav pursuit-evasion with online planning in un- known environments by deep reinforcement learning,”arXiv preprint arXiv:2409.15866, 2024

    Pith/arXiv arXiv 2024

  33. [33]

    Making friends on the fly: Cooperating with new teammates,

    S. Barrett, A. Rosenfeld, S. Kraus, and P. Stone, “Making friends on the fly: Cooperating with new teammates,”Artificial Intelligence, vol. 242, pp. 132–171, 2017

    2017

  34. [34]

    A general learning framework for open ad hoc teamwork using graph- based policy learning,

    A. Rahman, E. Hopkinson, J. Rawling, S. D. Varma, and J. Leitner, “A general learning framework for open ad hoc teamwork using graph- based policy learning,”Journal of Machine Learning Research, vol. 24, no. 99, pp. 1–62, 2023

    2023

  35. [35]

    Open ad hoc teamwork with cooperative game theory,

    J. Wang, Y . Li, Y . Zhang, W. Pan, and S. Kaski, “Open ad hoc teamwork with cooperative game theory,” inProceedings of the 41st International Conference on Machine Learning, vol. 235 ofProceedings of Machine Learning Research, pp. 50902–50930, PMLR, 2024

    2024

  36. [36]

    Shapley machine: A game- theoretic framework for n-agent ad hoc teamwork,

    J. Wang, Y . Li, S. Kaski, and J. Lawry, “Shapley machine: A game- theoretic framework for n-agent ad hoc teamwork,”arXiv preprint arXiv:2506.11285, 2025

    Pith/arXiv arXiv 2025

  37. [37]

    Off-belief learning,

    H. Hu, A. Lerer, B. Cui, L. Pineda, N. Brown, and J. Foerster, “Off-belief learning,” inInternational Conference on Machine Learning, pp. 4369– 4379, PMLR, 2021

    2021

  38. [38]

    Maximum entropy population-based training for zero- shot human-AI coordination,

    R. Zhao, J. Song, Y . Qi, S. Yuan, H. Hu, Y . Gao, F. Wu, Q. Zhang, and W. Wang, “Maximum entropy population-based training for zero- shot human-AI coordination,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 37, pp. 6145–6153, 2023

    2023

  39. [39]

    Coop- erative open-ended learning framework for zero-shot coordination,

    Y . Li, S. Zhang, J. Sun, Y . Du, Y . Wen, X. Wang, and W. Pan, “Coop- erative open-ended learning framework for zero-shot coordination,” in Proceedings of the 40th International Conference on Machine Learning, vol. 202 ofProceedings of Machine Learning Research, pp. 20470– 20484, PMLR, 2023

    2023

  40. [40]

    Tackling cooperative incompatibility for zero-shot human- ai coordination,

    Y . Li, S. Zhang, J. Sun, W. Zhang, Y . Du, Y . Wen, X. Wang, and W. Pan, “Tackling cooperative incompatibility for zero-shot human- ai coordination,”Journal of Artificial Intelligence Research, vol. 80, pp. 1139–1185, 2024

    2024

  41. [41]

    UPDeT: Universal multi- agent reinforcement learning via policy decoupling with transformers,

    S. Hu, F. Zhu, X. Chang, and X. Liang, “UPDeT: Universal multi- agent reinforcement learning via policy decoupling with transformers,” inInternational Conference on Learning Representations, 2021

    2021

  42. [42]

    Multi-agent reinforcement learning is a sequence modeling problem,

    M. Wen, J. G. Kuba, R. Lin, W. Zhang, Y . Wen, J. Wang, and Y . Yang, “Multi-agent reinforcement learning is a sequence modeling problem,” in Advances in Neural Information Processing Systems, vol. 35, pp. 16509– 16521, 2022

    2022

  43. [43]

    Evo- lutionary population curriculum for scaling multi-agent reinforcement learning,

    Q. Long, Z. Zhou, A. Gupta, F. Fang, Y . Wu, and X. Wang, “Evo- lutionary population curriculum for scaling multi-agent reinforcement learning,” inInternational Conference on Learning Representations, 2020

    2020

  44. [44]

    Skilled population curriculum for multi-agent reinforcement learning,

    R. Wang, M. Everett, and J. P. How, “Skilled population curriculum for multi-agent reinforcement learning,” inInternational Conference on Learning Representations, 2023

    2023

  45. [45]

    Maximum entropy population-based training for zero-shot human-ai coordination,

    R. Zhao, J. Song, Y . Yuan, H. Hu, Y . Gao, Y . Wu, Z. Sun, and W. Yang, “Maximum entropy population-based training for zero-shot human-ai coordination,” inThirty-Seventh AAAI Conference on Artificial Intelli- gence, AAAI 2023, Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence, IAAI 2023, Thirteenth Symposium on Educational Adv...

    2023

  46. [46]

    Group chasing tactics: How to catch a faster prey,

    M. Janosov, C. Vir ´agh, G. V ´as´arhelyi, and T. Vicsek, “Group chasing tactics: How to catch a faster prey,”New Journal of Physics, vol. 19, 05 2017

    2017

  47. [47]

    Dueling network architectures for deep reinforcement learning,

    Z. Wang, T. Schaul, M. Hessel, H. Hasselt, M. Lanctot, and N. Freitas, “Dueling network architectures for deep reinforcement learning,” in International conference on machine learning, pp. 1995–2003, PMLR, 2016

    1995

  48. [48]

    Equivalence between policy gradients and soft q-learning,

    J. Schulman, X. Chen, and P. Abbeel, “Equivalence between policy gradients and soft q-learning,”arXiv preprint arXiv:1704.06440, 2017

    Pith/arXiv arXiv 2017

  49. [49]

    On the utility of learning about humans for human- ai coordination,

    M. Carroll, R. Shah, M. K. Ho, T. Griffiths, S. Seshia, P. Abbeel, and A. Dragan, “On the utility of learning about humans for human- ai coordination,”Advances in neural information processing systems, vol. 32, 2019

    2019

  50. [50]

    Collaborating with humans without human data,

    D. Strouse, K. McKee, M. Botvinick, E. Hughes, and R. Everett, “Collaborating with humans without human data,”Advances in Neural Information Processing Systems, vol. 34, pp. 14502–14515, 2021

    2021