Pith. sign in

REVIEW 3 major objections 7 minor 83 references

A pointer-network actor and a shape-changing GNN critic let multi-agent policies keep working when agents and tasks appear or disappear, without fixed bounds or retraining.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 01:48 UTC pith:U7LV5QEE

load-bearing objection Solid joint AO+TO MARL system with real experiments; the appendix’s AO “agent pointer” lemma does not match the statistics query actually implemented. the 3 major comments →

arxiv 2607.25082 v2 pith:U7LV5QEE submitted 2026-07-27 cs.AI cs.MA

PLATO: Pointer Learner for Agent and Task Openness

Alireza Saleh Abadi , Leen-Kiat Soh , Daniel Alan Redder , Adam Eck , Prashant Doshi This is my paper
classification cs.AI cs.MA
keywords open agent systemsmulti-agent reinforcement learningagent opennesstask opennesspointer networksgraph neural network criticCTDEwildfire suppression
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Standard multi-agent reinforcement learning assumes fixed sets of agents and tasks, but many real systems are open: teammates join or leave and jobs appear or vanish mid-operation. This paper argues that those two forms of openness can be handled together by replacing fixed-index action heads with content-based pointing and by scoring the open global state with a graph whose nodes change with the current team and task set. The resulting method, PLATO, is formalized in a Task-and-Agent-Open Markov Game and proved well-defined and permutation-invariant over unbounded agent–task spaces. On a wildfire-suppression benchmark built for open multi-agent evaluation, it matches or beats strong baselines when openness is present and transfers more consistently to unseen grid sizes without retraining. A sympathetic reader cares because openness is the default in fielded teams, and prior methods either pad to artificial limits or address only one dimension of change.

Core claim

PLATO is presented as the first multi-agent actor–critic architecture that jointly handles agent openness and task openness without padding, masking, or fixed bounds: each agent forms a permutation-invariant team query and points, via additive attention, at the currently available tasks (plus a no-op), while a centralized bipartite GNN critic evaluates the open agent–task graph. The authors formalize the setting as a TaAgO-MG, prove the actor and critic are well-defined over the resulting unbounded spaces, and show stronger native returns and more consistent zero-shot grid transfer than DICG, DGN, and MOHITO under endogenous and exogenous task openness with temporary agent exit.

What carries the argument

Content-based pointer actor: a fixed-length query built from team statistics (mean, variance, min, max) is scored against keys for every current task plus NO-OP, so the policy support always equals the live task set; paired with a centralized bipartite agent–task GNN critic whose node sets grow and shrink with composition.

Load-bearing premise

Each task has exactly one associated action plus a shared no-op, so selecting a task fully determines what the agent does; if tasks had multi-way action menus, the current pointer interface would not apply as stated.

What would settle it

Train and evaluate on the same wildfire setups but give each fire a multi-action menu (e.g., suppress intensity levels or tools); if PLATO then loses its advantage over bounded or task-open-only baselines, or if zero-shot returns collapse when agent or task counts far exceed those seen in training, the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Policies need not reserve fixed action slots or retrain when the live task set changes; support tracks the current tasks by construction.
  • Agent arrivals and departures can be absorbed through a permutation-invariant team summary without rewriting the actor architecture.
  • Centralized critics can evaluate open multi-agent states by letting the agent–task graph change shape rather than padding to a maximum size.
  • Zero-shot transfer to larger spatial layouts becomes a fair test of openness handling rather than a forced architecture change.
  • TaAgO-MG gives a shared formal target for methods that claim to handle agent and task openness together.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The one-action-per-task restriction is the natural next stress test: a hierarchical or two-stage pointer (task then action-within-task) would show whether the openness story survives richer action menus.
  • Partial observability of agents or tasks would force the team statistics and task keys to be built from beliefs rather than facts, which is a direct extension the conclusion already flags.
  • The same pointer-plus-variable-graph pattern may transfer to other open domains (disaster response, on-demand logistics) where both workforce and job queues fluctuate.
  • Ablation hints that recurrent team memory plus additive scoring helps most under joint openness out of distribution, suggesting history is a cheap substitute for explicit openness models when transfer is the goal.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper introduces PLATO, a CTDE multi-agent RL method for open agent systems in which both the agent set (AO) and task set (TO) change online. The actor is a one-step pointer network: each agent builds a query from a permutation-invariant team summary (mean/var/min/max statistics over agent features, passed through an MLP) and scores the currently available tasks plus a NO-OP key via additive attention, so the policy's support always equals the current task set. A centralized bipartite agent–task GNN critic handles variable state shape. The setting is formalized as a Task-and-Agent-Open Markov Game (TaAgO-MG), with proofs of permutation invariance (Lemma 1), well-definedness over unbounded spaces (Theorem 1), support-adaptive rank-consistent pointing under TO and AO (Lemmas 2–3), and a complexity bound (Theorem 2). Experiments on the MOASEI wildfire domain (setups S0–S3, grids 2×3 to 5×5) compare against DICG, DGN, MOHITO, and four heuristics, showing PLATO generally strongest under combined openness (S2–S3) natively and more consistent in zero-shot grid transfer, with an encoder/scorer ablation.

Significance. If the results hold, this is a useful contribution: content-based pointing is a clean, natural mechanism for open action spaces, and joint AO+TO without padding or fixed bounds is genuinely under-addressed. The manuscript ships several things that raise confidence: explicit well-definedness and permutation-invariance arguments, a per-timestep complexity analysis, seed-matched evaluation (50 execution seeds × 3 checkpoints), Shapiro–Wilk-adaptive tests with clearly defined Bonferroni families, equal-budget Optuna tuning for all baselines, a 2×2 ablation, and a thorough reproducibility appendix including compute budget. The empirical story is honestly reported, including cells where baselines win. Impact is tempered by evaluation on a single domain from the authors' own ecosystem and by a formal gap in the AO guarantee (Major Comment 1).

major comments (3)
  1. [Appendix A.3 (Lemma 3) vs. Algorithm 1 lines 10–11 / §4.2] Lemma 3, which supplies the substantive AO guarantee (rank-consistency of relative agent weightings under arrival/departure), is stated and proved for an attention-based 'agent pointer': per-agent keys k_j = MLP_n(o_j), additive-attention scores against the ego key k_i, and query q_t = Σ_j α_j k_j. This mechanism appears nowhere in Algorithm 1 or §4.2, which implement a statistics-based query: d_t = [µ, Var, min, max](o_N), q_t = MLP_s(d_t). For the implemented query, an agent departure changes µ/Var/min/max essentially arbitrarily, so no analogue of Lemma 3(iii) holds — a departure can re-rank two remaining tasks' scores through the query, contrary to the narrative in A.4 ('a change in the team membership by itself cannot re-rank the remaining agents'). Additionally, Lemma 3's proof of (ii) claims permutation invariance of the attention read 'is exactly Lemma 1,' but Lemma 1 covers the
  2. [§5 (Metrics) and Appendix B (Hyperparameter tuning)] All methods are tuned with the trial objective equal to mean validation return on the 3×3 grid under S3, and native 3×3 S3 is then a headline result (Table 1). Because tuning is symmetric across methods the comparison is not unfair, but (i) the reported 3×3 S3 numbers are optimistically biased for every method relative to untuned setups, and (ii) all other grid/setup cells use hyperparameters selected on a single, most-open configuration, which may interact with each method's sensitivity to openness. Please state this explicitly, and ideally report a held-out validation (e.g., tune on S3 but also show a tuning-free robustness check, or tune per-grid) so the reader can gauge how much of the margin is tuning-induced.
  3. [Abstract / §6 (Generalizability) / §8, Table 3] The claim of 'more consistent zero-shot generalization than state-of-the-art baselines' is only partially supported by Table 3. DICG leads S1 and S3 at 4×4 (trained on 2×3) and S1–S2 at 4×4 (trained on 3×3); MOHITO leads S0 at 4×4; and PLATO's zero-shot S3 return trained on 2×3 tested on 3×3 is 4.95±28.12 versus DICG's 25.31±18.21 — a large, significant deficit. PLATO's advantage is concentrated in S2–S3 at 5×5 and in consistency across metrics (putouts, NOOP), which is a real but narrower claim. The abstract and conclusion should be revised to state precisely where PLATO wins and where it does not, rather than a blanket consistency claim.
minor comments (7)
  1. [§4.1 / §3] The one-action-per-task assumption (selecting a task uniquely determines the non-NO-OP action) is load-bearing for Lemma 2 and the actor interface. It is acknowledged as future work, but the limitation should be flagged earlier (e.g., in the abstract or introduction), since many OASYS domains have multi-action tasks.
  2. [§8 / §3] Full observability of relevant agents and tasks is assumed; combined with the fixed train seed (300) resetting every training episode to the same initial configuration, the diversity of training conditions is narrower than the 'openness' framing suggests. A short discussion of how sensitive results are to the fixed initial configuration would help.
  3. [Appendix A.2, proof of Theorem 1] The conclusions of proofs (3) and (4) as typeset ('(s,∆A_1),(s,∆A_2)∈π' and '(s,v_1),(s,v_2)∈V') appear to assert the opposite of the stated uniqueness conditions (3) and (4); presumably exclusivity is meant. Please correct the notation.
  4. [Appendix A.6] The reward bounds and the integer program (Eqs. 1–7) are presented without a clear connection to the method or experiments — they are used neither for evaluation nor for shaping. Clarify their purpose or move to a separate analysis section.
  5. [Table 5] MOHITO is tuned with hidden dimension 32 and batch size 16, versus 128/2048 for PLATO; the differing search ranges ({32,64,128} vs {64,128,256}) may handicap MOHITO. A sentence justifying the asymmetric ranges (beyond 'anchored to published defaults') would strengthen the fairness claim.
  6. [Various] Typos/typesetting: 'reewards' (A.6); missing spaces around 'open agent systems' in §1 ('referred to asopenagent systems'); 'I' stray character at the start of Proof (2) in A.2; Figure 1 is low-resolution and adds little beyond the text; Tables 19–21 and 31–36 are very dense — consider graphical summaries.
  7. [§1 / §7] The 'first MARL architecture to handle both AO and TO jointly' claim should be scoped more carefully given GPL (Rahman et al., 2021) for AO in ad hoc teamwork and MOHITO's hypergraph being, by the authors' own admission, expressive enough to represent AO. 'First to jointly model, train, and evaluate AO+TO without bounds' would be more defensible.

Circularity Check

0 steps flagged

No significant circularity: well-definedness proofs are self-contained function properties, and performance claims are external episode-return comparisons, not identities forced by fitted inputs or self-citation chains.

full rationale

PLATO is a methods-and-benchmarks paper. The load-bearing derivation chain has two independent pillars: (1) architectural definitions (pointer actor over current tasks; GNN critic on a shape-changing bipartite graph) plus appendix proofs that those maps are well-defined and permutation-invariant on unbounded TaAgO-MG spaces, and (2) MAPPO training with evaluation by episode return and related metrics against DICG, DGN, MOHITO, and heuristics on MOASEI wildfire setups. Pillar (1) proves properties of explicitly constructed functions (team statistics → MLP query; per-task keys → additive attention; GCN + mean pool → value); it does not define a quantity in terms of the quantity it claims to derive, nor fit a parameter and relabel the fit as a prediction. Pillar (2) is falsifiable external comparison under shared seeds and Bonferroni-corrected tests. Self-citations (Eck/Doshi/Soh openness survey, MOHITO/TaO-MG, MOASEI wildfire) supply problem framing, a baseline, and the domain; they do not force the empirical ranking or substitute for the well-definedness arguments. The known Algorithm-1 vs Lemma-3 mismatch (statistics query implemented; attention-based agent pointer proved) is a correctness/coverage gap about which formal AO guarantee applies, not a circular reduction of a claimed result to its inputs. No self-definitional loop, fitted-input-as-prediction, load-bearing uniqueness import, or renaming of a known empirical law was found.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 3 invented entities

Central claims rest on standard MARL/CTDE machinery, pointer and GNN modules, a domain-specific one-action-per-task wildfire model, and the new TaAgO-MG generator extension. Empirical superiority further depends on tuned optimizer hyperparameters and fixed openness probabilities in MOASEI setups. No new physical entities; invented constructs are the formal game and the PLATO modules.

free parameters (4)
  • Actor/critic hidden dimensions and learning rates (per method) = PLATO: hid=128, actor_lr=5e-4, critic_lr=9e-4 (Table 5)
    Chosen via Optuna (50 trials) on 3×3 S3 validation return; best PLATO values include hidden 128, actor lr 5e-4, critic lr 9e-4. Downstream native and zero-shot wins depend on these fits.
  • Openness event probabilities (spread, ignition, refill) = S1 spread=0.6; S2 ign=0.6 refill=0.8; S3 both 0.6 refill=0.6
    Hand-set environment knobs defining S0–S3 (e.g., spread 0.6, ignition 0.6, refill 1.0/0.8/0.6). They determine how hard joint AO+TO is and thus the measured gaps.
  • Team-statistic featurization (mean, var, min, max) = four feature-wise stats → R^{4 f_N}
    Architectural choice compressing variable |N_t| into fixed d_t; not learned end-to-end as a set encoder alternative. Lemma 1 invariance holds for these stats but sufficiency for AO is empirical.
  • PPO/GAE and training budget hyperparameters = batch=2048; horizon=100 steps; top-3 checkpoints by μ̂_k
    Batch size 2048, gradient clip 1.0, episode/horizon choices, checkpoint selection over second half of training affect reported means.
axioms (6)
  • domain assumption CTDE with shared policy and centralized critic is a valid training paradigm for cooperative wildfire agents.
    Section 4.4 / MAPPO setup; standard in MARL but not forced by TaAgO-MG alone.
  • ad hoc to paper Each task induces exactly one non-NO-OP action (suppress(x)); action space = {NO-OP} ∪ {a_x | x ∈ X_t}.
    Section 3–4.1 and Lemma 2; authors list multi-action tasks as future work.
  • domain assumption Local observations factor into fixed-width agent and task feature matrices for all agents/tasks relevant at t (full observability of those entities).
    Section 4.2 preprocessor; Section 8 flags partial observability as open.
  • ad hoc to paper Generator Ψ maps (M, X_t, N_t) to well-defined time-indexed MG components (Ψ_S, Ψ_A, Ψ_T, Ψ_R).
    TaAgO-MG definition in Section 3 extending MOHITO’s TaO-MG.
  • standard math Additive attention scores and MLP/GCN modules with fixed latent width define measurable policies/values on countably infinite open sets when combined with set aggregation.
    Theorem 1 / Appendix A.2; relies on standard facts about softmax, permutation-invariant aggregators, and GCNs on variable graphs.
  • domain assumption Wildfire reward structure (±2s putout/burnout) and fixed agent positions with range-limited suppress actions adequately represent OASYS coordination pressure.
    Section 5 and Appendix C.1; single-domain evaluation basis for generalization claims.
invented entities (3)
  • TaAgO-MG (Task-and-Agent-Open Markov Game) no independent evidence
    purpose: Formal setting that extends TaO-MG so both N and X vary under generator Ψ.
    Section 3 definition; enables well-definedness statements for joint openness.
  • PLATO pointer actor (team-stat query + task keys + NO-OP key) no independent evidence
    purpose: Produce categorical policies whose support matches the live task set under AO context.
    Algorithm 1 POINTERACTOR; core methodological object.
  • Bipartite agent–task GNN critic with residual mean pool no independent evidence
    purpose: Value estimation on variable |N_t|, |X_t| without fixed concatenation.
    Section 4.3; paired with pointer actor under MAPPO.

pith-pipeline@v1.2.0-grok45-kimik3 · 42988 in / 4168 out tokens · 79781 ms · 2026-07-31T01:48:00.689889+00:00 · methodology

0 comments
read the original abstract

Open agent systems (OASYS) are increasingly prevalent in real-world domains where the sets of agents and tasks change unpredictably over time. Such openness, including agent openness (AO) and task openness (TO), poses a fundamental challenge to multi-agent reinforcement learning (MARL), which typically assumes fixed state and action spaces. Existing methods address openness only partially: padding and masking approaches introduce artificial bounds, while recent graph-based or hypergraph methods handle one dimension of openness but still depend on restrictive assumptions. In this paper, we introduce Pointer Learner for Agent and Task Openness (PLATO), a pointer-network-based actor combined with a centralized graph neural network (GNN) critic, trained with multi-agent proximal policy optimization under a centralized training and decentralized execution paradigm. Our pointer-based actor outputs distributions directly over the current task set. This directly supports changing action spaces without masking or retraining. Our GNN critic encodes agent-task interactions as a graph that changes shape with task and agent composition. Together, these components consider AO and TO without the boundedness of existing approaches. We formalize PLATO in a Task-and-Agent-Open Markov Game (TaAgO-MG), extending prior task-open formulations, and prove it is well-defined over the resulting unbounded state and action spaces. We evaluate PLATO with the Methods for Open Agent Systems Evaluation Initiative (MOASEI) wildfire suppression domain, an environment designed for open multi-agent system evaluation, and we demonstrate strong performance and more consistent zero-shot generalization than state-of-the-art baselines in OASYS.

Figures

Figures reproduced from arXiv: 2607.25082 by Adam Eck, Alireza Saleh Abadi, Daniel Alan Redder, Leen-Kiat Soh, Prashant Doshi.

Figure 1
Figure 1. Figure 1: PLATO architecture for OASYS. It uses a pointer actor approach and CTDE training. Algorithm 1 PLATO: POINTER ACTOR, GNN CRITIC AND TRAINING 1: Inputs: env e, horizon T, actor πθ, critic Vϕ, initial observation O0 2: for t = 1 . . . T do 3: a t i ∼ POINTERACTOR(Ot i ) ∀i ∈ Nt 4: Ot , Statet , rt ← e(a t ) 5: Vˆ t ← GNNCRITIC(Statet ; ϕ); store (·) 6: compute GAE [Schulman et al., 2015] Aˆt ; update θ by PPO… view at source ↗
Figure 2
Figure 2. Figure 2: WILDFIRE example (3×3). Fires (tasks Xj ) ignite, spread, burn out, or are put out (i.e., extinguished) while agents (Ni) deplete suppressant or leave for repairs. Both sets evolve over time, creating an open multi￾agent system. → 3 × 3 (3 agents, 5 fires) and 4 × 4 (3 agents, 7 fires); 3 × 3 (3 agents, 4 fires) → 4 × 4 (3 agents, 7 fires) and 5 × 5 (3 agents, 8 fires) [PITH_FULL_IMAGE:figures/full_fig_p0… view at source ↗
Figure 3
Figure 3. Figure 3: Setups (S0–S3) on 2×3 showing the initial suppressant level that each firefighter agent has and the fire size. Setup 1 enables fire spread (endogenous TO). Setup 2 replaces spread with random ignitions (exogenous TO) and introduces a larger fire type, testing a qualitatively different openness mechanism. Setup 3 is the most demanding: it reintroduces fire spread alongside random ignitions and adds uneven i… view at source ↗
Figure 4
Figure 4. Figure 4: Setups (S0–S3) on 3×3 showing the suppressant level that each firefighter agent has and the fire size. Setup 1 enables fire spread (endogenous TO). Setup 2 replaces spread with random ignitions (exogenous TO) and introduces two larger fire types, testing a distinct openness mechanism. Setup 3 reintroduces fire spread alongside random ignitions and adds uneven initial suppressant levels, combining both TO m… view at source ↗
Figure 5
Figure 5. Figure 5: Setups (S0–S3) on 4 × 4 showing the suppressant level that each firefighter agent has and the fire size. Setup 1 enables fire spread (endogenous TO). Setup 2 replaces spread with random ignitions (exogenous TO) and introduces two larger fire types, testing a distinct openness mechanism. Setup 3 reintroduces fire spread alongside random ignitions and adds uneven initial suppressant across all six agents, co… view at source ↗
Figure 6
Figure 6. Figure 6: Example zero-shot evaluation. Agents are trained on a [PITH_FULL_IMAGE:figures/full_fig_p022_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

83 extracted references · 1 canonical work pages

  1. [1]

    Biometrika , volume=

    The Probable Error of a Mean , author=. Biometrika , volume=

  2. [2]

    2017 , publisher=

    Design and Analysis of Experiments , author=. 2017 , publisher=

    2017

  3. [3]

    and Molter, Hendrik and Niedermeier, Rolf and Zamaraev, Viktor and Zschoche, Philipp , title =

    Mertzios, George B. and Molter, Hendrik and Niedermeier, Rolf and Zamaraev, Viktor and Zschoche, Philipp , title =. Journal of Computer and System Sciences , year =

  4. [4]

    1987 , publisher=

    Multiple Comparison Procedures , author=. 1987 , publisher=

    1987

  5. [5]

    Shapiro, S. S. and Wilk, M. B. , title =. Biometrika , year =

  6. [6]

    Individual Comparisons by Ranking Methods , urldate =

    Frank Wilcoxon , journal =. Individual Comparisons by Ranking Methods , urldate =

  7. [7]

    Journal of Machine Learning Research , volume=

    Statistical Comparisons of Classifiers over Multiple Data Sets , author=. Journal of Machine Learning Research , volume=

  8. [8]

    1988 , publisher=

    Statistical Power Analysis for the Behavioral Sciences , author=. 1988 , publisher=

    1988

  9. [9]

    Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems , pages=

    Collaborative human task assignment for open systems , author=. Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems , pages=

    2016

  10. [10]

    IEEE Transactions on Systems, Man, and Cybernetics: Systems , volume=

    Strategic capability-learning for improved multiagent collaboration in ad hoc environments , author=. IEEE Transactions on Systems, Man, and Cybernetics: Systems , volume=. 2013 , publisher=

    2013

  11. [11]

    Software Architecture Attributes of Multi-agent Systems

    Shehory, Onn. Software Architecture Attributes of Multi-agent Systems. Agent-Oriented Software Engineering. 2001

    2001

  12. [12]

    AI Magazine , author =

    Decision making in open agent systems , volume =. AI Magazine , author =. 2023 , pages =. doi:10.1002/aaai.12131 , abstract =

    work page doi:10.1002/aaai.12131 2023

  13. [13]

    Advances in neural information processing systems , volume=

    Pointer networks , author=. Advances in neural information processing systems , volume=

  14. [14]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Deep reinforcement learning that matters , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  15. [15]

    and Morales, Peter and Allen, Ross and Kochenderfer, Mykel J

    Li, Sheng and Gupta, Jayesh K. and Morales, Peter and Allen, Ross and Kochenderfer, Mykel J. , month = feb, year =. Deep. doi:10.48550/arXiv.2006.11438 , abstract =

    Pith/arXiv arXiv doi:10.48550/arxiv.2006.11438 2006

  16. [16]

    arXiv preprint arXiv:1506.02438 , year=

    High-dimensional continuous control using generalized advantage estimation , author=. arXiv preprint arXiv:1506.02438 , year=

    Pith/arXiv arXiv

  17. [17]

    Graph convolutional reinforcement learning , journal =

    Jiang, Jiechuan and Dun, Chen and Huang, Tiejun and Lu, Zongqing , year =. Graph convolutional reinforcement learning , journal =

  18. [18]

    arXiv preprint arXiv:1911.04936 , year=

    Combinatorial optimization by graph pointer networks and hierarchical reinforcement learning , author=. arXiv preprint arXiv:1911.04936 , year=

    Pith/arXiv arXiv 1911

  19. [19]

    1997 , publisher=

    Information theory and statistics , author=. 1997 , publisher=

    1997

  20. [20]

    and Hopner, Niklas and Christianos, Filippos and Albrecht, Stefano V

    Rahman, Muhammad A. and Hopner, Niklas and Christianos, Filippos and Albrecht, Stefano V. , year =. Towards open ad hoc teamwork using graph-based policy learning , isbn =. International conference on machine learning , publisher =

  21. [21]

    Journal of Machine Learning Research , volume=

    Monotonic value function factorisation for deep multi-agent reinforcement learning , author=. Journal of Machine Learning Research , volume=

  22. [22]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Ad hoc autonomous agent teams: Collaboration without pre-coordination , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  23. [23]

    IEEE transactions on neural networks , author =

    The graph neural network model , volume =. IEEE transactions on neural networks , author =. 2008 , note =

    2008

  24. [24]

    Proximal policy optimization algorithms , journal =

    Schulman, John and Wolski, Filip and Dhariwal, Prafulla and Radford, Alec and Klimov, Oleg , year =. Proximal policy optimization algorithms , journal =

  25. [25]

    Advances in neural information processing systems , author =

    The surprising effectiveness of ppo in cooperative multi-agent games , volume =. Advances in neural information processing systems , author =. 2022 , pages =

    2022

  26. [26]

    nature , volume=

    Learning representations by back-propagating errors , author=. nature , volume=. 1986 , publisher=

    1986

  27. [27]

    Machine learning , volume=

    Q-learning , author=. Machine learning , volume=. 1992 , publisher=

    1992

  28. [28]

    Advances in Neural Information Processing Systems , volume=

    N-agent ad hoc teamwork , author=. Advances in Neural Information Processing Systems , volume=

  29. [29]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Graph pointer neural networks , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  30. [30]

    Proceedings of the Workshop on Autonomous Cybersecurity , pages=

    Entity-based reinforcement learning for autonomous cyber defence , author=. Proceedings of the Workshop on Autonomous Cybersecurity , pages=

  31. [31]

    Autonomous Agents and Multi-Agent Systems , volume=

    On-line estimators for ad-hoc task execution: learning types and parameters of teammates for effective teamwork , author=. Autonomous Agents and Multi-Agent Systems , volume=. 2022 , publisher=

    2022

  32. [32]

    arXiv preprint arXiv:2505.12811 , year=

    Dynamic Sight Range Selection in Multi-Agent Reinforcement Learning , author=. arXiv preprint arXiv:2505.12811 , year=

    Pith/arXiv arXiv

  33. [33]

    Proceedings of the IEEE , volume=

    Gradient-based learning applied to document recognition , author=. Proceedings of the IEEE , volume=. 2002 , publisher=

    2002

  34. [34]

    Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations , pages=

    Transformers: State-of-the-art natural language processing , author=. Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations , pages=

    2020

  35. [35]

    Advances in neural information processing systems , volume=

    Attention is all you need , author=. Advances in neural information processing systems , volume=

  36. [36]

    Neural computation , volume=

    Long short-term memory , author=. Neural computation , volume=. 1997 , publisher=

    1997

  37. [37]

    The journal of machine learning research , volume=

    Dropout: a simple way to prevent neural networks from overfitting , author=. The journal of machine learning research , volume=. 2014 , publisher=

    2014

  38. [38]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Deep residual learning for image recognition , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

  39. [39]

    AnnMathStatist , volume=

    Robust estimation of alocation parmneter , author=. AnnMathStatist , volume=

  40. [40]

    arXiv preprint arXiv:1409.0473 , year=

    Neural machine translation by jointly learning to align and translate , author=. arXiv preprint arXiv:1409.0473 , year=

    Pith/arXiv arXiv

  41. [41]

    arXiv preprint arXiv:1803.08475 , year=

    Attention, learn to solve routing problems! , author=. arXiv preprint arXiv:1803.08475 , year=

    Pith/arXiv arXiv

  42. [42]

    arXiv preprint arXiv:1412.6980 , year=

    Adam: A method for stochastic optimization , author=. arXiv preprint arXiv:1412.6980 , year=

    Pith/arXiv arXiv

  43. [43]

    Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining , pages=

    Optuna: A next-generation hyperparameter optimization framework , author=. Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining , pages=

  44. [44]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Scalable decision-theoretic planning in open and typed multiagent systems , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  45. [45]

    Biometrika , volume=

    The generalization of ‘STUDENT'S’problem when several different population varlances are involved , author=. Biometrika , volume=. 1947 , publisher=

    1947

  46. [46]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Counterfactual multi-agent policy gradients , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  47. [47]

    arXiv preprint arXiv:2507.05469 , year=

    Inaugural MOASEI Competition at AAMAS'2025: A Technical Report , author=. arXiv preprint arXiv:2507.05469 , year=

    Pith/arXiv arXiv 2025

  48. [48]

    arXiv preprint arXiv:1609.02907 , year=

    Semi-supervised classification with graph convolutional networks , author=. arXiv preprint arXiv:1609.02907 , year=

    Pith/arXiv arXiv

  49. [49]

    arXiv preprint arXiv:2001.05992 , year=

    Provable benefit of orthogonal initialization in optimizing deep linear networks , author=. arXiv preprint arXiv:2001.05992 , year=

    Pith/arXiv arXiv 2001

  50. [50]

    Advances in neural information processing systems , volume=

    Deep sets , author=. Advances in neural information processing systems , volume=

  51. [51]

    Advances in neural information processing systems , volume=

    Multi-agent actor-critic for mixed cooperative-competitive environments , author=. Advances in neural information processing systems , volume=

  52. [52]

    The 41st Conference on Uncertainty in Artificial Intelligence , year=

    MOHITO: Multi-Agent Reinforcement Learning using Hypergraphs for Task-Open Systems , author=. The 41st Conference on Uncertainty in Artificial Intelligence , year=

  53. [53]

    IEEE transactions on knowledge and data engineering , volume=

    A survey on multi-task learning , author=. IEEE transactions on knowledge and data engineering , volume=. 2021 , publisher=

    2021

  54. [54]

    Journal of Machine Learning Research , volume=

    Curriculum learning for reinforcement learning domains: A framework and survey , author=. Journal of Machine Learning Research , volume=

  55. [55]

    arXiv preprint arXiv:1909.10008 , year=

    Multi-task learning and catastrophic forgetting in continual reinforcement learning , author=. arXiv preprint arXiv:1909.10008 , year=

    Pith/arXiv arXiv 1909

  56. [56]

    Plos one , volume=

    The limitations of automatically generated curricula for continual learning , author=. Plos one , volume=. 2024 , publisher=

    2024

  57. [57]

    Sensors , volume=

    Reinforcement-learning-based routing and resource management for Internet of Things environments: Theoretical perspective and challenges , author=. Sensors , volume=. 2023 , publisher=

    2023

  58. [58]

    arXiv preprint arXiv:1412.3555 , year=

    Empirical evaluation of gated recurrent neural networks on sequence modeling , author=. arXiv preprint arXiv:1412.3555 , year=

    Pith/arXiv arXiv

  59. [59]

    International conference on machine learning , pages=

    Set transformer: A framework for attention-based permutation-invariant neural networks , author=. International conference on machine learning , pages=. 2019 , organization=

    2019

  60. [60]

    ACM Computing Surveys , volume=

    Efficient Transformers: A Survey , author=. ACM Computing Surveys , volume=

  61. [61]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Film: Visual reasoning with a general conditioning layer , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  62. [62]

    Operations research , volume=

    The traveling-salesman problem , author=. Operations research , volume=. 1956 , publisher=

    1956

  63. [63]

    Leslie Lamport , title =

  64. [64]

    The Knowledge Engineering Review , volume =

    Intelligent Agents: Theory and Practice , author =. The Knowledge Engineering Review , volume =

  65. [65]

    Artificial Intelligence , volume =

    Collaborative Plans for Complex Group Action , author =. Artificial Intelligence , volume =

  66. [66]

    Logics of programs: axiomatics and descriptive power

    David Harel. Logics of programs: axiomatics and descriptive power. 1978

    1978

  67. [67]

    arXiv preprint arXiv:2510.27659 , year=

    Challenges in Credit Assignment for Multi-Agent Reinforcement Learning in Open Agent Systems , author=. arXiv preprint arXiv:2510.27659 , year=

    arXiv

  68. [68]

    Clarkson

    Kenneth L. Clarkson. Algorithms for Closest-Point Problems (Computational Geometry). 1985

    1985

  69. [69]

    A More Perfect Union

    Barack Obama. A More Perfect Union. 2008

    2008

  70. [70]

    The fountain of youth

    Joseph Scientist. The fountain of youth. 2009

    2009

  71. [71]

    Sam Anzaroot and Andrew McCallum , title =

  72. [72]

    Proceedings of the 20th International Colloquium on Automata, Languages and Programming , series =

    Maintaining Discrete Probability Distributions Optimally , author =. Proceedings of the 20th International Colloquium on Automata, Languages and Programming , series =. 1993 , publisher =

    1993

  73. [73]

    Donald E. Knuth. The Art of Computer Programming, Vol. 1: Fundamental Algorithms. 1997

    1997

  74. [74]

    Anisi , title =

    David A. Anisi , title =

  75. [75]

    Calmet and A

    J. Calmet and A. Daemi and R. Endsuleit and T. Mie , booktitle =. A Liberal Approach to Openness in Societies of Agents , volume =

  76. [76]

    Jamroga and A

    W. Jamroga and A. Meski and M. Szreter , booktitle =. Modularity and Openness in Modeling Multi-Agent Systems , year =

  77. [77]

    Individual Planning in Open and Typed Agent Systems , year =

    Chandrasekaran, Muthukumaran and Eck, Adam and Doshi, Prashant and Soh, Leenkiat , booktitle =. Individual Planning in Open and Typed Agent Systems , year =

  78. [78]

    Decision-theoretic planning with communication in open multiagent systems , year =

    Kakarlapudi, Anirudh and Anil, Gayathri and Eck, Adam and Doshi, Prashant and Soh, Leen-Kiat , booktitle =. Decision-theoretic planning with communication in open multiagent systems , year =

  79. [79]

    Littman , booktitle =

    Michael L. Littman , booktitle =. Markov games as a framework for multi-agent reinforcement learning , year =

  80. [80]

    L. S. Shapley , doi =. Stochastic Games* , url =. https://www.pnas.org/doi/pdf/10.1073/pnas.39.10.1095 , journal =

    doi:10.1073/pnas.39.10.1095

Showing first 80 references.