Pith. sign in

REVIEW 3 major objections 4 minor 57 references

Investigating Relational State Abstraction in Collaborative MARL

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A critic that represents the state as a graph of relative spatial relations between agents and objects—discarding absolute positions, distances, and velocities—learns collaborative multi-agent tasks faster and to competitive final…

desk verdict Solid empirical recipe with a real inductive bias, but the abstract overclaims the continuous results and the distance-free abstraction has a known cost that should be stated up front. read the letter →

arxiv 2412.15388 v1 pith:C6AKUXLY submitted 2024-12-19 cs.AI cs.LGcs.MA

Sharlin Utke , Jeremie Houssineau , Giovanni Montana This is my paper · ORCID
classification cs.AIcs.LGcs.MA
keywords multi-agentreinforcementlearningrelationalstateabstractionspatialinductivebiasgraphneuralnetworksconvolutionalsampleefficiencycollaborativetasksheterogeneousagents
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a deliberately lossy representation—keeping only the relative spatial arrangement of agents and objects and throwing away absolute coordinates, distances, and velocities—can make multi-agent reinforcement learning more sample-efficient without hurting final performance. The answer, according to the authors, is yes: their MARC architecture transforms each observation into a spatial graph and processes it with a shared relational graph network in the critic, and on four discrete collaborative tasks it learns as fast or faster than six baselines, with up to a 7.3x step advantage over the second-best method. On a novel heterogeneous pick-and-place task it ends with 69.9% higher reward, and on a continuous navigation domain it beats the graph-based InforMARL baseline while staying competitive with attention-based methods. The paper also reports improved generalization to changed numbers of agents and objects, and an ablation shows the specific choice of coarse spatial relations matters: local-only relations and full-grid graphs both underperform.

What carries the argument

The central object is the relational state abstraction: an observation is turned into a spatial graph whose nodes are agents and environment objects, and whose directed edges are six spatial predicates (left, right, top, bottom, adjacent, aligned) evaluated from relative coordinates. Absolute positions, distances, and velocities are deliberately dropped, which makes the representation translation-invariant and invariant to entity ordering. This graph is passed through a shared R-GCN (Relational Graph Convolutional Network) layer with per-relation weight matrices, followed by channel-wise max-pooling to produce a fixed-size observation encoding; each agent then has its own small MLP head that combines this encoding with the joint action to output a Q-value in a soft actor-critic loop.

What would settle it

In the continuous Target task, augment MARC's graph with distance and velocity edge features and compare final reward to the published lean version; if the augmented critic clearly outperforms it, the abstraction's core assumption is violated. The paper's own report that MARC takes 1-2 extra steps and collides more often in that domain is a concrete symptom to look for.

Watch

Extended reading notes

Core claim

The paper claims that replacing an agent's raw observation with a graph that keeps only relative spatial relationships between entities—left/right/top/bottom, adjacency, and alignment—and processing that graph through a shared relational graph neural network in the critic makes multi-agent reinforcement learning dramatically more sample-efficient while matching or beating state-of-the-art baselines on final performance. On the discrete tasks, MARC reaches the same performance as the best baseline in as little as 1/7.3 of the environment steps, and on a new heterogeneous pick-and-place environment it ends with 69.9% higher reward. The same principle carries over to a continuous navigation task, where MARC beats the graph-based InforMARL baseline but trails the attention-based MAAC and GA-AC slightly, with the paper attributing the small gap to the loss of distance and velocity information.

Load-bearing premise

The load-bearing premise is that the relative spatial layout of entities, without distances, velocities, or absolute locations, carries enough information for near-optimal decisions in these tasks; if a task truly requires knowing exactly how far or how fast something is, this abstraction deletes the information the policy needs.

Editorial extensions

If this is right

  • If MARC's claim holds, then a minimal relational bias—just six spatial predicates, no distances—can replace explicit agent-to-agent communication or attention in collaborative settings, lowering the bar for sample-efficient MARL.
  • The reported 7.3x step advantage on LBF-15x15 suggests that on large-agent collaborative tasks, relational abstraction can be the difference between feasibility and impracticality.
  • The abstraction transfers across grid and continuous domains, and the paper's MAPPO+RC appendix result indicates the encoder is backbone-agnostic; other MARL algorithms can bolt it on.
  • The translation invariance means policies learned in one absolute layout should transfer to the same relative layout elsewhere, which the generalization experiments (varying agent/fruit counts) begin to support.
  • The fixed sparse-relation graph also cuts the critic's input dimensionality relative to joint-observation critics, which is part of why the method is computationally cheap as well as sample-efficient.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next step, unstated in the paper, is to learn the relation set itself rather than fixing six predicates; the paper's own ablation shows that both too-coarse and too-fine relation sets hurt, suggesting there is an optimal granularity that could be discovered per task.
  • Because the shared R-GCN is the only cross-agent information channel, MARC effectively implements implicit coordination through a learned global representation; one could test whether the learned entity embeddings become interpretable role or state codes.
  • The paper only tests collaborative reward settings; the same abstraction should be probed in mixed or competitive games, where relative spatial structure also matters but the value decomposition may behave differently.
  • The continuous-domain trade-off (faster learning but 1-2 extra steps and collisions) suggests a hybrid that starts with the lean graph and gradually adds distance edges during training; this is a testable extension the paper does not explore.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces MARC (Multi-Agent Relational Critic), an actor-critic MARL method whose critic encodes observations as a graph over agents and environment objects. The graph edges are Boolean spatial predicates (left/right/top/bottom/adjacent/aligned) and the node features are non-spatial attributes; the representation is passed through a shared R-GCN and max-pooled before each agent's value head. The authors evaluate MARC on four discrete collaborative tasks (a new Collaborative Pick and Place environment, two Level-Based Foraging variants, and Wolfpack) and one continuous task (Target), comparing against MAAC, GA-AC, QMIX, MAA2C, MAPPO, and InforMARL. They report improved sample efficiency and asymptotic performance on the discrete tasks, competitive performance on the continuous task, generalization results on Level-Based Foraging, and ablations over relation sets, graph architectures, and entity granularity. They also show that the relational encoder can be combined with a different backbone (MAPPO).

Significance. If the reported discrete-task results are robust, the paper makes a useful empirical contribution: it shows that a cheap relational abstraction, implemented purely in the critic, can improve sample efficiency and asymptotic performance in cooperative MARL tasks with spatial structure. The strengths of the paper include public code for MARC and the new CPP environment, a reasonably broad set of baselines, explicit ablations on the relation set and graph architecture, and a supporting experiment that combines the encoder with MAPPO. The significance is tempered, however, because the central claim is stated more broadly than the evidence supports: the continuous-domain results do not show an asymptotic advantage, and the abstraction is not shown to preserve decision-relevant information in general.

major comments (3)
  1. [Section 4.3, Table 1, Figure 2] The claim in Section 4.3 that 'asymptotically, MARC is competitive and outperforms all baselines across the implemented tasks' is contradicted by the continuous Target results in Table 1. For 3 agents, MARC (212.7 ± 5.7) trails MAAC (236.1 ± 2.9), GA-AC (236.6 ± 3.5), and MAA2C (233.5 ± 2.1); for 7 agents, MARC (468.2 ± 4.2) trails MAAC (527.9 ± 5.4) and GA-AC (530.6 ± 3.4). The abstract and Section 4.3 need to restrict the 'outperforms all baselines' claim to the discrete tasks, or provide a substantive argument for why the continuous shortfall is acceptable for the paper's central claim.
  2. [Section 3.2 and Appendix 'Invariances of the State Abstraction'] The paper asserts that removing absolute positions and distances 'does not sacrifice essential information for decision-making' (Section 3.2), but the appendix only proves translation invariance of individual spatial predicates, not that the pooled abstract state preserves the optimal value function or transition structure. In the Boolean-edge representation with max-pooling, configurations with different distances to a target can map to the same abstract state even when the optimal action differs, so the abstraction is not shown to be a sufficient statistic for returns. Section 4.5 itself reports that in the continuous domain MARC takes 1-2 extra steps and suffers collisions because it lacks fine-grained proximity and velocity information. The authors should either provide a value-preservation argument for the discrete settings or explicitly qualify the 'without sacrificing essential information' claim as an empirical design assumption whose failure modes are documented in the continuous case.
  3. [Section 4.5 and Table 1 vs. Figure 7] The continuous-domain results are hard to interpret because Table 1 reports a single 'MARC' entry, while Figure 7 in the appendix plots two variants, 'MARC (5 relations)' and 'MARC (9 relations)', and the technical appendix says the finer-grained octagonal relations improve asymptotic performance. The main text should specify which MARC variant is used in Table 1 and explain whether the 5-relation or 9-relation results correspond to the reported numbers. This is necessary for reproducibility and for assessing the paper's claim that the default abstraction is competitive in continuous domains.
minor comments (4)
  1. [Section 4.4] The generalization evaluation compares MARC only against MAPPO and only on a single source configuration (LBF-10x10-4a-4f-coop). Reporting error bars or multiple seeds for the generalization numbers, and ideally comparing against one additional baseline, would make the claim of 'potential for generalization' stronger. As written, the evidence is suggestive but thin.
  2. [Technical Appendix, Table 2] The hyperparameter search for MARC, MAAC, and GA-AC is performed on LBF-10x10-4a-4f-coop and then applied across all tasks; the text should explicitly acknowledge that this could favor the selected method on that task and that per-task tuning might change the relative results in other environments.
  3. [Section 4.5] The MAA2C result for 7 agents in Table 1 has a standard deviation of 393.4, which is several times larger than the mean; this suggests highly unstable runs and should be discussed or flagged in the table caption, since it materially affects the comparison in the 7-agent setting.
  4. [General] There are minor formatting and typographical issues: axis labels in Figure 2 show '0.00 0.25 0.50...' instead of a proper tick format, the reference for Abdel-Aziz et al. has a misplaced 'In', and some numerical values are given as '1e6' rather than '10^6'. These should be cleaned up in the final version.

Circularity Check

1 steps flagged · score 2.0 of 10

Mostly external empirical comparison; only mild circularity from selecting MARC's hyperparameters on the same LBF-10x10-4a-4f-coop task used for a headline performance claim.

  1. fitted input called prediction [Technical Appendix (Training Procedure and Hyperparameter) and Section 4.3]
    "To find the optimal hyperparameters, we engaged in a random search for MARC on the complex task of LBF-10x10-4a-4f-coop, selecting the hyperparameters that perform best in terms of asymptotic performance. These hyperparameters, documented in Table 2, are then applied consistently across all scenarios."

    The headline claims for LBF-10x10-4a-4f-coop (35.2% asymptotic gain and 26% of max returns in 1e6 steps in Section 4.3) are reported for the same task on which MARC hyperparameters were selected by maximizing asymptotic performance (appendix). The asymptotic and sample-efficiency numbers on that task are thus partly selection outcomes rather than independent predictions. Other tasks are not fitted this way, so the circularity is partial and task-specific.

full rationale

This is an empirical architecture paper rather than a derivation, and its central comparative claims are tested against external baseline implementations (MAAC, GA-AC, QMIX, MAA2C, MAPPO, InforMARL), so the main result is not equivalent by construction to an assumed quantity. There is no load-bearing self-citation, no imported uniqueness theorem, and no ansatz smuggled in via citation: the relational abstraction is presented as an explicit hypothesis about which spatial information matters, and the appendix only proves translation invariance of the chosen predicates, not value equivalence. The distance-blindness of the abstraction is a real correctness risk, and Section 4.5 documents its concrete cost (1-2 extra steps and collisions) in the continuous domain, but that is a limitation, not circularity. The one genuine circular-adjacent issue is the hyperparameter search on LBF-10x10-4a-4f-coop: the appendix states that MARC's hyperparameters were selected on that task using asymptotic performance, and Section 4.3 uses the same task to report MARC's largest asymptotic gain and a headline sample-efficiency number. That makes the LBF-10x10 comparison partly self-referential, though the remaining tasks and ablations retain independent content, so the overall circularity is mild.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The empirical claims rest on several domain assumptions about the observability and sufficiency of spatial predicates, plus hyperparameter choices made by the authors. No new physical or mathematical entities are introduced.

free parameters (6)
  • Entropy coefficient alpha = 0.05 (main text) / 0.01 (Table 2)
    Chosen hyperparameter that controls the exploration-exploitation trade-off; inconsistent values are reported in the text and table.
  • Learning rate = 0.001
    Selected via random search on LBF-10x10-4a-4f-coop and then applied across all tasks.
  • Batch size = 1024
    Selected via random search on LBF-10x10-4a-4f-coop and applied across all tasks.
  • Discount factor gamma = 0.99
    Standard hyperparameter set by random search and used uniformly.
  • Replay buffer length = 1e5
    Chosen to save memory; the paper states performance is robust to buffer length changes.
  • Number of R-GCN layers = 1 (single-layered in main procedure)
    Selected from the range 1|2|3 via random search; affects the depth of relational abstraction.
assumptions (5)
  • domain assumption The relative spatial relations can be extracted from observations, and entities' positions and attributes are available.
    Section 3.2 assumes positions and attributes of agents and objects can be extracted to construct entity set V and spatial predicates.
  • domain assumption Only relative spatial information is essential; absolute positions and distances are unnecessary.
    Section 3.2 states 'the relative positioning of entities is relevant, not their absolute positions' and the edges do not feature distance.
  • ad hoc to paper Translation invariance of the relational rules preserves optimal decision-making.
    The Technical Appendix shows translation shifts leave relational rules unchanged, but relies on the premise that translation-equivalent states have equal value, which is assumed, not proven for the tasks.
  • domain assumption Sparse-reward spatial tasks are representative of sample-efficiency challenges in MARL.
    Section 4.1 hypothesizes the abstraction works in spatially complex coordination tasks with sparse rewards; generalization beyond this setting is not established.
  • domain assumption A shared encoder allows agents to exchange information implicitly without communication.
    Section 3.3: each critic receives only its own observation and exchanges information by collectively learning encoder parameters; this is assumed sufficient for coordination.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Investigating Relational State Abstraction in Collaborative MARL." pith.science (2026). https://pith.science/paper/C6AKUXLY

@misc{pith2026241215388,
  author       = {Pith},
  title        = {Pith review of: Investigating Relational State Abstraction in Collaborative MARL},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C6AKUXLY}},
  note         = {Machine review of arXiv:2412.15388}
}
read the original abstract

This paper explores the impact of relational state abstraction on sample efficiency and performance in collaborative Multi-Agent Reinforcement Learning. The proposed abstraction is based on spatial relationships in environments where direct communication between agents is not allowed, leveraging the ubiquity of spatial reasoning in real-world multi-agent scenarios. We introduce MARC (Multi-Agent Relational Critic), a simple yet effective critic architecture incorporating spatial relational inductive biases by transforming the state into a spatial graph and processing it through a relational graph neural network. The performance of MARC is evaluated across six collaborative tasks, including a novel environment with heterogeneous agents. We conduct a comprehensive empirical analysis, comparing MARC against state-of-the-art MARL baselines, demonstrating improvements in both sample efficiency and asymptotic performance, as well as its potential for generalization. Our findings suggest that a minimal integration of spatial relational inductive biases as abstraction can yield substantial benefits without requiring complex designs or task-specific engineering. This work provides insights into the potential of relational state abstraction to address sample efficiency, a key challenge in MARL, offering a promising direction for developing more efficient algorithms in spatially complex environments.

Figures

Figures reproduced from arXiv: 2412.15388 by the authors.

Figure 1
Figure 1. Overview of our MARC architecture on the example of level-based foraging. Without adding information, the obser [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Mean average performance and 95% confidence interval for all discrete tasks. For each model, we run 3 random seeds. in abstraction between a simple and efficient abstraction and removing too much information. For example, as environ￾ment objects also have velocity in the target task, the agents ideally have a more fine-grained understanding of the prox￾imity to other objects, rather than just knowing once they are a… view at source ↗
Figure 3
Figure 3. Collaborative pick and place environment on a 5x5 [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (8 more)
Figure 6
Figure 6. Figure 6: Navigation task with 3 agents aiming to reach tar [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 5
Figure 5. Figure 5: Wolfpack environment with 3 predator agents co [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 7
Figure 7. Figure 7: Mean average performance and 95% confidence interval for the continuous target tasks. For each model, we run 3 random seeds. 0 1 2 3 4 5 6 7 8 Environment Steps ×106 −10 −8 −6 −4 −2 0 2 Average Rewards full grid RGAT GAT all relations default relations local relations …
Figure 8
Figure 8. Figure 8: Training curves for a 10x10 CPP environment with [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Examples of the spatial determination rules employed in our methodology: given an entity and its position on a grid, [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Example of the spatial clusters created by the 9 [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: Mean average performance and 95% confidence interval for a test version of CPP on a 7x7 grid with 1 picker agent, 1 delivery agent, 1 box and 1 goal locations. MAPPO denotes the original algorithm and MAPPO+RC the combi￾nation of MAPPO with a relational critic. For ea…
Figure 12
Figure 12. Figure 12: Examples of states that are considered to be equivalent in our critic architecture: translation shifts do not affect the [PITH_FULL_IMAGE:figures/full_fig_p016_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 47 canonical work pages

  1. [1]

    K.; Elbamby, M

    Abdel-Aziz, M. K.; Elbamby, M. S.; Samarakoon, S.; and Bennis, M. 2024. Cooperative Multi-Agent Learning for Navigation via Structured State Abstraction. In IEEE Transactions on Communications

    work page 2024

  2. [2]

    Abel, D. 2022. A Theory of Abstraction in Reinforcement Learning. Ph.D. thesis, Brown University

    work page 2022

  3. [3]

    Agarwal, A.; Kumar, S.; Sycara, K.; and Lewis, M. 2020. Learning Transferable Cooperative Behavior in Multi-Agent Teams. In Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, 1741–1743

    work page 2020

  4. [4]

    K.; and Tumer, K

    Agogino, A. K.; and Tumer, K. 2012. A multiagent approach to managing air traffic flow. Autonomous Agents and Multi-Agent Systems, 24: 1--25

    work page 2012

  5. [5]

    L.; Kohli, P.; Battaglia, P

    Bapst, V.; Sanchez-Gonzalez, A.; Doersch, C.; Stachenfeld, K. L.; Kohli, P.; Battaglia, P. W.; and Hamrick, J. B. 2019. Structured agents for physical construction. In Proceedings of the 36th International Conference on Machine Learning

    work page 2019

  6. [6]

    Battaglia, P.; Hamrick, J. B. C.; Bapst, V.; Sanchez, A.; Zambaldi, V.; Malinowski, M.; Tacchetti, A.; Raposo, D.; Santoro, A.; Faulkner, R.; Gulcehre, C.; Song, F.; Ballard, A.; Gilmer, J.; Dahl, G. E.; Vaswani, A.; Allen, K.; Nash, C.; Langston, V. J.; Dyer, C.; Heess, N.; Wierstra, D.; Kohli, P.; Botvinick, M.; Vinyals, O.; Li, Y.; and Pascanu, R. 2018...

    arXiv 2018

  7. [7]

    C.; and Vincent, P

    Bengio, Y.; Courville, A. C.; and Vincent, P. 2012. Representation Learning: A Review and New Perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35: 1798--1828

    work page 2012

  8. [8]

    Busbridge, D.; Sherburn, D.; Cavallo, P.; and Hammerla, N. Y. 2019. Relational Graph Attention Networks. arXiv:1904.05811

    arXiv 2019

Show all 57 references
  1. [9]

    Christianos, F.; Schäfer, L.; and Albrecht, S. V. 2020. Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning. In Advances in Neural Information Processing Systems, 10707--10717

    2020

  2. [10]

    Croonenborghs, T.; Tuyls, K.; Ramon, J.; and Bruynooghe, M. 2006. Multi-agent Relational Reinforcement Learning. In Learning and Adaption in Multi-Agent Systems, 192--206

    2006

  3. [11]

    Driessens, K.; and D z eroski, S. 2001. Integrating guidance into relational reinforcement learning. Machine Learning, 116--127

    2001

  4. [12]

    D z eroski, S.; De Raedt, L.; and Driessens, K. 2001. Relational reinforcement learning. Machine Learning, 7--52

    2001

  5. [13]

    Garnelo, M.; Arulkumaran, K.; and Shanahan, M. 2016. Towards Deep Symbolic Reinforcement Learning. arXiv:1609.05518

    2016 arXiv

  6. [14]

    S.; Riley, P

    Gilmer, J.; Schoenholz, S. S.; Riley, P. F.; Vinyals, O.; and Dahl, G. E. 2017. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning, 1263--1272

    2017

  7. [15]

    Haarnoja, T.; Zhou, A.; Abbeel, P.; and Levine, S. 2018. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. In Proceedings of the 35th International Conference on Machine Learning, 1861--1870

    2018

  8. [16]

    Hamilton, W.; Ying, Z.; and Leskovec, J. 2017. Inductive representation learning on large graphs. Advances in neural information processing systems, 30

    2017

  9. [17]

    Iqbal, S.; and Sha, F. 2019. Actor-Attention-Critic for Multi-Agent Reinforcement Learning. In Proceedings of the 36th International Conference on Machine Learning, 2961--2970

    2019

  10. [18]

    Jiang, J.; Dun, C.; Huang, T.; and Lu, Z. 2020. Graph Convolutional Reinforcement Learning. In International Conference on Learning Representations

    2020

  11. [19]

    Jiang, Z.; Minervini, P.; Jiang, M.; and Rockt\" a schel, T. 2021. Grid-to-Graph: Flexible Spatial Relational Inductive Biases for Reinforcement Learning. In Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, 674–682

    2021

  12. [20]

    V.; Ribeiro, A.; and Kumar, V

    Khan, A.; Tolstaya, E. V.; Ribeiro, A.; and Kumar, V. R. 2019. Graph Policy Gradients for Large Scale Robot Control. In Conference on Robot Learning, 823--834

    2019

  13. [21]

    P.; and Ba, J

    Kingma, D. P.; and Ba, J. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings

    2015

  14. [22]

    Kipf, T.; Fetaya, E.; Wang, K.-C.; Welling, M.; and Zemel, R. 2018. Neural Relational Inference for Interacting Systems. In Proceedings of the 35th International Conference on Machine Learning, 2688--2697

    2018

  15. [23]

    Kipf, T.; van der Pol, E.; and Welling, M. 2020. Contrastive Learning of Structured World Models. In International Conference on Learning Representations

    2020

  16. [24]

    Li, G.; Xiao, G.; Zhang, J.; Liu, J.; and Shen, Y. 2022. Towards Relational Multi-Agent Reinforcement Learning via Inductive Logic Programming. In Artificial Neural Networks and Machine Learning, 99--110

    2022

  17. [25]

    Li, J.; Yang, F.; Tomizuka, M.; and Choi, C. 2020. EvolveGraph: Multi-Agent Trajectory Prediction with Dynamic Relational Reasoning. In Advances in Neural Information Processing Systems, 19783--19794

    2020

  18. [26]

    Li, L.; Walsh, T.; and Littman, M. 2006. Towards a Unified Theory of State Abstraction for MDPs. In Proceedings of the Ninth International Symposium on Artificial Intelligence and Mathematics

    2006

  19. [27]

    A.; and Schwing, A

    Liu, I.-J.; Ren, Z.; Yeh, R. A.; and Schwing, A. G. 2021. Semantic tracklets: An object-centric representation for visual multi-agent reinforcement learning. In International Conference on Intelligent Robots and Systems, 5603--5610

    2021

  20. [28]

    Liu, Y.; Wang, W.; Hu, Y.; Hao, J.; Chen, X.; and Gao, Y. 2020. Multi-agent game abstraction via graph attention neural network. In Proceedings of the AAAI conference on artificial intelligence, 7211--7218

    2020

  21. [29]

    Lowe, R.; Wu, Y.; Tamar, A.; Harb, J.; Abbeel, P.; and Mordatch, I. 2017. Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments. In Advances in Neural Information Processing Systems

    2017

  22. [30]

    Problems of Redundancy in Information and Control Systems

    Malysheva, A.; Kudenko, D.; and Shpilman, A. 2019. MAGNet: Multi-agent Graph Network for Deep Multi-agent Reinforcement Learning. In XVI International Symposium "Problems of Redundancy in Information and Control Systems" (REDUNDANCY), 171--176

    2019

  23. [31]

    Matignon, L.; Jeanpierre, L.; and Mouaddib, A.-I. 2012. Coordinated Multi-Robot Exploration Under Communication Constraints Using Decentralized Markov Decision Processes. In Proceedings of the AAAI Conference on Artificial Intelligence, 2017--2023

    2012

  24. [32]

    A.; Veness, J.; Bellemare, M

    Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A. A.; Veness, J.; Bellemare, M. G.; Graves, A.; Riedmiller, M. A.; Fidjeland, A.; Ostrovski, G.; Petersen, S.; Beattie, C.; Sadik, A.; Antonoglou, I.; King, H.; Kumaran, D.; Wierstra, D.; Legg, S.; and Hassabis, D. 2015. Human-leve...

    2015

  25. [33]

    Mohan, A.; Zhang, A.; and Lindauer, M. 2024. Structure in Deep Reinforcement Learning: A Survey and Open Problems. Journal of Artificial Intelligence Research, 79

    2024

  26. [34]

    Nayak, S.; Choi, K.; Ding, W.; Dolan, S.; Gopalakrishnan, K.; and Balakrishnan, H. 2023. Scalable Multi-Agent Reinforcement Learning through Intelligent Information Aggregation. In Proceedings of the 40th International Conference on Machine Learning, 25817--25833

    2023

  27. [35]

    Niu, Y.; Paleja, R.; and Gombolay, M. 2021. Multi-Agent Graph-Attention Communication and Teaming. In Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, 964–973

    2021

  28. [36]

    Papoudakis, G.; Christianos, F.; Sch \"a fer, L.; and Albrecht, S. V. 2020. Benchmarking multi-agent deep reinforcement learning algorithms in cooperative tasks. In NeurIPS Datasets and Benchmarks

    2020

  29. [37]

    Ponsen, M.; Croonenborghs, T.; Tuyls, K.; Ramon, J.; Driessens, K.; Herik, H.; and Postma, E. 2010. Learning with Whom to Communicate Using Relational Reinforcement Learning, 45--63. Studies in Computational Intelligence. Springer

    2010

  30. [38]

    Rahman, A.; Carlucho, I.; H \"o pner, N.; and Albrecht, S. V. 2023. A general learning framework for open ad hoc teamwork using graph-based policy learning. Journal of Machine Learning Research, 24: 1--74

    2023

  31. [39]

    Rashid, T.; Samvelyan, M.; Witt, C. S. D.; Farquhar, G.; Foerster, J. N.; and Whiteson, S. 2018. QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning. In Proceedings of the 35th International Conference on Machine Learning

    2018

  32. [40]

    Samet, H. 1984. The Quadtree and Related Hierarchical Data Structures. ACM Computing Surveys, 16: 187–260

    1984

  33. [41]

    Sanner, S.; and Boutilier, C. 2009. Practical solution techniques for first-order MDPs. Artificial Intelligence, 173: 748--788

    2009

  34. [42]

    C.; Hagenbuchner, M.; and Monfardini, G

    Scarselli, F.; Gori, M.; Tsoi, A. C.; Hagenbuchner, M.; and Monfardini, G. 2009. The Graph Neural Network Model. IEEE Transactions on Neural Networks, 20: 61--80

    2009

  35. [43]

    N.; Bloem, P.; van den Berg, R.; Titov, I.; and Welling, M

    Schlichtkrull, M.; Kipf, T. N.; Bloem, P.; van den Berg, R.; Titov, I.; and Welling, M. 2018. Modeling Relational Data with Graph Convolutional Networks. In The Semantic Web, 593--607

    2018

  36. [44]

    Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; and Klimov, O. 2017. Proximal Policy Optimization Algorithms. arXiv:1707.06347

    2017 arXiv

  37. [45]

    Shalev-Shwartz, S.; Shammah, S.; and Shashua, A. 2016. Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving. arXiv:1610.03295

    2016 arXiv

  38. [46]

    Song, A.; Stojanovic, M.; and Chitre, M. 2019. Underwater Acoustic Communications: Where we Stand and What is Next? IEEE Journal of Oceanic Engineering, 44

    2019

  39. [47]

    F.; Mediano, P

    Tacchetti, A.; Song, H. F.; Mediano, P. A. M.; Zambaldi, V.; Kramár, J.; Rabinowitz, N. C.; Graepel, T.; Botvinick, M.; and Battaglia, P. W. 2019. Relational Forward Models for Multi-Agent Learning. In International Conference on Learning Representations

    2019

  40. [48]

    N.; Kaiser, L

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L. u.; and Polosukhin, I. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems

    2017

  41. [49]

    Veličković, P.; Cucurull, G.; Casanova, A.; Romero, A.; Liò, P.; and Bengio, Y. 2018. Graph Attention Networks. In International Conference on Learning Representations

    2018

  42. [50]

    Welling, M.; and Kipf, T. N. 2016. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations

    2016

  43. [51]

    Yu, C.; Velu, A.; Vinitsky, E.; Gao, J.; Wang, Y.; Bayen, A.; and WU, Y. 2022. The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games. In Advances in Neural Information Processing Systems, 24611--24624

    2022

  44. [52]

    Zambaldi, V.; Raposo, D.; Santoro, A.; Bapst, V.; Li, Y.; Babuschkin, I.; Tuyls, K.; Reichert, D.; Lillicrap, T.; Lockhart, E.; Shanahan, M.; Langston, V.; Pascanu, R.; Botvinick, M.; Vinyals, O.; and Battaglia, P. 2019. Deep reinforcement learning with relational inductive bi...

    2019

  45. [53]

    T.; Calandra, R.; Gal, Y.; and Levine, S

    Zhang, A.; McAllister, R. T.; Calandra, R.; Gal, Y.; and Levine, S. 2021 a . Learning Invariant Representations for Reinforcement Learning without Reconstruction. In International Conference on Learning Representations

    2021

  46. [54]

    Zhang, X.; Liu, Y.; Xu, X.; Huang, Q.; Mao, H.; and Carie, A. 2021 b . Structural relational inference actor-critic for multi-agent reinforcement learning. Neurocomputing, 459: 383--394

    2021

  47. [55]

    Zucker, J. 2003. A grounded theory of Abstraction in Artificial Intelligence. Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 358: 1293--309

    2003

  48. [56]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  49. [57]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.