Pith. sign in

REVIEW 5 major objections 7 minor 39 references

Policy Gradient with Self-Attention for Model-Free Distributed Nonlinear Multi-Agent Games

T0 review · 5 major / 7 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proposes learning distributed policies for model-free nonlinear multi-team games as state-dependent feedback gains parameterized by self-attention, trained from cost signals alone.

desk verdict A useful engineering paper on attention-based distributed policies for multi-agent games, but the theoretical framing oversells the nonlinear extension and the policy class has a zero-state restriction that the paper never confronts. read the letter →

arxiv 2509.18371 v2 pith:D7UULDNN submitted 2025-09-22 eess.SY cs.MAcs.ROcs.SY

Eduardo Sebastián , Maitrayee Keskar , Eeman Iqbal , Eduardo Montijano , Carlos Sagüés , Nikolay Atanasov This is my paper
classification eess.SYcs.MAcs.ROcs.SY
keywords multi-agentgamesdistributedpolicygradientself-attentionnonlinearfeedbackgainmodel-freecontrolpursuit-evasionmulti-robotsystems
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Model-free multi-team games — where agents observe only local states and receive a per-team cost signal, with no access to the dynamics or cost functions — are hard because equilibria shift as the game evolves. The paper proposes a distributed policy class in which each agent's action is a linear combination of its neighbors' states, with the combination coefficients produced by a self-attention network applied to those neighbor states at the current time step. This "nonlinear feedback gain" parameterization (Π(x,θ)=−K(x,θ)x) is designed so that execution respects the time-varying communication graph by construction, and the parameters are learned by policy gradient from the observed cost sequence. The authors show that, despite using no model, the learned policies closely match model-based optimal controllers in distributed LQR and nonlinear navigation, and that a policy trained in simulation for a two-team pursuit-evasion game transfers zero-shot to physical robots while preserving strategic behaviors such as deception and cornering. If the claim holds, it gives a practical recipe for scalable, communication-respecting, model-free multi-robot control in adversarial settings.

What carries the argument

The key machinery is the nonlinear feedback-gain policy Π(x,θ)=−K(x,θ)x with K built from self-attention layers. In Eq. (8), agent l of team i computes its gain blocks k_{l,p}_{i,j} by applying the same learned attention network to the concatenated states of its current neighbors; reshaping the final layer yields the coefficients that multiply each neighbor's state. Because attention weights are recomputed at every step from the current neighbor set, the policy respects the time-varying directed graph and is invariant to the number of neighbors; because only neighbor states enter, execution is fully distributed. The parameters θ are then tuned by policy gradient (with a multi-agent proximal-

What would settle it

Run the method on a small nonlinear two-team game (e.g., three agents per side) whose true optimal policies are computable by brute-force dynamic programming or iterative best response, and compare cumulative costs. If the learned feedback-gain policies still underperform the true optimum at any reasonable training budget, the −K(x,θ)x class is too restrictive.

Watch

Extended reading notes

Core claim

The paper's central claim is that near-optimal policies for nonlinear multi-team games with time-varying directed communication lie in the class Π(x,θ)=−K(x,θ)x, where K is a state-dependent gain built from self-attention layers and each agent's gain block depends only on the states of its current intra- and inter-team neighbors. This makes the policy distributed by construction, independent of the number of neighbors, and free of future-topology assumptions. Parameters are learned by policy gradient from cost signals alone. Empirically, the learned policies match the constrained optimum in distributed LQR, perform comparably to a model-based solver on nonlinear navigation, and beat MLP and

Load-bearing premise

The load-bearing premise is that near-optimal policies for the nonlinear multi-team games fall inside the class −K(x,θ)x with K depending only on each agent's current neighbor states; the paper offers empirical support but no optimality or approximation guarantee for this class, and it does not extend the linear-quadratic convergence guarantee to the nonlinear case.

Editorial extensions

If this is right

  • Effective distributed policies for nonlinear multi-team games can be learned without any model of dynamics or cost, using only transition samples and per-team cost signals.
  • The same fixed-size attention network handles arbitrary and time-varying neighbor sets, so the policy scales to teams of any size and adapts online to changing communication graphs.
  • In distributed linear-quadratic regulation, learning with an unknown proximity-based graph reaches the same cost as learning with the true graph, suggesting the method discovers which communication links matter.
  • Policies trained in simulation transfer zero-shot to physical robots without fine-tuning, preserving strategic behaviors like deception and cornering.
  • Because teams can have distinct parameters, the approach supports heterogeneous team objectives within one game.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An untested consequence of the attention parameterization is that it may implicitly learn to weight neighbors by task relevance; inspecting attention weights in the pursuit-evasion experiments could reveal whether the network learns an interpretable routing of information.
  • The policy class assumes linear-in-state structure at each step; for games requiring memory (e.g., where optimal play depends on past states), the state-dependent gain would need an augmented state or recurrence, a direction the paper does not explore.
  • The paper's own stated limitation—absence of benchmarks with more than two teams or with multi-agent teams of varying sizes—leaves scaling claims untested; a natural next experiment is a perimeter-defense game with three or more competing teams.
  • Because training relies on a bounded cost signal and local-dominance-style assumptions from the LQ guarantee, its sensitivity to sparse or discontinuous reward signals is unexplored; a stress test with sparse catches would show how much the policy gradient depends on smooth cost landscapes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper addresses model-free distributed policy learning for multi-team nonlinear dynamic games. The main proposal is a policy parameterization Π(x(k), θ) = −K(x(k), θ)x(k), where K is produced by a self-attention network over local neighbor states, so that each agent's action is a linear combination of neighbor states with attention-dependent gains (Eq. (8)). Parameters are learned by policy gradient / PPO using only cost samples and transitions. The paper reports experiments in (i) a distributed LQR problem with an optimal centralized baseline, (ii) a nonlinear multi-agent navigation game compared with distributed potential iLQR, (iii) a two-team pursuit-evasion game in BenchMARL compared with MLP and GNN policies, and (iv) a zero-shot transfer to the Robotarium. The paper claims strong performance and states that the structure extends LQ distributed policies to nonlinear games.

Significance. If the empirical claims held, this would be a practically useful contribution: a model-free, distributed, permutation-invariant policy class for nonlinear multi-agent games that handles time-varying neighbor sets and transfers to hardware. The attention-based gain structure is a sensible inductive bias, and enforcing the communication graph by construction is valuable. The LQ experiment is well motivated and the Robotarium deployment is a useful stress test. However, the paper does not provide an expressiveness or convergence guarantee for the nonlinear setting, and the evidence presented is not yet statistically robust. The paper would be strengthened by clearly delimiting the claim as a heuristic architecture with empirical support, adding a closed-loop model-based baseline, and reporting error bars, seeds, and significance tests.

major comments (5)
  1. [§III-B, Eq. (6) and (8)] The proposed policy class satisfies Π(0, θ)=0 for all θ, because the action is exactly a state-dependent gain multiplied by the local state vector. Any game in which the optimal action at the zero local-neighbor-state is nonzero—e.g., tracking a nonzero reference state as in the navigation cost of §IV-B, or an evader that must move even when all neighbors are co-located—cannot be represented, regardless of training effort or network width. Moreover, the claim in §III-B that this 'extends Theorem 1 to a nonlinear game' is not supported: Theorem 1 requires linear dynamics, quadratic costs, a known finite-horizon graph sequence, and a gradient-dominance condition, and no analogous expressiveness or convergence statement is supplied for Eq. (6). Please either (i) explicitly restrict the claim to a heuristic motivated by LQ theory and demonstrate empirically that the class is sufficient for t
  2. [§IV-B, Fig. 3] The comparison against DP-iLQR does not isolate the quality of the policy class. DP-iLQR is an open-loop trajectory optimizer, whereas the proposed policy is evaluated as a closed-loop controller. A model-based closed-loop baseline (e.g., receding-horizon iLQR or an LQR around the open-loop plan) is needed to separate the contribution of the policy parameterization from the advantage of feedback and from training difficulty. As it stands, the 'near-optimal performance' claim in §IV-B is not established against an appropriate nonlinear baseline.
  3. [§IV-A and §IV-B, Figs. 2–4] The principal quantitative comparisons are single learning curves with no error bars, seeds, or multiple runs. The LQR experiment uses random initial conditions, random dynamics parameters b_i, and random sparse graphs, so a single run cannot support the claims that the method 'quickly converges' and 'recovers the performance under network constraints.' Similarly, the navigation result in Fig. 3 reports one trajectory per method. Please report means and standard deviations over at least 5–10 seeds for both final costs and learning curves, and state the number of seeds used.
  4. [§IV-C, Tables I–III] The pursuit-evasion results show very large variances (e.g., 9.1±14.9 catches in the Ours/Ours cell of Table II, 10.3±21.1 in Ours/MLP) and overlapping intervals across methods. The conclusions that 'our method is the best competitor' and that training with this parameterization 'has reached an equilibrium with better performance' are not supported without statistical comparisons (e.g., paired tests or confidence intervals on the differences) and without reporting the number of training seeds. Table III also reports rewards without standard deviations, so the rank ordering may be noise.
  5. [§IV-D, Figs. 6–7] The real-robot section is qualitative. It reports no quantitative success criteria, no repeated episodes, no baselines, and no comparison with the simulated performance. Because the Robotarium applies safety filters that alter the closed-loop dynamics, the observed behavior may be substantially affected by the safety layer. The claim that this section 'assesses the effectiveness' of the learned policies in a real deployment is therefore overstated. Please either provide quantitative metrics (e.g., catches, minimum distances, episode duration over several runs) or clearly label this as a demonstration only.
minor comments (7)
  1. [Eq. (10)] The expression E[ρ_i^l(x_N^l(k), θ_i^l)] = π_i^l(ρ_i(...)) is not well-formed; the right-hand side appears to use ρ_i as the argument of π_i^l. It should be π_i^l(x_N^l(k), θ_i^l).
  2. [Theorem 1] The theorem statement is vague about what θ* denotes ('the learned distributed policy' vs. the global optimum) and about the probability space. Since it is adapted from [23], please state the precise result and conditions in this paper's notation, or quote it as a remark rather than a numbered theorem.
  3. [Eq. (2)–(3) and Algorithm 1] The problem is first written as a minimization over action sequences (Eq. (2)), then over policy parameters (Eq. (3)); the transition between the two, including the role of the discount factor γ, is never made explicit. Algorithm 1 says it computes ∂J_i^∞/∂θ_i^t from cost signals but does not specify the critic or advantage estimator used by PPO; a precise reference to the PPO update would help reproducibility.
  4. [Table II] The formula for cumulative catches appears to be missing the summation over time k; as written, the Iverson bracket over a min over j at a single unspecified k does not define a cumulative count.
  5. [§III-B, Eq. (9)] The output-dimension statement says X_W^i has U·X rows and |N^l| columns, but then says each column corresponds to a block-element k^{l,p}_{i,j}; please clarify the exact reshaping that maps columns to the block gains in Eq. (8), and how zero entries for non-neighbors are implemented.
  6. [§IV-A] The LQR cost is described as bounded, but a quadratic cost over an infinite horizon is not bounded in general; the finite horizon K=30 and bounded initial conditions mitigate this, but the statement should be qualified.
  7. [§V] The main-limitation paragraph mentions only the lack of multi-team benchmarks. Given the structural restriction of Eq. (6), the absence of expressiveness guarantees and the statistical variability of the results should also be acknowledged as limitations.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the nonlinear policy class is an explicitly motivated ansatz, not derived from its own outputs; Theorem 1 is external and non-load-bearing.

full rationale

The paper's central construction is Eq. (6), Π(x(k),θ) = -K(x(k),θ)x(k), with K parameterized by self-attention. This is presented as an ansatz 'inspired by the structure of distributed policies in linear quadratic games' and explicitly justified as a combination of the LQ forms in Eqs. (4)-(5), not derived from a theorem. The only theoretical result, Theorem 1, is adapted from the external reference [23] (Furieri et al.), is restricted to linear dynamics, quadratic costs, and known finite-horizon graph sequences, and is used only as motivation; the paper itself states that 'key challenges must be addressed' before extending to general games. No central claim reduces to a self-citation, and none of the authors' prior works are load-bearing. Training uses policy gradient on the cost signal and is evaluated against external baselines (centralized LQR, [23], DP-iLQR, MLP, GNN); no fitted parameter is renamed as a prediction. The main weakness is that the policy class in Eq. (6)/(8) is an unproven function class for nonlinear games and forces Π(0,θ)=0, but that is a correctness/expressiveness limitation, not circularity. The stated limitation in the conclusions concerns benchmarks, not a circular derivation.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical entities and relies mostly on standard MARL components. The key unexamined assumption is whether the attention feedback-gain class is expressive enough for the nonlinear games, and the experimental sections depend on several hand-chosen hyperparameters.

free parameters (4)
  • Communication radius / proximity threshold = 0.2 (LQR), 0.5 m (navigation), 1 m (pursuit-evasion)
    Determines the graph E(k) and hence the information available to each agent; chosen by hand per experiment, not learned.
  • Learning rate η = 1e-5 (LQR), 1e-3 (navigation); unspecified for MAPPO
    Hand-chosen; no sensitivity analysis.
  • Attention network architecture = hidden dims [64,64]; number of layers W unspecified
    Architecture size is a manual choice; the paper does not report W or activation details β, χ, ψ fully.
  • Discount factor γ = not reported
    Required in the infinite-horizon cost (2); never reported for the experiments.
assumptions (4)
  • domain assumption Stage costs c_i are bounded
    Stated in Sec. II to guarantee finite policy gradients.
  • standard math LQ optimal policies are linear feedback gains (discrete Riccati)
    Standard result cited [31], used as motivation for Eq. (6).
  • domain assumption Theorem 1 of Furieri et al. [23] holds with its assumptions (µ-gradient dominance, known graph sequence, linear dynamics)
    Reproduced as Theorem 1 and used to motivate learning; not proven in this paper.
  • ad hoc to paper The function class -K(x,θ)x with self-attention K can represent near-optimal nonlinear distributed policies
    This is the paper's core architectural heuristic; no approximation bound or universality argument is given (Sec. III-B).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Policy Gradient with Self-Attention for Model-Free Distributed Nonlinear Multi-Agent Games." pith.science (2026). https://pith.science/paper/D7UULDNN

@misc{pith2026250918371,
  author       = {Pith},
  title        = {Pith review of: Policy Gradient with Self-Attention for Model-Free Distributed Nonlinear Multi-Agent Games},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D7UULDNN}},
  note         = {Machine review of arXiv:2509.18371}
}
read the original abstract

Multi-agent games in dynamic nonlinear settings are challenging due to the time-varying interactions among the agents and the non-stationarity of the (potential) Nash equilibria. In this paper we consider model-free games, where agent transitions and costs are observed without knowledge of the transition and cost functions that generate them. We propose a novel distributed policy structure that follows the communication constraints in multi-team games, with multiple agents per team, and learned through policy gradients. Our formulation is inspired by the structure of distributed policies in linear quadratic games, which take the form of time-varying linear feedback gains. In the nonlinear case, we model the policies as nonlinear feedback gains, parameterized by self-attention layers to account for the time-varying multi-agent communication topology. We demonstrate that our approach achieves strong performance in several settings, including distributed linear and nonlinear regulation, and simulated and real multi-robot pursuit-and-evasion games.

Figures

Figures reproduced from arXiv: 2509.18371 by the authors.

Figure 1
Figure 1. Multi-team perimeter-defense game. Each team has a different goal, number of agents, and constraints (e.g., the scouter team inspects the region to detect attackers whereas the deceiver team aims at confusing the scouters), leading to global cooperative and competitive behaviors (e.g., patrollers and defenders help each other to prevent attackers from crossing the perimeter). The agents only have access to local inf… view at source ↗
Figure 2
Figure 2. Comparison of our method with a centralized LQR baseline, a model-free zeroth-order optimal LQR under subspace constraints [23], and our method with known graph constraints. (LQR) for finite-horizon games, which is optimal under the assumption that the network topology is known. The second is the method proposed in [23], which learns the optimal linear quadratic regulator under known subspace (network) constraints, … view at source ↗
Figure 4
Figure 4. Qualitative result from the comparison between DP-iLQR at iteration 10 (a) and our method at iteration 100 (b). The agents are shown as grey circles, the goals as red crosses, and the agent trajectories as colored curves. is a 2×2 m2 square where agents move following holonomic dynamics. The arena contains two circular obstacles of radius 0.2 m. We consider N = 2 teams with Mi = 3 agents each. The agents observe the… view at source ↗
Figures from the paper (2 more)
Figure 6
Figure 6. Figure 6: presents snapshots of the Robotarium simulator, which is designed to be as close to the real setting as possible to ensure that the policies used in the real deployment comply with all safety requirements. Despite the gap in realism between the gym-based environment us…
Figure 7
Figure 7. Figure 7: Real Robotarium deployment. Initially, the evaders (green) can deceive the pursuers (red) and escape to the top-left corner of the arena. However, the pursuers react and try to trap the evaders. The evaders are able to sneak out through the topmost border and escape to…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

39 extracted references · 2 linked inside Pith

  1. [1]

    A real-time game theoretic planner for autonomous two-player drone racing,

    R. Spica, E. Cristofalo, Z. Wang, E. Montijano, and M. Schwager, “A real-time game theoretic planner for autonomous two-player drone racing,”IEEE Transactions on Robotics, vol. 36, no. 5, pp. 1389–1403, 2020

    2020

  2. [2]

    Multi-robot task allocation games in dynamically changing environments,

    S. Park, Y . D. Zhong, and N. E. Leonard, “Multi-robot task allocation games in dynamically changing environments,” inIEEE International Conference on Robotics and Automation, 2021, pp. 8678–8684

    2021

  3. [3]

    Adaptive multirobot im- plicit control of heterogeneous herds,

    E. Sebastián, E. Montijano, and C. Sagüés, “Adaptive multirobot im- plicit control of heterogeneous herds,”IEEE Transactions on Robotics, vol. 38, no. 6, pp. 3622–3635, 2022

    2022

  4. [4]

    Physics-informed multi-agent reinforcement learning for distributed multi-robot problems,

    E. Sebastián, T. Duong, N. Atanasov, E. Montijano, and C. Sagüés, “Physics-informed multi-agent reinforcement learning for distributed multi-robot problems,”IEEE Transactions on Robotics, 2025

    2025

  5. [5]

    A review of multi agent perimeter defense games,

    D. Shishika and V . Kumar, “A review of multi agent perimeter defense games,” inInternational Conference on Decision and Game Theory for Security, 2020, pp. 472–485

    2020

  6. [6]

    Accelerated K-serial stable coalition for dynamic capture and resource defense,

    J. Chen, Z. Tang, and M. Guo, “Accelerated K-serial stable coalition for dynamic capture and resource defense,”IEEE Robotics and Au- tomation Letters, vol. 9, no. 1, pp. 443–450, 2023

    2023

  7. [7]

    Ba¸ sar and G

    T. Ba¸ sar and G. J. Olsder,Dynamic noncooperative game theory. SIAM, 1998

    1998

  8. [8]

    Nash equilibrium seeking for N- coalition noncooperative games,

    M. Ye, G. Hu, and F. L. Lewis, “Nash equilibrium seeking for N- coalition noncooperative games,”Automatica, vol. 95, pp. 266–272, 2018

    2018

Show all 39 references
  1. [9]

    Generalized Nash equilib- rium seeking strategy for distributed nonsmooth multi-cluster game,

    X. Zeng, J. Chen, S. Liang, and Y . Hong, “Generalized Nash equilib- rium seeking strategy for distributed nonsmooth multi-cluster game,” Automatica, vol. 103, pp. 20–26, 2019

    2019

  2. [10]

    Analysis and interventions in large network games,

    F. Parise and A. Ozdaglar, “Analysis and interventions in large network games,”Annual Review of Control, Robotics, and Autonomous Systems, vol. 4, no. 1, pp. 455–486, 2021

    2021

  3. [11]

    Nash-minmax stra- tegy for multiplayer multiagent graphical games with reinforcement learning,

    B. Lian, W. Xue, F. L. Lewis, and A. Davoudi, “Nash-minmax stra- tegy for multiplayer multiagent graphical games with reinforcement learning,”IEEE Transactions on Control of Network Systems, 2024

    2024

  4. [12]

    Synthesis and stabilization of com- plex behaviors through online trajectory optimization,

    Y . Tassa, T. Erez, and E. Todorov, “Synthesis and stabilization of com- plex behaviors through online trajectory optimization,” inIEEE/RSJ International Conference on Intelligent Robots and Systems, 2012, pp. 4906–4913

    2012

  5. [13]

    Efficient iterative linear-quadratic approximations for nonlinear multi-player general-sum differential games,

    D. Fridovich-Keil, E. Ratner, L. Peters, A. D. Dragan, and C. J. Tom- lin, “Efficient iterative linear-quadratic approximations for nonlinear multi-player general-sum differential games,” inIEEE International Conference on Robotics and Automation, 2020, pp. 1475–1481

    2020

  6. [14]

    ALGAMES: A fast solver for constrained dynamic games,

    L. Cleac’h, M. Schwager, Z. Manchester,et al., “ALGAMES: A fast solver for constrained dynamic games,” inRobotics: Science and Systems, 2020

    2020

  7. [15]

    Escaping high-order saddles in policy optimization for linear quadratic gaussian (LQG) control,

    Y . Zheng, Y . Sun, M. Fazel, and N. Li, “Escaping high-order saddles in policy optimization for linear quadratic gaussian (LQG) control,” inIEEE Conference on Decision and Control, 2022, pp. 5329–5334

    2022

  8. [16]

    Potential games,

    D. Monderer and L. S. Shapley, “Potential games,”Games and economic behavior, vol. 14, no. 1, pp. 124–143, 1996

    1996

  9. [17]

    Potential games on cubic splines for self-interested multi-agent motion planning,

    S. Williams and J. Deshmukh, “Potential games on cubic splines for self-interested multi-agent motion planning,”IEEE Control Systems Letters, 2024

    2024

  10. [18]

    Strategic decision-making in multi- agent domains: A weighted constrained potential dynamic game approach,

    M. Bhatt, Y . Jia, and N. Mehr, “Strategic decision-making in multi- agent domains: A weighted constrained potential dynamic game approach,”IEEE Transactions on Robotics, 2025

    2025

  11. [19]

    Potential iLQR: A potential- minimizing controller for planning multi-agent interactive trajectories,

    T. Kavuncu, A. Yaraneri, and N. Mehr, “Potential iLQR: A potential- minimizing controller for planning multi-agent interactive trajectories,” inRobotics: Science and Systems, 2021

    2021

  12. [20]

    Distributed potential iLQR: Scal- able game-theoretic trajectory planning for multi-agent interactions,

    Z. Williams, J. Chen, and N. Mehr, “Distributed potential iLQR: Scal- able game-theoretic trajectory planning for multi-agent interactions,” inIEEE International Conference on Robotics and Automation, 2023

    2023

  13. [21]

    A variational inequality framework for network games: Existence, uniqueness, convergence and sensitivity analysis,

    F. Parise and A. Ozdaglar, “A variational inequality framework for network games: Existence, uniqueness, convergence and sensitivity analysis,”Games and Economic Behavior, vol. 114, pp. 47–82, 2019

    2019

  14. [22]

    Equilibrium selection for multi- agent reinforcement learning: A unified framework,

    R. Zhang, J. Shamma, and N. Li, “Equilibrium selection for multi- agent reinforcement learning: A unified framework,”arXiv preprint arXiv:2406.08844, 2024

    2024

  15. [23]

    Learning the globally optimal distributed LQ regulator,

    L. Furieri, Y . Zheng, and M. Kamgarpour, “Learning the globally optimal distributed LQ regulator,” inLearning for Dynamics and Control. PMLR, 2020, pp. 287–297

    2020

  16. [24]

    Constrained multi-cluster game: Distributed nash equilibrium seeking over directed graphs,

    D. T. A. Nguyen, M. Bianchi, F. Dörfler, D. T. Nguyen, and A. Nedi ´c, “Constrained multi-cluster game: Distributed nash equilibrium seeking over directed graphs,” inIEEE Conference on Decision and Control, 2024, pp. 4712–4719

    2024

  17. [25]

    A unifying system theory framework for distributed optimization and games,

    G. Carnevale, N. Mimmo, and G. Notarstefano, “A unifying system theory framework for distributed optimization and games,”IEEE Transactions on Automatic Control, 2025

    2025

  18. [26]

    An overview of multi-agent reinforce- ment learning from game theoretical perspective,

    Y . Yang and J. Wang, “An overview of multi-agent reinforce- ment learning from game theoretical perspective,”arXiv preprint arXiv:2011.00583, 2020

    2011 arXiv

  19. [27]

    Variational policy gradient method for reinforcement learning with general utilities,

    J. Zhang, A. Koppel, A. S. Bedi, C. Szepesvari, and M. Wang, “Variational policy gradient method for reinforcement learning with general utilities,”Advances in Neural Information Processing Systems, vol. 33, pp. 4572–4583, 2020

    2020

  20. [28]

    Independent policy gradient methods for competitive reinforcement learning,

    C. Daskalakis, D. J. Foster, and N. Golowich, “Independent policy gradient methods for competitive reinforcement learning,”Advances in Neural Information Processing Systems, vol. 33, pp. 5527–5540, 2020

    2020

  21. [29]

    On the convergence of policy gradient methods to Nash equilibria in general stochastic games,

    A. Giannou, K. Lotidis, P. Mertikopoulos, and E.-V . Vlatakis- Gkaragkounis, “On the convergence of policy gradient methods to Nash equilibria in general stochastic games,”Advances in Neural Information Processing Systems, vol. 35, pp. 7128–7141, 2022

    2022

  22. [30]

    Policy gradient play with networked agents in markov potential games,

    S. Aydin and C. Eksin, “Policy gradient play with networked agents in markov potential games,” inLearning for Dynamics and Control Conference. PMLR, 2023, pp. 184–195

    2023

  23. [31]

    The discrete Riccati equation of optimal control,

    V . Kuˇcera, “The discrete Riccati equation of optimal control,”Kyber- netika, vol. 8, no. 5, pp. 430–447, 1972

    1972

  24. [32]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in Neural Information Processing Systems, vol. 30, 2017

    2017

  25. [33]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  26. [34]

    Multi-agent actor-critic for mixed cooperative-competitive environ- ments,

    R. Lowe, Y . I. Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch, “Multi-agent actor-critic for mixed cooperative-competitive environ- ments,”Advances in Neural Information Processing Systems, 2017

    2017

  27. [35]

    VMAS: A vectorized multi-agent simulator for collective robot learning,

    M. Bettini, R. Kortvelesy, J. Blumenkamp, and A. Prorok, “VMAS: A vectorized multi-agent simulator for collective robot learning,” International Symposium on Distributed Autonomous Robotic Systems, 2022

    2022

  28. [36]

    BenchMARL: Benchmarking multi-agent reinforcement learning,

    M. Bettini, A. Prorok, and V . Moens, “BenchMARL: Benchmarking multi-agent reinforcement learning,”Journal of Machine Learning Research, vol. 25, no. 217, pp. 1–10, 2024

    2024

  29. [37]

    Graph attention networks,

    P. Veli ˇckovi´c, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y . Bengio, “Graph attention networks,” inInternational Conference on Learning Representations, 2018

    2018

  30. [38]

    The surprising effectiveness of PPO in cooperative multi-agent games,

    C. Yu, A. Velu, E. Vinitsky, J. Gao, Y . Wang, A. Bayen, and Y . Wu, “The surprising effectiveness of PPO in cooperative multi-agent games,”Advances in neural information processing systems, vol. 35, pp. 24 611–24 624, 2022

    2022

  31. [39]

    The Robotarium: A remotely accessible swarm robotics research testbed,

    D. Pickem, P. Glotfelter, L. Wang, M. Mote, A. Ames, E. Feron, and M. Egerstedt, “The Robotarium: A remotely accessible swarm robotics research testbed,” inIEEE International Conference on Robotics and Automation, 2017, pp. 1699–1706

    2017

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.