Pith. sign in

REVIEW 3 major objections 3 minor 43 references

Independent Learning in Performative Markov Potential Games

T0 review · 3 major / 3 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read In performative Markov potential games, where deployed policies change the rewards and transitions, independent policy gradient and natural policy gradient learners converge to approximate performatively stable equilibria; natural…

desk verdict The performative-MPG framework is new and worth studying, but the main convergence theorems are built on a false identity and need major repair. read the letter →

arxiv 2504.20593 v1 pith:CEQ5VLIU submitted 2025-04-29 cs.LG

Rilind Sahitaj , Paulius Sasnauskas , Yiğit Yal{i}n , Debmalya Mandal , Goran Radanović This is my paper · ORCID
classification cs.LG
keywords performativereinforcementlearningMarkovpotentialgamesmulti-agentperformativelystableequilibriumnaturalpolicygradientregretindependent
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether independent gradient-based learners still converge when the policies they deploy change the game itself: rewards and transitions shift with the joint policy. It defines a multi-agent solution concept, the performatively stable equilibrium, and proves one always exists under a Lipschitz sensitivity assumption. The paper claims that independent policy gradient ascent and independent natural policy gradient both converge to an approximate performatively stable equilibrium in the best-iterate sense, with natural policy gradient additionally achieving asymptotic last-iterate convergence. For a special class with agent-independent transitions, a repeated occupancy-measure optimization method attains finite-time last-iterate convergence. If these results hold, natural-gradient multi-agent learning remains dependable even when agents reshape their environment, with the degradation controlled by the strength of the performative effects.

What carries the argument

The central object is the performative Markov potential game $G(\bar{\pi})$, whose potential $\Phi^{\pi}_{\bar{\pi}}$ depends on both the played policy $\pi$ and the induced game $\bar{\pi}$, together with the performatively stable equilibrium defined through that game. The argument is carried by sensitivity-based improvement lemmas: Lemma 2 bounds value changes across different induced games by $\delta_{r,p}\|\pi'-\pi''\|_2$, and this bound is inserted into the potential-improvement inequalities inherited from prior Markov potential game analyses. The natural-gradient analysis also depends on the log-barrier regularizer used in regularized INPG and, for the last-iterate result, on a contraction-mapping argument over state-action occupancy measures under the agent-independent-transitions assumption.

What would settle it

Along one INPG iteration in a two-agent, two-state performative Markov potential game with nonzero sensitivity, compute both sides of $V^{t+1}_{i,t}(\rho)-V^{t}_{i,t}(\rho)=\Phi^{t+1}_t(\rho)-\Phi^{t}_t(\rho)$; the Theorem 4 proof asserts these are equal, so showing they differ in a valid performative MPG would falsify the bound as stated.

Watch

Extended reading notes

Core claim

The paper introduces performative Markov potential games, in which the deployed joint policy determines the reward function and transition kernel of the Markov game, and generalizes solution concepts to this setting. Its central claim is that a performatively stable equilibrium always exists under the sensitivity assumption, that every such equilibrium is a $\delta_{r,p}$-approximate Nash equilibrium, and that independent gradient algorithms converge to approximate performatively stable equilibria in the best-iterate sense. For independent natural policy gradient, the paper proves performative regret of order $O\!\left(\sqrt{\tilde{\kappa}_\rho(\sqrt{n}+M\delta_{r,p})/(c(1-\gamma)^3 T)}\right)$; with a log-barrier regularizer it proves a finite-time best-iterate bound independent of the optimal-action probability $c$. It further shows asymptotic last-iterate convergence of unregularized natural policy gradient and finite-time last-iterate convergence for repeated independent optimization over occupancy measures when transitions are agent-independent. As the performative effects vanish, the rates recover prior Markov potential game guarantees.

Load-bearing premise

The load-bearing premise is that after a joint policy update, each agent's value change equals the change in the whole game's potential; the definition of a Markov potential game guarantees this only for unilateral deviations, not for simultaneous joint updates.

Editorial extensions

If this is right

  • When the sensitivity parameters $\omega_r,\omega_p$ tend to zero, the guaranteed rates reduce to the known Markov potential game rates for IPGA, INPG, and regularized INPG, so the performative analysis degrades gracefully.
  • Every performatively stable equilibrium is a $\delta_{r,p}$-approximate Nash equilibrium, meaning small performative sensitivity makes stability and Nash play coincide up to a controllable gap.
  • IPGA's performative regret carries an additive term that does not vanish as $T\to\infty$, whereas unregularized INPG's performative contribution disappears asymptotically, so the two algorithms differ qualitatively under strong performativity.
  • For agent-independent transitions, repeated independent optimization over occupancy measures converges in finite time to an approximate performatively stable equilibrium in the last-iterate sense.
  • The averaged performative regret guarantees that at least one iterate is an approximate performatively stable equilibrium, not that the final policy is one, for the policy-gradient methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: the equality $V^{t+1}_{i,t}(\rho)-V^t_{i,t}(\rho)=\Phi^{t+1}_t(\rho)-\Phi^t_t(\rho)$ used in Lemma 8 is not implied by the definition of a Markov potential game; if it cannot be derived for a restricted class of potentials, Theorem 4's guarantee would hold only for that class.
  • Editorial: a direct test is whether the same natural-policy-gradient rates hold with sample-based gradient estimates; the paper provides a sample-based guarantee only for IPGA, leaving a concrete open extension.
  • Editorial: because $\mathrm{PReg}(T)$ averages over rounds, practical deployment of IPGA should track the best visited policy rather than relying on the final iterate; this guidance is implicit in the best-iterate statement.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper introduces performative effects into Markov potential games (MPGs), defining a performatively stable equilibrium (PSE) and a performative regret measure. It claims that a PSE always exists under a Lipschitz sensitivity assumption, that independent policy gradient ascent (IPGA) and independent natural policy gradient (INPG) converge to an approximate PSE in the best-iterate sense, that unregularized INPG converges to a PSE in the last-iterate sense, and that a repeated retraining method achieves finite-time last-iterate convergence under agent-independent transitions. The proofs adapt prior MPG analyses by adding terms that account for the dependence of the game on the deployed policy. Supporting experiments on safe-distancing and stochastic congestion games are included with released code.

Significance. If the convergence theorems were correct, the paper would be a useful extension of performative RL to multi-agent settings: the PSE notion is natural, the performative regret measure is appropriate, the recovery of prior rates as the sensitivity parameters vanish is appealing, and the experiments together with the released code would support the empirical claims. However, the main convergence results are built on an invalid identity in Lemma 8 and on an unjustified cross-game potential bound in Lemma 5, so the central theoretical contribution is not established by the submitted proofs.

major comments (3)
  1. [Appendix A.6.1, Lemma 8; used in Theorems 4, 5, and the last-iterate claim] The first identity in the proof of Lemma 8, V^{t+1}_{i,t+1}(rho)-V^t_{i,t}(rho)=Phi^{t+1}_t(rho)-Phi^t_t(rho), asserts that a single agent's value change under a simultaneous joint policy update equals the potential change of that joint update. Equation (2) only guarantees this equality for unilateral deviations of one agent with all other agents held fixed. For a joint update, the potential change is a telescoping sum of value changes at intermediate unilateral profiles, not the value change of any one agent. A concrete counterexample is a one-state, two-agent game with gamma=0, r_1(a,b)=2a+b, r_2(a,b)=a+3b, and Phi(a,b)=2a+3b, which satisfies the MPG condition; moving from (0,0) to (1,1) gives V_1(1,1)-V_1(0,0)=3 but Phi(1,1)-Phi(0,0)=5. Thus Lemma 8 is false, and the proofs of Theorems 4 and 5, as well as the asymptotic last-iterate convergence statement, do not follow from the stated assumptions.
  2. [Appendix A.4, Lemma 5; also used in Theorem 6 via Lemma 6] The proof of Lemma 5 bounds |Phi^{t+1}_{t+1}-Phi^{t+1}_t| by invoking Lemma 2. Lemma 2 bounds |V^pi_{i,pi'}-V^pi_{i,pi''}|, i.e., a value-function difference between two underlying games, not a potential-function difference. No argument is given that the potentials for different underlying games can be chosen so that cross-game potential differences are controlled by delta_{r,p} ||pi^{t+1}-pi^t||. Since this cross-game potential bound produces the additive W_{r,p} term in Theorem 3, and the same argument is used in the oracle-free analysis leading to Theorem 6, these results also lack support.
  3. [Appendix A.1, Lemma 1 and Theorem 1] The proof of Lemma 1 asserts that Phi^{pi'}_pi is continuous in the second argument pi because of the (omega_r, omega_p)-sensitivity assumption. The definition in Eq. (2) only requires existence of a potential for each fixed game G(pi); it does not in itself guarantee that a selection of potentials can be made jointly continuous in pi. The Kakutani fixed-point argument for Theorem 1 requires this continuity, so the existence proof of a PSE is incomplete unless a canonical continuous construction of the potential is supplied.
minor comments (3)
  1. [Table 1 and Section 5.1.1] The definition of W_{r,p} is inconsistent: Theorem 3 in the main text defines W_{r,p} := T*(n+1)*n^2*S*delta_{r,p}, while the proof in Appendix A.4 defines W_{r,p} = n(n+1)/2 * delta_{r,p} * ||pi^{t+1}-pi^t||_2, and the errata appendix says W_{r,p} does not depend on T. These should be reconciled in a revision.
  2. [Assumption 5(iii)] The displayed inequality in Assumption 5(iii) reads ||nabla_pi Phi^pi_{pi'}(rho) - nabla_pi Phi^pi_{pi'}(rho)||_2 <= beta ||pi-pi'||_2, which has the same expression on both sides and is therefore either a typo or vacuous; presumably one of the gradients should be evaluated at a different policy or game.
  3. [Throughout] The manuscript contains several typos and inconsistencies, including the use of 'Perform-Regret' in Theorem 5 instead of PReg, the notation for the horizon H in Section 5.2, and the appendix errata items for Table 1 and Theorem 3. A careful proofreading pass is needed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: convergence bounds are derived from explicit sensitivity assumptions and potential-function identities, with self-citations used only as proof-technique sources.

full rationale

The paper's central claims are not produced by fitting data and renaming it as prediction, nor by importing an unverified uniqueness theorem from the authors' prior work. The performative regret bounds in Theorems 3-6 are derived from Assumptions 1-2, Lemma 2 (which bounds value changes under the sensitivity assumption), and standard fixed-game potential improvement lemmas from externally published work (Ding et al., Zhang et al., Alatur et al.). The performative correction terms appear as explicit additive contributions proportional to the sensitivity parameters omega_r, omega_p and the distribution-mismatch constants, rather than as quantities presupposed by the equilibrium definition. The existence of a PSE is established by a Kakutani fixed-point argument over the potential function, not by assuming the conclusion. Self-citations to Mandal et al. (2023) and Mandal and Radanovic (2025) occur in the related-work discussion and in Section 6 as a proof-template and technical lemma source; the repeated-occupancy-measure analysis is substantially adapted to the multi-agent, agent-independent-transition setting and does not reduce the new theorem to the cited single-agent result. A separate soundness concern exists around the equality between a single agent's joint-update value change and the potential change in Lemma 8, but that is a correctness issue, not a circularity: the identity is asserted as part of the proof, not obtained by redefining the potential or the equilibrium concept in terms of the target result. Overall, the derivation chain is self-contained relative to its stated assumptions, and no load-bearing circular step is exhibited.

Assumptions & free parameters 2 free parameters · 7 assumptions · 0 invented entities

The theory introduces no fitted constants; the results are expressed in terms of assumed sensitivity parameters and standard RL constants. The repeated optimization section relies on an occupancy-measure formulation without new physical entities. The main additional assumptions are the sensitivity conditions and the problematic implicit assumptions about the potential function's behavior across games.

free parameters (2)
  • omega (performative sensitivity) = 0.01, 0.03, 0.05, 0.10 in stochastic congestion game
    Simulation parameter controlling how strongly the policy changes rewards and transitions; the theory treats it as an assumed constant.
  • alpha (intervention probability) = 0.01, 0.05, 0.10, 0.15 in safe-distancing game
    Simulation parameter controlling the probability that the influencer agent overrides the principal agent's action; the theory does not depend on this value.
assumptions (7)
  • domain assumption Assumption 1: rewards and transition probabilities are Lipschitz in the policy with constants omega_r and omega_p.
    Standard sensitivity condition in performative learning, stated in Section 3.2 and used throughout the proofs.
  • domain assumption Assumption 2: the discounted state visitation distribution is bounded below for all policies and games.
    Standard exploration assumption needed to define M_rho and to use KL-based arguments in the INPG analysis.
  • domain assumption Assumption 3: the stationary points of the INPG update are isolated.
    Used to turn gradient-norm convergence into last-iterate convergence for unregularized INPG.
  • domain assumption Assumption 4: transition probabilities are independent of the agents' actions in the special case.
    Restricts the repeated occupancy-measure optimization result to games with agent-independent transitions.
  • domain assumption Assumption 5: rewards and transitions are Lipschitz in the occupancy measure, and the potential gradient is smooth.
    Used in the contraction argument for the repeated optimization method in Section 6.
  • ad hoc to paper Implicit: the family of potential functions Phi_{pi_bar} can be chosen so that it is continuous in pi_bar and so that cross-game potential differences are bounded by value-function sensitivity.
    The paper never defines a normalization or continuity construction for the potentials across different games, yet Lemma 1 and Lemma 5 require such a property.
  • ad hoc to paper Implicit: for a joint policy update, the change in an individual agent's value equals the change in the potential function.
    Used in the proof of Lemma 8, but not a consequence of the Markov potential game definition; this is a false premise for general MPGs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Independent Learning in Performative Markov Potential Games." pith.science (2026). https://pith.science/paper/CEQ5VLIU

@misc{pith2026250420593,
  author       = {Pith},
  title        = {Pith review of: Independent Learning in Performative Markov Potential Games},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CEQ5VLIU}},
  note         = {Machine review of arXiv:2504.20593}
}
read the original abstract

Performative Reinforcement Learning (PRL) refers to a scenario in which the deployed policy changes the reward and transition dynamics of the underlying environment. In this work, we study multi-agent PRL by incorporating performative effects into Markov Potential Games (MPGs). We introduce the notion of a performatively stable equilibrium (PSE) and show that it always exists under a reasonable sensitivity assumption. We then provide convergence results for state-of-the-art algorithms used to solve MPGs. Specifically, we show that independent policy gradient ascent (IPGA) and independent natural policy gradient (INPG) converge to an approximate PSE in the best-iterate sense, with an additional term that accounts for the performative effects. Furthermore, we show that INPG asymptotically converges to a PSE in the last-iterate sense. As the performative effects vanish, we recover the convergence rates from prior work. For a special case of our game, we provide finite-time last-iterate convergence results for a repeated retraining approach, in which agents independently optimize a surrogate objective. We conduct extensive experiments to validate our theoretical findings.

Figures

Figures reproduced from arXiv: 2504.20593 by the authors.

Figure 2
Figure 2. Comparison of IPGA-D, INPG (unreg.), INPG (reg.), showing the distance from the current policy to the average of the last 10 in that run: 1 N PN i [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Comparison of IPGA-L and IPGA-D in the safe-distancing game varying the performativity strength [PITH_FULL_IMAGE:figures/full_fig_p035_3.png] view at source ↗
Figure 4
Figure 4. Comparison of IPGA-L and IPGA-D in the stochastic congestion game varying the performativity [PITH_FULL_IMAGE:figures/full_fig_p036_4.png] view at source ↗
Figures from the paper (5 more)
Figure 5
Figure 5. Figure 5: Comparison of INPG regularized vs. unregularized version in the safe-distancing game varying the [PITH_FULL_IMAGE:figures/full_fig_p037_5.png]
Figure 6
Figure 6. Figure 6: Comparison of INPG regularized vs. unregularized version in the stochastic congestion game varying [PITH_FULL_IMAGE:figures/full_fig_p037_6.png]
Figure 7
Figure 7. Figure 7: Comparison of different values of α (linear scale) and η (log scale) for IPGA-L and IPGA-D in the safe-distancing environment. Circles indicate the runs converged in under 30000 rounds with the number of rounds indicated by the color scale shown on the right. Black squ…
Figure 8
Figure 8. Figure 8: An illustration of the Safe-Distancing environment that is used in the experiments. [PITH_FULL_IMAGE:figures/full_fig_p039_8.png]
Figure 9
Figure 9. Figure 9: An illustration of the stochastic congestion game. From states [PITH_FULL_IMAGE:figures/full_fig_p040_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 41 canonical work pages

  1. [1]

    Alatur, P., Barakat, A., and He, N. (2024a). Independent policy mirror descent for markov potential games: Scaling to large number of players. In 2024 IEEE 63rd Conference on Decision and Control (CDC)

    work page 2024

  2. [2]

    Alatur, P., Ramponi, G., He, N., and Krause, A. (2024b). Provably learning N ash policies in constrained M arkov potential games. In AAMAS 2024

    work page 2024

  3. [3]

    Anagnostides, I., Panageas, I., Farina, G., and Sandholm, T. (2023). On the convergence of no-regret learning dynamics in time-varying games. In NeurIPS 2023

    work page 2023

  4. [4]

    and Catoni, O

    Audibert, J. and Catoni, O. (2011). Robust linear least squares regression. Annals of Statistics , 39(5)

    work page 2011

  5. [5]

    Brown, G., Hod, S., and Kalemaj, I. (2022). Performative prediction in a stateful world. In AISTATS 2022

    work page 2022

  6. [6]

    and Sandholm, T

    Brown, N. and Sandholm, T. (2019). Superhuman AI for multiplayer poker. Science , 365(6456)

    work page 2019

  7. [7]

    Cai, S., Han, F., and Cao, X. (2024). Performative control for linear dynamical systems. In NeurIPS 2024

    work page 2024

  8. [8]

    R., Abernethy, J

    Cardoso, A. R., Abernethy, J. D., Wang, H., and Xu, H. (2019). Competing against nash equilibria in adversarially changing zero-sum games. In ICML 2019

    work page 2019

Show all 43 references
  1. [9]

    Ding, D., Wei, C., Zhang, K., and Jovanovic, M. R. (2022). Independent policy gradient for large-scale M arkov potential games: Sharper rates, function approximation, and game-agnostic convergence. In ICML 2022

    2022

  2. [10]

    Duvocelle, B., Mertikopoulos, P., Staudigl, M., and Vermeulen, D. (2023). Multiagent online learning in time-varying games. Mathematics of Operations Research , 48(2)

    2023

  3. [11]

    M., Overman, W., and Panageas, I

    Fox, R., McAleer, S. M., Overman, W., and Panageas, I. (2022). Independent natural policy gradient always converges in M arkov potential games. In AISTATS 2022

    2022

  4. [12]

    Glicksberg, I. L. (1952). A further generalization of the K akutani fixed point theorem, with application to N ash equilibrium points. Proceedings of the American Mathematical Society , 3(1)

    1952

  5. [13]

    Guo, X., Li, X., Maheshwari, C., Sastry, S., and Wu, M. (2024). Markov -potential games. CoRR , abs/2305.12553

    2024 arXiv

  6. [14]

    Hardt, M., Jagadeesan, M., and Mendler - D \" u nner, C. (2022). Performative power. In NeurIPS 2022

    2022

  7. [15]

    and Mendler-Dünner, C

    Hardt, M. and Mendler-Dünner, C. (2023). Performative prediction: Past and future. CoRR , abs/2310.16608

    2023 arXiv

  8. [16]

    Izzo, Z., Zou, J., and Ying, L. (2022). How to learn when data gradually reacts to your model. In AISTATS 2022

    2022

  9. [17]

    Jagadeesan, M., Zrnic, T., and Mendler - D \" u nner, C. (2022). Regret minimization with performative feedback. In ICML 2022

    2022

  10. [18]

    Jiang, H., Cui, Q., Xiong, Z., Fazel, M., and Du, S. S. (2024). A black-box approach for non-stationary multi-agent reinforcement learning. In ICLR 2024

    2024

  11. [19]

    Jordan, P., Barakat, A., and He, N. (2024). Independent learning in constrained M arkov potential games. In AISTATS 2024

    2024

  12. [20]

    Kulynych, B. (2022). Causal prediction can induce performative stability. In ICML 2022: Workshop on Spurious Correlations, Invariance and Stability

    2022

  13. [21]

    Leonardos, S., Overman, W., Panageas, I., and Piliouras, G. (2022). Global convergence of multi-agent policy gradient in M arkov potential games. In ICLR 2022

    2022

  14. [22]

    and Wai, H

    Li, Q. and Wai, H. (2022). State dependent performative prediction with stochastic approximation. In AISTATS 2022

    2022

  15. [23]

    Li, Q., Yau, C.-Y., and Wai, H.-T. (2022). Multi-agent performative prediction with greedy deployment and consensus seeking agents. In NeurIPS 2022

    2022

  16. [24]

    Maheshwari, C., Wu, M., Pai, D., and Sastry, S. (2024). Independent and decentralized learning in markov potential games. CoRR , abs/2205.14590

    2024 arXiv

  17. [25]

    and Radanovic, G

    Mandal, D. and Radanovic, G. (2025). Performative reinforcement learning with linear markov decision processes. In AISTATS 2025

    2025

  18. [26]

    Mandal, D., Triantafyllou, S., and Radanovic, G. (2023). Performative reinforcement learning. In ICML 2023

    2023

  19. [27]

    Mao, W., Yang, L., Zhang, K., and Basar, T. (2022). On improving model-free algorithms for decentralized multi-agent reinforcement learning. In ICML 2022

    2022

  20. [28]

    Mendler-D\" u nner, C., Ding, F., and Wang, Y. (2022). Anticipating performativity by predicting from predictions. In NeurIPS 2022

    2022

  21. [29]

    Mendler-D\" u nner, C., Perdomo, J., Zrnic, T., and Hardt, M. (2020). Stochastic optimization for performative prediction. In NeurIPS 2020

    2020

  22. [30]

    Mofakhami, M., Mitliagkas, I., and Gidel, G. (2023). Performative prediction with neural networks. In AISTATS 2023

    2023

  23. [31]

    Narang, A., Faulkner, E., Drusvyatskiy, D., Fazel, M., and Ratliff, L. J. (2023). Multiplayer performative prediction: Learning in decision-dependent games. Journal of Machine Learning Research , 24(202)

    2023

  24. [32]

    C., Zrnic, T., Mendler - D \" u nner, C., and Hardt, M

    Perdomo, J. C., Zrnic, T., Mendler - D \" u nner, C., and Hardt, M. (2020). Performative prediction. In ICML 2020

    2020

  25. [33]

    and Yu, F.-Y

    Piliouras, G. and Yu, F.-Y. (2023). Multi-agent performative prediction: From global stability and optimality to chaos. In EC 2023

    2023

  26. [34]

    Pollatos, V., Mandal, D., and Radanovic, G. (2025). On corruption-robustness in performative reinforcement learning. In AAAI 25

    2025

  27. [35]

    Rank, B., Triantafyllou, S., Mandal, D., and Radanovic, G. (2024). Performative reinforcement learning in gradually shifting environments. In UAI 2024

    2024

  28. [36]

    J., Drusvyatskiy, D., and Fazel, M

    Ray, M., Ratliff, L. J., Drusvyatskiy, D., and Fazel, M. (2022). Decision-dependent risk minimization in geometrically decaying dynamic environments. In AAAI 2022

    2022

  29. [37]

    Shapley, L. S. (1953). Stochastic games. Proceedings of the National Academy of Sciences , 39(10)

    1953

  30. [38]

    P., Hui, F., Sifre, L., van den Driessche, G., Graepel, T., and Hassabis, D

    Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., Chen, Y., Lillicrap, T. P., Hui, F., Sifre, L., van den Driessche, G., Graepel, T., and Hassabis, D. (2017). Mastering the game of Go without human kno...

    2017

  31. [39]

    R., and Shahrampour, S

    Sun, Y., Liu, T., Zhou, R., Kumar, P. R., and Shahrampour, S. (2023). Provably fast convergence of independent natural policy gradient for M arkov potential games. In NeurIPS 2023

    2023

  32. [40]

    S., McAllester, D., Singh, S., and Mansour, Y

    Sutton, R. S., McAllester, D., Singh, S., and Mansour, Y. (1999). Policy gradient methods for reinforcement learning with function approximation. In NeurIPS 1999

    1999

  33. [41]

    M., Mathieu, M., Dudzik, A., Chung, J., Choi, D

    Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019). Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature , 575(7782)

    2019

  34. [42]

    Zhang, R., Mei, J., Dai, B., Schuurmans, D., and Li, N. (2022). On the global convergence rates of decentralized softmax gradient play in M arkov potential games. In NeurIPS 2022

    2022

  35. [43]

    Zhou, Z., Chen, Z., Lin, Y., and Wierman, A. (2023). Convergence rates for localized actor-critic in networked M arkov potential games. In UAI 2023

    2023

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.