Aggregate Fictitious Play for Learning in Anonymous Polymatrix Games (Extended Version)

Semih Kara; Tamer Ba\c{s}ar

REVIEW 3 major objections 4 minor 30 references

Aggregate Fictitious Play for Learning in Anonymous Polymatrix Games (Extended Version)

T0 review · 3 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read In anonymous polymatrix games, aggregate fictitious play—tracking only counts of opponents' actions—converges under exactly the same conditions as classical fictitious play, and the aggregation is lossless for best responses.

desk verdict Aggregate FP is a sound idea with a genuinely useful equivalence lemma, but the model-free convergence theorem has a provable gap in the exploration modeling that needs fixing before the extended claims stand. read the letter →

arxiv 2508.19371 v1 pith:SILCFMWA submitted 2025-08-26 cs.GT cs.LGcs.MAcs.SYeess.SY

Semih Kara , Tamer Başar This is my paper

classification cs.GTcs.LGcs.MAcs.SYeess.SY MSC 91A2691A1062L20

keywords aggregatefictitiousplayanonymousgamespolymatrixNashequilibriumlearningmodel-freereinforcementtwo-timescalestochasticapproximationsuccinctgamerepresentationmulti-agent

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Classical fictitious play lets each agent learn to best-respond by remembering every other agent's action history. In anonymous games, where payoffs depend only on which actions are taken rather than who takes them, this paper asks whether agents can instead remember only the frequency with which each action is played by the others. The answer is yes, provided the game is also polymatrix, meaning rewards decompose into pairwise interactions. The paper proves that in anonymous polymatrix games the aggregate belief yields the same expected rewards and therefore the same best responses as the full individual belief. Consequently aggregate fictitious play converges to Nash equilibrium whenever classical fictitious play does, and the reduced state space speeds up model-free learning.

What carries the argument

The load-bearing structure is the combination of anonymity and polymatrix separability: each agent's reward is a sum of pairwise terms r_i(a_i,a_-i)=Σ_{j≠i} r_ij(a_i,a_j), and by anonymity it depends on other agents only through the count vector x_-i=σ(a_-i). Under this structure, expected reward becomes linear in the aggregate belief, a distribution over count vectors, so only the first moment matters. The paper defines the aggregate belief update as a weighted empirical frequency of count vectors and the aggregate best response using the succinct reward function. Lemma 3's summation swap is the mechanism that equates aggregate and individual expected rewards; Corollary 1 turns that equalit

What would settle it

Construct an anonymous non-polymatrix game, such as three agents with two actions and reward r_i(a_i,x)=(x(a_i))^2 where x(a_i) counts opponents playing that action. Compute the expected reward under the aggregate belief and under the product of individual beliefs for the same history; they will differ, and running fictitious play and aggregate fictitious play on this game would produce divergent action trajectories, showing that the lossless-compression claim is specific to polymatrix games rather than general anonymous games.

Watch

Extended reading notes

Core claim

The central claim is that for anonymous polymatrix games, counting opponents' actions is a lossless compression of tracking each opponent individually. Lemma 3 shows that the expected reward computed from the aggregate belief, a distribution over count vectors, equals the expected reward computed from the product of individual beliefs at every history. Because polymatrix rewards are sums of pairwise terms, the summation over opponents can be reordered and collected into counts, making the expected reward linear in the count vector. Corollary 1 upgrades this equality to best-response equivalence, and Lemma 4 concludes that aggregate fictitious play converges to Nash equilibrium if and only if

Load-bearing premise

The central premise is that rewards are polymatrix, meaning each agent's payoff is a sum of independent pairwise interactions, so expected rewards are linear in the count of opponents' actions; if this linearity fails, aggregating opponents into counts can change best responses and the paper's convergence guarantee lapses.

Editorial extensions

If this is right

Learning in agent populations can scale to mid-size regimes, where mean-field approximations are invalid but full joint-action tracking is infeasible.
Any convergence guarantee already proved for classical fictitious play in polymatrix games, such as zero-sum and potential games, transfers automatically to aggregate fictitious play.
Model-free agents need only Q-tables indexed by one's own action and the opponent count vector, shrinking memory from n^N to n·C(N+n-2,n-1) entries.
Two-timescale aggregate fictitious play converges to an ε-Nash equilibrium in random-payoff anonymous polymatrix games whenever the δ-greedy best-response dynamics is attractive for the expected rewards.
Simulations on a four-agent rock-paper-scissors network show faster Q-table learning and faster approach to equilibrium than two-timescale fictitious play and individual Q-learning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Outside polymatrix games, the aggregate belief generally disagrees with individual beliefs; a testable question is whether aggregate fictitious play still converges in some non-polymatrix anonymous classes, or whether higher moments of the aggregate belief must be tracked.
The reduction in Q-table size grows combinatorially: for 3 actions and 5 agents, 243 reward entries become 45, and the gap widens further with more agents, making the method attractive for settings where identity information is private or costly to share.
The collective exploration scheme is one design choice; the authors suggest agent-independent and decaying exploration rates should also work, which would relax the synchronization requirement in applications.
The attractiveness assumption in the model-free theorem relies on a Lyapunov function for zero-sum and potential polymatrix games, so the practical scope of the guarantee is concrete and can be checked before deployment.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The kernel of this paper is good. The authors define aggregate fictitious play, where agents track the distribution of the count vector of others' actions instead of individual actions, and show that for anonymous polymatrix games this compressed belief preserves expected rewards. That's Lemma 3, and it's correct in spirit, though the proof's first display omits the product over other agents' beliefs, so it reads as an invalid summation until you mentally fill in the missing marginals. The result is right; the presentation is sloppy.

The deterministic consequence, Lemma 4 and Corollary 2, is a clean inheritance result: agg-FP converges whenever FP does for anonymous polymatrix games. That part I'd accept. Lemma 1, the proof that the two standard definitions of anonymous games are equivalent, is also a nice fill-in of a gap that most papers cite without proof.

The problem is the model-free extension. Theorem 1's proof goes through the δ-greedy agg-BR ODE in Eq. (10), which is derived from the assumption that exploration places uniform probability δ/|X| on every count vector. But Algorithm 1's exploration is each agent drawing uniformly from its own action set, so the count vector is multinomial, not uniform over the count simplex. The martingale difference error in Eq. (9) therefore does not have the claimed conditional mean, and the ODE (10) is not the mean field of the implemented algorithm. That's a real gap in the proof of Theorem 1 as written. It does not kill the deterministic part, and it is likely repairable: the first moments of the two exploration distributions coincide, and since polymatrix rewards are linear in counts, the action-level dynamics may survive. But the repair is not in the paper.

The simulations are illustrative but thin: one run, no error bars. For a paper claiming faster convergence, I'd want a few seeds at minimum.

Overall: the idea is worth engaging with, and the deterministic agg-FP result is a solid contribution. The model-free theorem needs work before it should be stated as proved. I'd send it to peer review, because the core contribution is real, but I'd make clear the exploration-model mismatch needs to be fixed.

Referee Report

3 major / 4 minor

Summary. This paper introduces aggregate fictitious play (agg-FP), a variant of fictitious play for anonymous matrix games in which each agent maintains a belief over the count vector of other agents' actions rather than over each individual action. For anonymous polymatrix games, the authors prove (Lemma 3 / Corollary 1) that these aggregate beliefs are best-response equivalent to standard FP beliefs, and hence agg-FP inherits FP's convergence guarantees (Lemma 4, Corollary 2). The paper then adds a faster-timescale Q-learning estimator to handle model-free random-payoff games (Algorithm 1), and claims convergence to a neighborhood of Nash under attractiveness of the δ-greedy best-response dynamics (Theorem 1). Numerical experiments on a 4-agent rock-paper-scissors game illustrate faster convergence than two-timescale FP and individual Q-learning.

Significance. If the results hold, the paper makes a useful and clean contribution: it identifies a structural condition (anonymity + polymatrix separability) under which the aggregate count statistic is a sufficient statistic for best responses, reducing the relevant joint-action space from n^N to n·C(N+n−2, n−1). The deterministic inheritance result is conceptually important, and the paper is honest about the failure of the equivalence outside polymatrix games (Remarks 6 and 8). The central proof idea is simple and elegant, and the paper ships a fully formal definition of the algorithm, explicit limitations, and reproducible simulation parameters. However, several load-bearing proofs as written contain summation errors, and the exploration model in the model-free theorem is not the one implemented in Algorithm 1. These issues are repairable, but they currently prevent the paper from being accepted as is.

major comments (3)

[Section IV-A, Lemma 3, Eq. (5)] The displayed chain of equalities starts with R_i(a_i, π̂^{-i}_k) = Σ_{a−i} Σ_{j≠i} r_i(a_i,a_j) Σ_{l=0}^{k} ᾱ_l 1{a_j = a_j^l}. This omits the product over m≠j of π̂_m^k(a_m). As written, summing over a−i introduces an extraneous factor n^{N−2}, so the displayed equality is false. The lemma statement is nevertheless true in anonymous polymatrix games; a correct proof should first reduce R_i to Σ_j Σ_{a_j} r_i(a_i,a_j) π̂_j^k(a_j) (using the fact that anonymity makes the pairwise reward independent of j), then substitute π̂_j^k and pass to the aggregate count x. Because Corollary 1 and Lemma 4 are built directly on Lemma 3, this proof must be corrected.
[Section IV-B2, Lemma 5] The proof of dR_i/dt contains the same summation error. It writes dR_i/dt as a sum over a−i ∈ A^{⊗(N−1)} of r_i(a_i,a_j) times the BR/exploration terms, but the correct derivative must marginalize over a_m for m≠j: dR_i/dt = Σ_j Σ_{a_j} r_i(a_i,a_j) [ (1−δ)1{a_j = a_j^{t,*}} + δ/n − π_j^t(a_j) ]. Similarly, the conversion to aggregate rewards should use this marginal expression. The version in the manuscript introduces an extraneous factor n^{N−2}. This affects Corollary 3, which is used in the proof of Theorem 1. The equality of limit sets may still be true, but the proof as written is invalid.
[Section IV-B, Eq. (9) and Remark 9 vs. Algorithm 1] Eq. (9) defines E_i^k = 1{X^{-i}_k} − [ (1−δ)1{σ(a^{−i}_{k,*})} + δ 1/|X| ] and claims E_i^k is a zero-mean martingale difference because the conditional expectation of 1{X^{-i}_k} given μ̂_k is the bracketed expression. This is not the conditional expectation under Algorithm 1's exploration rule, where each agent draws A_i ∼ Uniform(A) independently: conditional on exploration, X^{-i}_k is multinomial(N−1; uniform over A), not uniform over X. For example, with N=4 and n=3, P(X^{-i}=(3,0,0)) = 1/27 under independent uniform actions, whereas uniform over X assigns probability 1/10. Thus ODE (10) is not the mean-field limit of Algorithm 1's μ-update, and Theorem 1's proof, which passes through (10) and Corollary 3, lacks a valid proof for the algorithm as stated. The issue is plausibly repairable by replacing δ 1/|X| with δ times the multinomial exploration distribution; for anonymous poly

minor comments (4)

[Remark 9 / Algorithm 1] The phrase 'collectively explore' is ambiguous. Remark 9's distribution over x^{−i} suggests uniform exploration over the count space X, while Algorithm 1's pseudocode samples each A_i independently from Uniform(A). These are different distributions over X. The authors should either change Algorithm 1 to sample from X uniformly (if that is the intended model) or rewrite Eq. (9)–(10) with the multinomial distribution.
[Section II-A, polymatrix definition] The polymatrix definition writes r_i(a_i,a_{−i}) = Σ_{j≠i} r_{ij}(a_i,a_j). In an anonymous polymatrix game, anonymity forces r_{ij}(a_i,·) to be independent of j. This fact is used implicitly in Lemma 3 and Lemma 5; it should be stated explicitly, since the proofs otherwise appear to assume it without justification.
[Section VI, Eq. (13)] In the simulation, the game is stated to be zero-sum after taking expectations. It would help to state explicitly that the perturbing variables Θ_i have zero mean, so E[r_i^{Θ_i}] = r_i. This is true for the chosen distribution but is not written out.
[General] Definition 3 and the surrounding text say agg-FP converges if the empirical action frequencies converge to NE(G). Since the same weighted empirical frequencies are also the belief sequence in standard FP, this definition is natural, but the paper could clarify that γ̂_k and π̂_k coincide when the same step sizes are used.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: aggregate/individual equivalence is proved, and convergence inheritance is a corollary, not a re-labeling.

full rationale

Adopting the review standard, I looked for points where a result is equivalent to its inputs by construction. The central inheritance claim (Lemma 4 via Corollary 1/Lemma 3) is a genuine equivalence proof, not a circular one: Lemma 3 shows that for anonymous polymatrix games the aggregate belief \hat\mu and the individual beliefs \hat\pi generate the same expected rewards, with the polymatrix separability making the double sum over time and agents commute. Corollary 1 then gives identical action trajectories, and Definition 3's \hat\gamma coincides with the FP belief \hat\pi, so convergence inheritance is a proved corollary rather than a fitted or self-referential prediction. The two-timescale theorem uses external stochastic-approximation results [29, 30], and its assumption about the δ-greedy BR dynamics is not the same as the conclusion about the algorithm. Self-citations [20], [21], [2], [22] appear only as context or standard references and are not load-bearing. I also weighed the manuscript's own caveats: Remarks 6, 7, 8, and 14 explicitly restrict the equivalence to polymatrix games and admit information loss, which supports non-circularity. Two non-circular proof-quality concerns exist: Lemma 3's displayed calculation compresses the product structure of \hat\pi^{-i} (the factor |A|^{N-2} is not written), and Remark 9/Eq. (9)-(10) model exploration as uniform over the count space X, whereas Algorithm 1's collective uniform-action exploration yields a multinomial count distribution. These are correctness gaps, not cases where an output is defined to be an input, so they do not raise the circularity score beyond the minor range.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

No fitted parameters or invented entities are used to force the theoretical results. The free parameters listed are simulation hyperparameters. The theory relies on standard external results in stochastic approximation and fictitious play convergence, plus the explicit domain assumptions of anonymous random-payoff polymatrix games.

free parameters (2)

exploration probability delta = 0.1 in simulations
Algorithmic parameter of delta-greedy exploration; the theory allows sufficiently small delta depending on the game. Not fitted to data.
step-size exponents for alpha_k and beta_k = 0.7 and 0.6 (alpha_k=(k+1)^(-0.7), beta_k=(k+1)^(-0.6))
Hand-chosen sequences satisfying the Robbins-Monro and two-timescale conditions; specific values used only in simulation.

assumptions (5)

standard math Nash existence for finite games
Invoked in Section II-A: NE(G) is always nonempty [22].
standard math FP converges for zero-sum polymatrix games and potential games
Used in Corollary 2 to transfer FP convergence to agg-FP, citing [7] and [6].
standard math Stochastic approximation theory of Benaïm-Hofbauer-Sorin [29] and Perkins-Leslie [30]
Used to pass from discrete updates to differential inclusions in Section IV-B and in the proof sketch of Theorem 1.
domain assumption Random payoff model: Theta_i i.i.d., bounded, finite mean and variance, independent across agents and actions
Assumed in Section II-C for the model-free setting.
domain assumption NE_epsilon(G_P) is attractive under the delta-greedy BR dynamics for the expected reward game
Stated hypothesis of Theorem 1; guaranteed for zero-sum polymatrix and potential games for sufficiently small delta (Remark 12).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Aggregate Fictitious Play for Learning in Anonymous Polymatrix Games (Extended Version)." pith.science (2026). https://pith.science/paper/SILCFMWA

@misc{pith2026250819371,
  author       = {Pith},
  title        = {Pith review of: Aggregate Fictitious Play for Learning in Anonymous Polymatrix Games (Extended Version)},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SILCFMWA}},
  note         = {Machine review of arXiv:2508.19371}
}

read the original abstract

Fictitious play (FP) is a well-studied algorithm that enables agents to learn Nash equilibrium in games with certain reward structures. However, when agents have no prior knowledge of the reward functions, FP faces a major challenge: the joint action space grows exponentially with the number of agents, which slows down reward exploration. Anonymous games offer a structure that mitigates this issue. In these games, the rewards depend only on the actions taken; not on who is taking which action. Under such a structure, we introduce aggregate fictitious play (agg-FP), a variant of FP where each agent tracks the frequency of the number of other agents playing each action, rather than these agents' individual actions. We show that in anonymous polymatrix games, agg-FP converges to a Nash equilibrium under the same conditions as classical FP. In essence, by aggregating the agents' actions, we reduce the action space without losing the convergence guarantees. Using simulations, we provide empirical evidence on how this reduction accelerates convergence.

Figures

Figures reproduced from arXiv: 2508.19371 by the authors.

**Figure 2.** Empirical action frequencies of agent 1 from agg-FP. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** Total Q-error across all agents and action profiles. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Total l1 distance to Nash equilibrium across all agents. VII. CONCLUSION AND ACKNOWLEDGMENT We have studied learning Nash equilibria in anonymous games through repeated play. We have introduced agg-FP and proved that it converges under the same conditions as FP for ano…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

30 extracted references · 29 canonical work pages

[1]

Fudenberg and D

D. Fudenberg and D. Levine, The Theory of Learning in Games , ser. Economics Learning and Social Evolution Series. MIT Press, 1998

work page 1998
[2]

Zhang, Z

K. Zhang, Z. Yang, and T. Bas ¸ar,Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms . Cham: Springer International Publishing, 2021, pp. 321–384

work page 2021
[3]

Independent Learning in Stochastic Games

A. Ozdaglar, M. O. Sayin, and K. Zhang, “Independent learning in stochastic games,” 2021, arXiv:2111.11743 [cs.GT]

work page Pith review arXiv 2021
[4]

G. W. Brown, Some Notes on Computation of Games Solutions. Santa Monica, CA: RAND Corporation, 1949

work page 1949
[5]

An iterative method of solving a game,

J. Robinson, “An iterative method of solving a game,” Annals of Mathematics, vol. 54, no. 2, pp. 296–301, 1951

work page 1951
[6]

Potential games,

D. Monderer and L. S. Shapley, “Potential games,” Games and Economic Behavior, vol. 14, no. 1, pp. 124–143, 1996

work page 1996
[7]

Fictitious play in networks,

C. Ewerhart and K. Valkanova, “Fictitious play in networks,” Games and Economic Behavior , vol. 123, pp. 182–206, 2020

work page 2020
[8]

M. J. Osborne and A. Rubinstein, A Course in Game Theory . MIT Press, 1994

work page 1994

Show all 30 references

[9]

Convergent multiple-timescales rein- forcement learning algorithms in normal form games,

D. S. Leslie and E. J. Collins, “Convergent multiple-timescales rein- forcement learning algorithms in normal form games,” The Annals of Applied Probability, vol. 13, no. 4, pp. 1231–1251, 2003

2003
[10]

Individual Q-learning in normal form games,

——, “Individual Q-learning in normal form games,” SIAM Journal on Control and Optimization , vol. 44, no. 2, pp. 495–514, 2005

2005
[11]

Fictitious play in zero-sum stochastic games,

M. O. Sayin, F. Parise, and A. Ozdaglar, “Fictitious play in zero-sum stochastic games,” SIAM Journal on Control and Optimization, vol. 60, no. 4, pp. 2095–2114, 2022
[12]

Generalized individual Q-learning for polymatrix games with partial observations,

A. S. Donmez and M. O. Sayin, “Generalized individual Q-learning for polymatrix games with partial observations,” 2024, arXiv:2409.02663 [cs.GT]

2024 arXiv
[13]

Congestion games with player-specific payoff func- tions,

I. Milchtaich, “Congestion games with player-specific payoff func- tions,” Games and Economic Behavior , vol. 13, no. 1, pp. 111–124, 1996

1996
[14]

Characterization of pure strategy equilibria in finite anonymous games,

M. Blonski, “Characterization of pure strategy equilibria in finite anonymous games,” Journal of Mathematical Economics , vol. 34, no. 2, pp. 225–233, 2000

2000
[15]

Z. Han, D. Niyato, W. Saad, T. Bas ¸ar, and A. Hjørungnes, Game Theory in Wireless and Communication Networks: Theory, Models, and Applications. Cambridge University Press, 2011

2011
[16]

O’Hara, Market Microstructure Theory

M. O’Hara, Market Microstructure Theory. John Wiley & Sons, 1998

1998
[17]

Equilibria of polymatrix games,

J. T. Howson, “Equilibria of polymatrix games,” Management Science, vol. 18, no. 5, pp. 312–318, 1972

1972
[18]

Zero- sum polymatrix games: A generalization of minmax,

Y . Cai, O. Candogan, C. Daskalakis, and C. H. Papadimitriou, “Zero- sum polymatrix games: A generalization of minmax,” Mathematics of Operations Research, vol. 41, pp. 648–655, 2016

2016
[19]

Learning in mean field games: A survey,

M. Lauri `ere, S. Perrin, J. P ´erolat, S. Girgin, P. Muller, R. ´Elie, M. Geist, and O. Pietquin, “Learning in mean field games: A survey,” 2024, arXiv:2205.12944 [cs.LG]

2024 arXiv
[20]

Differential equation approxima- tions for population games using elementary probability,

S. Kara and N. C. Martins, “Differential equation approxima- tions for population games using elementary probability,” 2023, arXiv:2312.07598 [cs.GT]

2023 arXiv
[21]

Oracle-free reinforcement learning in mean-field games along a single sample path,

M. A. U. Zaman, A. Koppel, S. Bhatt, and T. Bas ¸ar, “Oracle-free reinforcement learning in mean-field games along a single sample path,” in Proceedings of the 26th International Conference on Artificial Intelligence and Statistics, vol. 206. PMLR, 2023, pp. 10 178–10 206

2023
[22]

Bas ¸ar and G

T. Bas ¸ar and G. J. Olsder, Dynamic Noncooperative Game Theory, 2nd Edition. Society for Industrial and Applied Mathematics, 1998

1998
[23]

Notions of anonymity, fairness and symmetry for finite strategic-form games,

N. Ham, “Notions of anonymity, fairness and symmetry for finite strategic-form games,” 2021, arXiv:1311.4766 [math.CO]

2021 arXiv
[24]

Symmetry in n-player games,

A. Plan, “Symmetry in n-player games,” Journal of Economic Theory, vol. 207, p. 105549, 2023

2023
[25]

Zero-zero chance- constrained games,

A. Charnes, M. J. L. Kirby, and W. M. Raike, “Zero-zero chance- constrained games,” Theory of Probability & Its Applications , vol. 13, no. 4, pp. 628–646, 1968

1968
[26]

Games with randomly disturbed payoffs: A new rationale for mixed-strategy equilibrium points,

J. C. Harsanyi, “Games with randomly disturbed payoffs: A new rationale for mixed-strategy equilibrium points,” International Journal of Game Theory , vol. 2, no. 5, pp. 312–318, 1973

1973
[27]

Computing equilibria in anony- mous games,

C. Daskalakis and C. Papadimitriou, “Computing equilibria in anony- mous games,” in 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS’07) , 2007, pp. 83–93

2007
[28]

Symmetries and the complexity of pure Nash equilibrium,

F. Brandt, F. Fischer, and M. Holzer, “Symmetries and the complexity of pure Nash equilibrium,” in STACS 2007. Springer Berlin Heidel- berg, 2007, pp. 212–223

2007
[29]

Stochastic approximations and differential inclusions,

M. Bena ¨ım, J. Hofbauer, and S. Sorin, “Stochastic approximations and differential inclusions,” SIAM Journal on Control and Optimization , vol. 44, no. 1, pp. 328–348, 2005

2005
[30]

Asynchronous stochastic approximation with differential inclusions,

S. Perkins and D. S. Leslie, “Asynchronous stochastic approximation with differential inclusions,” Stochastic Systems, vol. 2, no. 2, pp. 409– 446, 2013

2013

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

Fudenberg and D

D. Fudenberg and D. Levine, The Theory of Learning in Games , ser. Economics Learning and Social Evolution Series. MIT Press, 1998

work page 1998

[2] [2]

Zhang, Z

K. Zhang, Z. Yang, and T. Bas ¸ar,Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms . Cham: Springer International Publishing, 2021, pp. 321–384

work page 2021

[3] [3]

Independent Learning in Stochastic Games

A. Ozdaglar, M. O. Sayin, and K. Zhang, “Independent learning in stochastic games,” 2021, arXiv:2111.11743 [cs.GT]

work page Pith review arXiv 2021

[4] [4]

G. W. Brown, Some Notes on Computation of Games Solutions. Santa Monica, CA: RAND Corporation, 1949

work page 1949

[5] [5]

An iterative method of solving a game,

J. Robinson, “An iterative method of solving a game,” Annals of Mathematics, vol. 54, no. 2, pp. 296–301, 1951

work page 1951

[6] [6]

Potential games,

D. Monderer and L. S. Shapley, “Potential games,” Games and Economic Behavior, vol. 14, no. 1, pp. 124–143, 1996

work page 1996

[7] [7]

Fictitious play in networks,

C. Ewerhart and K. Valkanova, “Fictitious play in networks,” Games and Economic Behavior , vol. 123, pp. 182–206, 2020

work page 2020

[8] [8]

M. J. Osborne and A. Rubinstein, A Course in Game Theory . MIT Press, 1994

work page 1994