Pith. sign in

REVIEW 3 major objections 4 minor 55 references

Learning Recommender Mechanisms for Bayesian Stochastic Games

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper gives the first approach for learning recommender mechanisms in Bayesian stochastic games: a neural map from reported types to policies, trained so lying, deviating, and opting out do not pay.

desk verdict First to frame recommender mechanisms for Bayesian stochastic games, but the stochastic-game incentive results rest on critics that can't see misreports and were never trained on them, so the headline claim is unverified. read the letter →

arxiv 2505.22979 v1 pith:NY3OD2G2 submitted 2025-05-29 cs.GT cs.MA

Bengisu Guresti , Chongjie Zhang , Yevgeniy Vorobeychik This is my paper
classification cs.GTcs.MA
keywords recommendermechanismsBayesianstochasticgamesmechanismdesignmulti-agentreinforcementlearningincentivecompatibilityindividualrationalitysocialwelfarebi-leveloptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper takes up equilibrium selection in games with private information: instead of hoping independent players converge to a good equilibrium, a designer collects reported types and recommends a policy to each player. The paper claims to give the first recommender mechanism for Bayesian stochastic games, where earlier recommender-mechanism work covered only one-shot or extensive-form games. Its central proposal, ReMBo, is a neural network mapping type reports to Markov stationary, deterministic policies, trained by bi-level reinforcement learning from a simulator. The training objective maximizes social welfare while penalizing, through learned value and Q-functions, any player's gain from misreporting a type, deviating from the recommended policy, or opting out. If the method works as claimed, a designer can deploy an approximately incentive-compatible, individually rational coordination device without payments and without private recommendation channels.

What carries the argument

The load-bearing object is the bi-level reinforcement-learning formulation. The outer level learns parameters $\phi$ of the mechanism $M$, split into $M^n$ for full participation and $M^{n-1}_{-i}$ for the contingency where player $i$ opts out. The inner level is the best-response problem inside each incentive term. Rather than solving a fresh RL problem at every mechanism update, ReMBo uses learned critics: the IC loss sums, over each player $i$, misreported type $\theta'_i$, and action $a_i$, the positive part of $Q_i(s,a_i,M^n_{-i}(\theta'_i,\theta_{-i}),\theta_i)-V_i(s,M^n(\theta),\theta_i)$, and the IR loss does the same with the opt-out critic $Q^{n-1}_i$ relative to $V_i$. Straight-through Gumbel-Softmax makes these discrete-action penalties differentiable in $\phi$, while $Q_i$, $V_i$, and $Q^{n-1}_i$ are trained with conventional DQN/TD3-style updates whose targets use the mechanism's own evolving policies. A shared replay buffer pools data from the welfare objective and from deviation learning. Theorem 1 is the formal bridge: satisfying the two constraints makes truthful reporting, compliance, and participation an approximate Bayes-Nash equilibrium.

What would settle it

In a small Bayesian stochastic game with two states, two actions, and two types per player, enumerate every deterministic Markov stationary policy profile and every unilateral deviation by brute force. If the exact $\epsilon$-BNIC and $\epsilon$-IR values from that enumeration disagree systematically with the values ReMBo's converged critics report at the end of training, then the incentive guarantees claimed by the paper are not certified by its own evaluation.

Watch

Extended reading notes

Core claim

The paper's central claim is that recommender mechanisms can be extended from static and extensive-form games to Bayesian stochastic games, and can be learned end-to-end rather than derived analytically. ReMBo treats the mechanism as a parametric meta-policy $M:\Theta\to\Pi$ that takes a reported type profile and returns a full profile of deterministic Markov stationary policies. The designer's objective is a Lagrangian relaxation of welfare maximization under two coupled incentive constraints: $\epsilon$-BNIC couples truthful reporting with compliance, so that no player gains from reporting a false type and then best-responding to the policies prescribed to others, and $\epsilon$-IR requires that no player gains by keeping the type private and best-responding to the mechanism's policies for everyone else. Theorem 1 states that a mechanism solving the constrained problem makes truth-telling, compliance, and participation an $\epsilon$-Bayes-Nash equilibrium. The experiments argue that the learned mechanism reaches social welfare competitive with cooperative multi-agent reinforcement learning baselines, and in the stochastic-game domains often surpasses them, while exhibiting smaller IC and IR violations.

Load-bearing premise

The whole method depends on the learned value and Q-functions used to measure incentives being accurate for every lie, deviation, and opt-out a player could try; if those value estimates are off, the mechanism is optimized against the wrong incentives and the reported incentive-violation numbers are unreliable.

Editorial extensions

If this is right

  • A designer who only has a simulator, not a closed-form game model, can train an approximately incentive-compatible mechanism.
  • At execution, the recommendations are produced by a single forward pass of a neural network on the reported type profile, so deployment cost is low.
  • Public, Markov stationary, deterministic recommendations suffice, removing the need for private communication channels or correlated-signal security.
  • Varying the $\alpha_0$, $\alpha_1$, $\alpha_2$ weights lets the designer choose where to sit on the welfare-versus-incentives frontier.
  • In the lane-changing and congestion domains, the incentive-aware training often reaches higher social welfare faster than the same RL algorithms without mechanism losses, so the incentive terms act as an exploration signal rather than a pure tax.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the IC loss sums positive parts over misreports and actions rather than taking a single maximum, it is an upper bound on the true best-response gain; the paper does not pursue the tighter route of solving the inner maximization exactly, so a natural follow-up is to measure how much slack this upper bound introduces.
  • The opt-out branch $M^{n-1}_{-i}$ is in effect learning deterrence policies against a deviator; extending ReMBo to make the punishing agents themselves willing to follow those policies would upgrade the guarantee from Bayes-Nash to subgame-perfect equilibrium, an issue the paper explicitly leaves open.
  • The paper's stochastic-game evaluation uses the same converged critics used for training to report IC and IR violations; a stronger validation, not pursued here, would compare those numbers with independent Monte Carlo rollouts against a separately trained best-response policy.
  • The observed welfare gains from the IC and IR losses in lane-changing and congestion games suggest that deviation-aware exploration carries useful information about the game; one extension would be to add ReMBo-style auxiliary incentive losses to ordinary cooperative MARL and test whether exploration improves even without deploying a mechanism.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces ReMBo, a bi-level reinforcement learning method for designing recommender mechanisms in Bayesian stochastic games. The mechanism maps reported type profiles to deterministic Markov stationary policy recommendations, without payments, and aims to maximize social welfare subject to approximate Bayes-Nash incentive compatibility (IC) and individual rationality (IR). The authors formulate surrogate IC and IR losses (Eqs. 5-7) using learned Q and V functions with straight-through Gumbel-Softmax, and evaluate on two repeated matrix games, a lane-changing game, and two congestion games, comparing against MA-TD3, VDN, and DQN-T baselines. The paper claims the first approach for this setting, with social welfare competitive with cooperative MARL and significantly improved incentive properties.

Significance. If the stochastic-game incentive results are validated, this is a substantial contribution: it extends recommender mechanisms from one-shot and extensive-form games to stochastic games with private types, and the parametric mechanism representation is efficient at execution time. The exact brute-force verification of IC and IR for the repeated matrix games (Appendix B.1) is a genuine strength and provides a credible proof-of-concept. However, the central claim for the lane-changing and congestion games rests on learned critics that serve both as the training objective and the evaluation metric; without independent verification, the significance of those results cannot yet be assessed.

major comments (3)
  1. [§3, Eq. (6) and Eq. (8)] The IC loss in Eq. (6) does not faithfully represent the best response to a misreported type. Q_i(s, a_i, M^n_{φ^n,-i}(θ'_i, θ_{-i}), θ_i) has no input for θ'_i, and the training target in Eq. (8) uses a'_{-i} = M^n_{φ^n,-i}(s', θ_{-i}), i.e., the truthful type profile. Thus Q_i is learned only under truthful reports of the other agents; plugging in actions induced by θ'_i is an out-of-distribution query, and the continuation value after the current state is that of the truthful mechanism, not of the mechanism induced by θ'_i. Consequently, optimizing L_IC need not reduce the actual BNIC violation of Definition 2, and the stochastic-game IC improvements in Figures 2 and 3 may be artifacts of critic extrapolation. In addition, the sum over θ'_i in Eq. (6) does not correspond to the max over θ'_i in Definition 2. Please condition Q_i on the full reported profile (including θ'_i), train its targets under M^n(θ'_i, θ_{-i}), and validate with an independent best-response computation.
  2. [§3, Algorithm 1] The replay buffer D is filled exclusively with transitions generated by M^n(θ) under ε-greedy exploration, yet the same D is used to update Q^{n-1}_{ψ^{n-1},-i}, Q^{n-1}_{ψ^{n-1},i}, and π'_{φ',i} for the opt-out contingency. These updates are off-policy with respect to the opt-out game (n-1 agents best responding without agent i), and no importance-weighting or off-policy correction is described. The IR loss in Eq. (7) therefore does not estimate the opt-out value in Definition 1; it reflects values of a critic trained on equilibrium-path data. The reported IR improvements for the lane-changing and congestion games are thus not trustworthy as measures of true individual rationality. Please collect rollouts under the opt-out and deviation policies, or add proper off-policy corrections, before reporting these quantities.
  3. [§4 and Appendix A.2] For the stochastic games, the IC and IR violations shown in Figures 2 and 3 are estimated with the same converged critic networks that define the ReMBo training objective; this is circular. Exact brute-force verification is provided only for the repeated matrix games (Section B.1), and the manuscript does not offer an independent check for the lane-changing or congestion games. The claim of 'significantly improved incentive properties' for the stochastic domains therefore lacks independent support. I recommend an evaluation that either solves small stochastic instances exactly (e.g., by dynamic programming over the small state/type spaces) or trains a separate best-response policy per type profile from scratch and uses its empirical value as the deviation benchmark, without reusing the ReMBo critics.
minor comments (4)
  1. [§2.2, Definition 1 and Eq. (7)] The IR loss in Eq. (7) writes Q^{n-1}_i(s, a_i, M^{n-1}_{φ^{n-1},-i}(θ_{-i}), θ) with the full type profile as the last argument, whereas the utility in Definition 1 depends only on θ_i; please clarify the intended input.
  2. [Algorithm 2] It is unclear how gradients of the mechanism loss in Eq. (5) flow through the argmax defining M^n(θ) = argmax_a Q^n_{ψ^n}(θ); a sentence explaining the straight-through treatment for this step would help.
  3. [Appendix B.1 vs. Figure 11] The text says IC and IR losses for matrix games are computed exactly by brute-force search, but the caption of Figure 11 says 'Estimated IC deviation'; please align the terminology.
  4. [References and text] The Lillicrap et al. entry appears twice (2015 and 2019); the reference 'V orobeychik' contains a stray space; and the phrase 'the third consideration corresponds as individual rationality' should be rephrased.

Circularity Check

1 steps flagged · score 6.0 of 10

Stochastic-game incentive improvements are measured with the same learned critics used as the ReMBo training loss, so the headline IC/IR gains partly reduce to optimizing the evaluation metric.

  1. fitted input called prediction [Eqs. (5)-(7), Sec. 4, Appendix A.2]
    "L^i_IC(φ^n, θ) = E_{s∼ρ}[ Σ_{θ'_i} Σ_{a_i∈A_i} ReLU( Q_i(s, a_i, M^n_{φ^n,−i}(θ'_i, θ_{−i}), θ_i) − V_i(s, M^n_{φ^n}(θ), θ_i) )] (Eq. 6). ... For evaluation, IC and IR losses are estimated with the converged critic neural networks at the end of the training. (Appendix A.2)"

    The ReMBo objective in Eq. (5) is min_φ E[α0 L_RL + α1 L_IC + α2 L_IR], where L_IC and L_IR are defined by exactly the ReLU(Q−V) surrogates in Eqs. (6)-(7) using learned critics Q_i, V_i, and Q^{n-1}_i. The reported IC and IR deviations for lane-changing and congestion games are then computed, per Appendix A.2, from the converged versions of the same critics. Thus the claimed 'significantly improved incentive properties' of ReMBo in stochastic games is largely a decrease in the very quantity ReMBo was trained to minimize; the mechanism can reduce the reported metric by exploiting the critics' extrapolation errors rather than by satisfying Definitions 1-2.

full rationale

The paper's derivation is self-contained in structure: Theorem 1 is a direct consequence of the definitions, social welfare is evaluated with realized returns (not learned critics), and repeated matrix games use exact brute-force IC/IR checks. No load-bearing self-citation or imported uniqueness theorem is present. However, the central stochastic-game claim of improved IC and IR properties is not independently verified: the same learned critics that define the ReMBo loss (Eqs. 6-7) are reused in Appendix A.2 to estimate the reported IC and IR violations. This makes the reported incentive improvements in lane-changing and congestion games partly circular, since optimizing the loss can reduce the evaluation metric without genuinely improving incentive properties. The paper's own contrast with the exact repeated-game evaluation highlights the missing independent check in stochastic games. Because the social-welfare claims and the repeated-game incentive results remain independent, the circularity is partial, not total, but it affects the headline claim of the experimental evaluation.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central claim rests on standard Bayesian-game assumptions, a simulator, deterministic stationary policies, no payments, and the accuracy of learned critics for the incentive losses. The only hand-tuned quantities in the objective are the IC/IR weights and the Gumbel temperature; they vary by domain. No novel physical or mathematical entities are introduced.

free parameters (3)
  • alpha1 (IC weight) = 50 (matrix, lane), 20 (MA-TD3 congestion), 100 (VDN/DQN congestion)
    Hand-chosen coefficient in the Lagrangian objective (Eq. 5) balancing social welfare against incentive compatibility; values vary by domain and are not supported by a sensitivity analysis.
  • alpha2 (IR weight) = same values as alpha1 per domain
    Hand-chosen coefficient for the individual-rationality loss; no sensitivity analysis is provided.
  • Gumbel-Softmax temperature = 1
    Temperature for the straight-through relaxation used to differentiate through discrete action selection; fixed across all domains, not tuned.
assumptions (6)
  • domain assumption Common prior Pi over each agent's finite type set Thetai
    Stated in Sec. 2.1; the mechanism optimizes ex-ante welfare and incentives under this prior.
  • domain assumption Simulator for environment dynamics and rewards is available
    Sec. 2.2: 'we do assume that a simulator for the game is available'; all learning and evaluation use this simulator.
  • domain assumption Restriction to deterministic Markov stationary policies
    Sec. 1 and Sec. 2.1; justified by purification results, but it limits the mechanism class and can reduce the achievable welfare/incentive tradeoff.
  • domain assumption No payments are allowed
    Sec. 2.2 states 'we allow no payments as part of the mechanism'; this makes exact incentive alignment impossible in general.
  • ad hoc to paper Learned Q/V networks are accurate best-response estimators
    Eqs. (6)-(7) and Appendix A.2; the IC/IR losses and the final stochastic-game evaluation both assume these critics estimate true deviation values, with no convergence certificate.
  • domain assumption Ex-ante Bayes-Nash equilibrium is the relevant solution concept
    Sec. 3 discusses punishments; the paper explicitly accepts Bayes-Nash rather than subgame perfection because policies are fixed code.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning Recommender Mechanisms for Bayesian Stochastic Games." pith.science (2026). https://pith.science/paper/NY3OD2G2

@misc{pith2026250522979,
  author       = {Pith},
  title        = {Pith review of: Learning Recommender Mechanisms for Bayesian Stochastic Games},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NY3OD2G2}},
  note         = {Machine review of arXiv:2505.22979}
}
read the original abstract

An important challenge in non-cooperative game theory is coordinating on a single (approximate) equilibrium from many possibilities - a challenge that becomes even more complex when players hold private information. Recommender mechanisms tackle this problem by recommending strategies to players based on their reported type profiles. A key consideration in such mechanisms is to ensure that players are incentivized to participate, report their private information truthfully, and follow the recommendations. While previous work has focused on designing recommender mechanisms for one-shot and extensive-form games, these approaches cannot be effectively applied to stochastic games, particularly if we constrain recommendations to be Markov stationary policies. To bridge this gap, we introduce a novel bi-level reinforcement learning approach for automatically designing recommender mechanisms in Bayesian stochastic games. Our method produces a mechanism represented by a parametric function (such as a neural network), and is therefore highly efficient at execution time. Experimental results on two repeated and two stochastic games demonstrate that our approach achieves social welfare levels competitive with cooperative multi-agent reinforcement learning baselines, while also providing significantly improved incentive properties.

Figures

Figures reproduced from arXiv: 2505.22979 by the authors.

Figure 1
Figure 1. Results for Chicken (top) and Stag Hunt (bottom). Left: social welfare, middle: IC deviation, right: IR deviation. Repeated Matrix Games We consider Bayesian variants of Chicken and Stag Hunt. In the former, types are θ ∈ {Risk Averse (RA), Risk Taking (RT)}, designed such that when both agents are RT, the game reduces to classic Chicken. For Stag Hunt, types θ ∈ {Stag (S), Rabbit (R)} determine whether the agent pr… view at source ↗
Figure 2
Figure 2. Results for the lane-changing game. Top: 15 agents. Bottom: 30 agents. Left: social [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Results for congestion games. Top: 3-destination games. Bottom: intersection games. Left: [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: (a) MLP actor architecture for the mechanism [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: (a) CNN actor architecture for the mechanism [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: (a) MLP critic architecture for Qn ψn (Mn ϕn (θ), θ), (b) MLP critic architecture for Q n−1 ψn−1,−i (Mn−1 ϕn−1,−i (θ−i), θ), (c) MLP critic architecture for Q n−1 ψn−1,i(π ′ ϕ′ ,i,Mn−1 ϕn−1,−i (θ−i), θi) (a) Mn ϕn (θ) (b) Mn−1 ϕn−1 (θ−i) (c) π ′ ϕn−1,i(θi) [PITH_FULL_…
Figure 7
Figure 7. Figure 7: (a) CNN critic architecture for Qn ψn (Mn ϕn (θ), θ), (b) CNN critic architecture for Q n−1 ψn−1,−i (Mn−1 ϕn−1,−i (θ−i), θ), (c) CNN critic architecture for Q n−1 ψn−1,i(π ′ ϕ′ ,i,Mn−1 ϕn−1,−i (θ−i), θi) (a) Q n ψn (θ) (b) Q n−1 ψn−1,−i (θ−i) (c) Q n−1 ψn−1,i(θi) [PIT…
Figure 8
Figure 8. Figure 8: (a) MLP critic architecture for the mechanism Q-network [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: (a) CNN critic architecture for the mechanism Q-network [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: (a) MLP critic architecture for the Q-networks [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Results for Chicken (top) and Stag Hunt (bottom). Left: Estimated IC deviation, right: Estimated IR deviation. B.2 Lane-Changing Game Model [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Lane￾Changing Game The game has a total N number of drivers, out of k number of drivers en￾tering the game every step. Agents have a target θ ∈ {Target 1, Target 2} they have to reach which is private information to them. The agents that enter the game start from the …
Figure 13
Figure 13. Figure 13: Left: 3-Destination Congestion Game. Right: Intersection Congestion Game. [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 42 canonical work pages

  1. [1]

    Pure nash equilibria and best-response dynamics in random games

    Ben Amiet, Andrea Collevecchio, Marco Scarsini, and Ziwen Zhong. Pure nash equilibria and best-response dynamics in random games. Mathematics of Operations Research, 46 0 (4): 0 1552--1572, 2021

    work page 2021

  2. [2]

    Dynamic mechanism design: An introduction

    Dirk Bergemann and Juuso V \"a lim \"a ki. Dynamic mechanism design: An introduction. Journal of Economic Literature, 57 0 (2): 0 235--274, 2019

    work page 2019

  3. [3]

    Evolutionary dynamics of multi-agent learning: A survey

    Daan Bloembergen, Karl Tuyls, Daniel Hennes, and Michael Kaisers. Evolutionary dynamics of multi-agent learning: A survey. Journal of Artificial Intelligence Research, 53: 0 659--697, 2015

    work page 2015

  4. [4]

    Convex optimization

    Stephen P Boyd and Lieven Vandenberghe. Convex optimization. Cambridge University Press, 2004

    2004

  5. [5]

    On equilibrium in pure strategies in games with many players

    Edward Cartwright and Myrna Wooders. On equilibrium in pure strategies in games with many players. International Journal of Game Theory, 38: 0 137--153, 2009

    work page 2009

  6. [6]

    Multiagent learning in the presence of memory-bounded agents

    Doran Chakraborty and Peter Stone. Multiagent learning in the presence of memory-bounded agents. Autonomous agents and multi-agent systems, 28: 0 182--213, 2014

    work page 2014

  7. [7]

    Mechanism design for facility location problems: a survey

    Hau Chan, Aris Filos-Ratsikas, Bo Li, Minming Li, and Chenhao Wang. Mechanism design for facility location problems: a survey. arXiv preprint arXiv:2106.03457, 2021

    arXiv 2021

  8. [8]

    Automated mechanism design for a self-interested designer

    Vincent Conitzer and Tuomas Sandholm. Automated mechanism design for a self-interested designer. In ACM conference on Electronic commerce, pages 232--233, 2003

    work page 2003

Show all 55 references
  1. [9]

    Awesome: A general multiagent learning algorithm that converges in self-play and learns a best response against stationary opponents

    Vincent Conitzer and Tuomas Sandholm. Awesome: A general multiagent learning algorithm that converges in self-play and learns a best response against stationary opponents. Machine Learning, 67: 0 23--43, 2007

    2007

  2. [10]

    Privacy and truthful equilibrium selection for aggregative games

    Rachel Cummings, Michael Kearns, Aaron Roth, and Zhiwei Steven Wu. Privacy and truthful equilibrium selection for aggregative games. In Web and Internet Economics: 11th International Conference, WINE 2015, Amsterdam, The Netherlands, December 9-12, 2015, Proceedings 11, pages ...

    2015

  3. [11]

    Differentiable economics for randomized affine maximizer auctions

    Michael Curry, Tuomas Sandholm, and John Dickerson. Differentiable economics for randomized affine maximizer auctions. In International Joint Conference on Artificial Intelligence, 2023

    2023

  4. [12]

    Optimal auctions through deep learning

    Paul D \"u tting, Zhe Feng, Harikrishna Narasimhan, David Parkes, and Sai Srivatsa Ravindranath. Optimal auctions through deep learning. In International Conference on Machine Learning, pages 1706--1715, 2019

    2019

  5. [13]

    Game theory

    Drew Fudenberg. Game theory. MIT press, 1991

    1991

  6. [14]

    Addressing function approximation error in actor-critic methods, 2018

    Scott Fujimoto, Herke van Hoof, and David Meger. Addressing function approximation error in actor-critic methods, 2018

    2018

  7. [15]

    Online mechanism design for electric vehicle charging

    Enrico H Gerding, Valentin Robu, Sebastian Stein, David C Parkes, Alex Rogers, and Nicholas R Jennings. Online mechanism design for electric vehicle charging. In International Conference on Autonomous Agents and Multiagent Systems, 2011

    2011

  8. [16]

    Deep learning for multi-facility location mechanism design

    Noah Golowich, Harikrishna Narasimhan, and David C Parkes. Deep learning for multi-facility location mechanism design. In IJCAI, pages 261--267, 2018

    2018

  9. [17]

    Generative adversarial networks

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63 0 (11): 0 139--144, 2020

    2020

  10. [18]

    Automated online mechanism design and prophet inequalities

    Mohammad Taghi Hajiaghayi, Robert Kleinberg, and Tuomas Sandholm. Automated online mechanism design and prophet inequalities. In AAAI, volume 7, pages 58--65, 2007

    2007

  11. [19]

    Multiagent reinforcement learning: theoretical framework and an algorithm

    Junling Hu, Michael P Wellman, et al. Multiagent reinforcement learning: theoretical framework and an algorithm. In ICML, volume 98, pages 242--250, 1998

    1998

  12. [20]

    A simple, fast, and safe mediator for congestion management

    Kei Ikegami, Kyohei Okumura, and Takumi Yoshikawa. A simple, fast, and safe mediator for congestion management. In AAAI Conference on Artificial Intelligence, pages 2030--2037, 2020

    2020

  13. [21]

    Mediated multi-agent reinforcement learning

    Dmitry Ivanov, Ilya Zisman, and Kirill Chernyshev. Mediated multi-agent reinforcement learning. In International Conference on Autonomous Agents and Multiagent Systems, 2023

    2023

  14. [22]

    Categorical reparameterization with gumbel-softmax, 2017

    Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax, 2017

    2017

  15. [23]

    Mechanism design in large games: incentives and privacy

    Michael Kearns, Mallesh Pai, Aaron Roth, and Jonathan Ullman. Mechanism design in large games: incentives and privacy. In Conference on Innovations in Theoretical Computer Science, pages 403--410, 2014

    2014

  16. [24]

    Robust mediators in large games

    Michael Kearns, Mallesh M Pai, Ryan Rogers, Aaron Roth, and Jonathan Ullman. Robust mediators in large games. arXiv preprint arXiv:1512.02698, 2015

    2015 arXiv

  17. [25]

    Zero tolerance for bias

    Terence Kelly. Zero tolerance for bias. Queue, 22 0 (2): 0 19--38, 2024

    2024

  18. [26]

    JAXRL: Implementations of Reinforcement Learning algorithms in JAX , 10 2021

    Ilya Kostrikov. JAXRL: Implementations of Reinforcement Learning algorithms in JAX , 10 2021. URL https://github.com/ikostrikov/jaxrl

    2021

  19. [27]

    End-to-end training of deep visuomotor policies

    Sergey Levine, Chelsea Finn, Trevor Darrell, and Pieter Abbeel. End-to-end training of deep visuomotor policies. Journal of Machine Learning Research, 17 0 (39): 0 1--40, 2016

    2016

  20. [28]

    Continuous control with deep reinforcement learning

    Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971, 2015

    2015 arXiv

  21. [29]

    Lillicrap, Jonathan J

    Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning, 2019

    2019

  22. [30]

    Multi-agent actor-critic for mixed cooperative-competitive environments

    Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. Multi-agent actor-critic for mixed cooperative-competitive environments. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, page 6382–6393, Red Hook, ...

    2017

  23. [31]

    Independent reinforcement learners in cooperative markov games: A survey regarding coordination problems

    Laetitia Matignon, Guillaume Laurent, and Nadine Fort-Piat. Independent reinforcement learners in cooperative markov games: A survey regarding coordination problems. The Knowledge Engineering Review, 27: 0 1 -- 31, 03 2012. doi:10.1017/S0269888912000057

    2012 doi

  24. [32]

    Congestion games with player-specific payoff functions

    Igal Milchtaich. Congestion games with player-specific payoff functions. Games and Economic Behavior, 13 0 (1): 0 111--124, 1996

    1996

  25. [33]

    Playing atari with deep reinforcement learning, 2013

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning, 2013

    2013

  26. [34]

    Strong mediated equilibrium

    Dov Monderer and Moshe Tennenholtz. Strong mediated equilibrium. Artificial Intelligence, 173 0 (1): 0 180--195, 2009

    2009

  27. [35]

    Optimal coordination mechanisms in generalized principal--agent problems

    Roger B Myerson. Optimal coordination mechanisms in generalized principal--agent problems. Journal of Mathematical Economics, 10 0 (1): 0 67--81, 1982

    1982

  28. [36]

    A Course in Game Theory

    Martin J Osborne and Ariel Rubinstein. A Course in Game Theory. MIT Press, 1994

    1994

  29. [37]

    An mdp-based approach to online mechanism design

    David C Parkes and Satinder Singh. An mdp-based approach to online mechanism design. Advances in neural information processing systems, 16, 2003

    2003

  30. [38]

    New criteria and a new algorithm for learning in multi-agent systems

    Rob Powers and Yoav Shoham. New criteria and a new algorithm for learning in multi-agent systems. Advances in neural information processing systems, 17, 2004

    2004

  31. [39]

    Approximate mechanism design without money

    Ariel D Procaccia and Moshe Tennenholtz. Approximate mechanism design without money. ACM Transactions on Economics and Computation (TEAC), 1 0 (4): 0 1--26, 2013

    2013

  32. [40]

    Marketplaces, markets, and market design

    Alvin E Roth. Marketplaces, markets, and market design. American Economic Review, 108 0 (7): 0 1609--1658, 2018

    2018

  33. [41]

    Chapter 18: Routing games

    Tim Roughgarden. Chapter 18: Routing games. In Algorithmic Game Theory. Cambridge University Press, 2007. URL https://api.semanticscholar.org/CorpusID:17967837

    2007

  34. [42]

    Perspectives on multiagent learning

    Tuomas Sandholm. Perspectives on multiagent learning. Artificial Intelligence, 171 0 (7): 0 382--391, 2007

    2007

  35. [43]

    Automated mechanism design via neural networks

    Weiran Shen, Pingzhong Tang, and Song Zuo. Automated mechanism design via neural networks. arXiv preprint arXiv:1805.03382, 2018

    2018 arXiv

  36. [44]

    If multi-agent learning is the answer, what is the question? Artificial intelligence, 171 0 (7): 0 365--377, 2007

    Yoav Shoham, Rob Powers, and Trond Grenager. If multi-agent learning is the answer, what is the question? Artificial intelligence, 171 0 (7): 0 365--377, 2007

    2007

  37. [45]

    Leibo, Karl Tuyls, and Thore Graepel

    Peter Sunehag, Guy Lever, Audrunas Gruslys, Wojciech Marian Czarnecki, Vinicius Zambaldi, Max Jaderberg, Marc Lanctot, Nicolas Sonnerat, Joel Z. Leibo, Karl Tuyls, and Thore Graepel. Value-decomposition networks for cooperative multi-agent learning, 2017

    2017

  38. [46]

    Multiagent cooperation and competition with deep reinforcement learning, 2015

    Ardi Tampuu, Tambet Matiisen, Dorian Kodelja, Ilya Kuzovkin, Kristjan Korjus, Juhan Aru, Jaan Aru, and Raul Vicente. Multiagent cooperation and competition with deep reinforcement learning, 2015

    2015

  39. [47]

    Multi-agent reinforcement learning: Independent versus cooperative agents

    Ming Tan. Multi-agent reinforcement learning: Independent versus cooperative agents. In Proceedings of the Tenth International Conference on Machine Learning (ICML 1993), pages 330--337, San Francisco, CA, USA, 1993. Morgan Kauffman. ISBN 1-55860-307-7. URL http://web.media.mi...

    1993

  40. [48]

    Of mechanism design and multiagent planning

    Roman Van Der Krogt, Mathijs De Weerdt, and Yingqian Zhang. Of mechanism design and multiagent planning. In ECAI, pages 423--427, 2008

    2008

  41. [49]

    Empirical mechanism design: Methods, with application to a supply-chain scenario

    Yevgeniy Vorobeychik, Christopher Kiekintveld, and Michael P Wellman. Empirical mechanism design: Methods, with application to a supply-chain scenario. In ACM conference on Electronic commerce, pages 306--315, 2006

    2006

  42. [50]

    Constrained automated mechanism design for infinite games of incomplete information

    Yevgeniy Vorobeychik, Daniel M Reeves, and Michael P Wellman. Constrained automated mechanism design for infinite games of incomplete information. Autonomous Agents and Multi-Agent Systems, 25: 0 313--351, 2012

    2012

  43. [51]

    The handbook of market design

    Nir Vulkan, Alvin E Roth, and Zvika Neeman. The handbook of market design. OUP Oxford, 2013

    2013

  44. [52]

    Deep contract design via discontinuous networks

    Tonghan Wang, Paul Duetting, Dmitry Ivanov, Inbal Talgam-Cohen, and David C Parkes. Deep contract design via discontinuous networks. Advances in Neural Information Processing Systems, 36, 2024

    2024

  45. [53]

    Polynomial-time optimal equilibria with a mediator in extensive-form games

    Brian Zhang and Tuomas Sandholm. Polynomial-time optimal equilibria with a mediator in extensive-form games. In Neural Information Processing Systems, pages 24851--24863, 2022

    2022

  46. [54]

    Computing optimal equilibria and mechanisms via learning in zero-sum extensive-form games

    Brian Zhang, Gabriele Farina, Ioannis Anagnostides, Federico Cacciamani, Stephen McAleer, Andreas Haupt, Andrea Celli, Nicola Gatti, Vincent Conitzer, and Tuomas Sandholm. Computing optimal equilibria and mechanisms via learning in zero-sum extensive-form games. In Neural Info...

    2023

  47. [55]

    Automated dynamic mechanism design

    Hanrui Zhang and Vincent Conitzer. Automated dynamic mechanism design. Advances in Neural Information Processing Systems, 34: 0 27785--27797, 2021

    2021

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.