Towards An Analytical Framework for Dynamic Potential Games

Potential game is an emerging notion and framework for studying N-player games, especially with heterogeneous players. In this paper, we build an analytical framework for dynamic potential games. We prove that a game is a dynamic potential game if and only if each player's value function can be decomposed as a potential function and a residual term which is solely dependent on other players' policies. This decomposition is consistent with the result in the static setting and enables us to identify and analyze an important and new class of dynamic potential games called the distributed game. Moreover, we prove that a game is a dynamic potential game if the value function has a symmetric Jacobian. This generalizes the differential characterization for static potential games by replacing the classical derivative with a new notation of functional derivative with respect to Markov policies. For a general class of continuous-time stochastic games, we explicitly characterize their potential functions. In particular, we show that the potential function of linear-quadratic games can be studied through a system of linear ODEs. Furthermore, under a rank condition on control coefficients, we prove a linear-quadratic game is a Markov potential game if and only if all players have identical cost functions.