REVIEW 3 major objections 5 minor 36 references
Linear-Quadratic Discrete-Time Dynamic Games with Unknown Dynamics
T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A linear-quadratic dynamic game can be solved for its feedback Nash equilibria using only offline input/output data, with the same equilibria as the known-dynamics game and a convergent rolling-horizon approximation in the…
desk verdict Genuinely new data-driven FNE characterization for finite-horizon LQ games with unknown i/o/s dynamics, but Theorem 3.3's equivalence claim is not proven as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The argument is carried by the behavioral-system representation of the unknown plant: instead of identifying the matrices $A,B,C,D$, the paper treats the set of possible input/output trajectories as a behavior and uses the Hankel matrix of offline trajectories to span that behavior (Lemma 2.3). Each stage's optimization is rewritten in terms of the data-only matrix $G_t = Y_{f_t} M_t^\dagger$, which predicts the current output from the historical data vector $U_t$; the equilibrium conditions then become linear equations in the feedback matrices $K_t^i$, grouped into the system $\tilde H_t(P_{t+1})K_t = \tilde g_t(P_{t+1})$. The invertibility of $\tilde H_t$ is the mechanism that turns a game equilibrium into a backward recursion of linear solves whose coefficients depend only on offline data and cost parameters.
What would settle it
Convertible check: take the two-player scalar example of Section 5, compute $K_i^*(T)$ by Algorithm 3.1 for increasing $T$, and compare the achieved infinite-horizon cost to the known-dynamics feedback-Nash-equilibrium cost; if the cost difference fails to vanish at the claimed quadratic rate, or if two offline datasets satisfying Assumption 3.1 produce different equilibrium strategies, the equivalence claim fails. More directly, finding any known-dynamics linear-quadratic game whose unique current-state feedback Nash equilibrium differs from a history-dependent equilibrium over the same feasible trajectory set would refute Theorem 3.3.
Extended reading notes
Core claim
The central claim is Theorem 3.3: for the discrete-time linear-quadratic game with unknown input/output/state dynamics, given initial data of length at least the system lag and offline trajectories whose Hankel matrix has full row rank, the finite-horizon game has the same set of feedback Nash equilibria as the same game with known dynamics, including the possibility that both sets are empty. Moreover every equilibrium strategy is affine in the observed history, $u_t^{i*} = K_t^{i*} U_{t-1} + L_t^{i*}$, and the equilibrium value is quadratic in the initial data; the coefficients are fixed by the offline data and the cost parameters. The paper then derives data-only coupled equations (1-1)--(1-5) that are necessary and sufficient for existence and uniqueness of the feedback Nash equilibrium, and shows that when each stage's coefficient matrix $\tilde H_t(P_{t+1})$ is invertible, the equilibrium is computed by backward linear solves. In the infinite-horizon case (Theorem 4.2), under controllability plus Assumption 1 of the companion paper, the rolling finite-horizon strategy converges in total cost to the known-dynamics feedback-Nash-equilibrium cost, with error $O(\max_{i\in\mathcal{N}}\|K_i^*(T_i)-K_i^*\|^2)$.
Load-bearing premise
The finite-horizon equivalence rests on treating equality of the feasible input-output trajectory sets as sufficient for equality of the equilibrium strategy sets, even though the known-dynamics game restricts strategies to current-state feedback while the unknown-dynamics game allows strategies on the full history; the infinite-horizon convergence additionally rests on controllability and the unstated Assumption 1 of the companion paper.
Editorial extensions
If this is right
- Players with no knowledge of the dynamics or the state can implement feedback-Nash-equilibrium strategies immediately from offline data, with no system identification step.
- Knowing the system dynamics confers no strategic advantage: if one player knows the dynamics and another does not, both can achieve the same equilibrium performance given sufficient offline data.
- Under the invertibility condition, computing a feedback Nash equilibrium reduces to solving $T$ sets of linear equations backward, avoiding coupled high-order matrix equations.
- In the infinite-horizon unknown-dynamics game, the rolling-horizon strategy makes each player's total cost converge to the known-dynamics infinite-horizon equilibrium cost as the minimum prediction horizon grows, with error at most quadratic in the feedback-matrix error.
- The same data-driven equations give necessary and sufficient conditions for existence and uniqueness of the feedback Nash equilibrium: if no solution to (1-1)--(1-5) exists, no equilibrium exists, and uniqueness is characterized by uniqueness of the strategy maps.
Reading between the lines
- An implicit consequence is that any dataset satisfying the rank condition carries enough information for equilibrium play, regardless of how the inputs were excited; this suggests testing whether weaker rank conditions on the data matrix would also suffice.
- The convergence result implies that players need not agree on a common prediction horizon: heterogeneous $T_i$ still yield convergence for every player, which may extend to online receding-horizon play and to noisy data through set-membership estimates.
- The affine-in-history form suggests that history-dependent strategies in unknown-dynamics games are not artifacts but can reproduce current-state feedback equilibria, raising the testable question of whether the same data-only technique extends to games with coupled control costs or partial observations.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies N-player discrete-time linear-quadratic dynamic games in which the underlying input/output/state (i/o/s) dynamics and the state trajectory are unknown, and players have access only to an offline input/output dataset and an initial trajectory. The main finite-horizon results are: (Theorem 3.3) the set of feedback Nash equilibria of the unknown-dynamics game coincides with that of the corresponding known-dynamics game; (Theorem 3.7) necessary and sufficient conditions for existence and uniqueness of FNE in terms of coupled data-dependent equations; and (Theorem 3.8) an invertibility condition under which one FNE is obtained by solving linear equations backward. For the infinite-horizon game, Theorem 4.2 claims, under controllability and an assumption from the companion paper [11], that the 'watching T steps into the future and moving one step now' strategy yields convergence of feedback matrices and total costs, with rate O(max_i ||K_i^*(T_i)-K_i^*||^2). A two-player non-scalar numerical example illustrates the convergence behavior.
Significance. If the equivalence theorem were established, the paper would provide a genuinely data-driven approach to LQ dynamic games without system identification or online exploration, with finite-horizon characterizations and an explicit computational algorithm. The strengths include the behavioral-system formulation, the affine-form derivation for history-dependent equilibria, the invertibility-based linear-equation algorithm, and the non-scalar numerical demonstration. However, the central equivalence claim currently rests on an information-structure gap, and the infinite-horizon convergence rate is not supported by the proof as written. The paper is therefore of interest to the data-driven control and dynamic-game communities, but it requires substantial revision before the claims can be accepted.
major comments (3)
- [Theorem 3.3 and Appendix A(b)] The proof of Theorem 3.3 claims that equality of the feasible input-output trajectory sets is sufficient for equality of the FNE sets because the cost functions are the same in both settings. This is not sufficient: Definition 2.1 restricts players to current-state feedback strategies, whereas Definition 3.2 allows strategies that depend on the full history U_{t-1}. Equality of feasible trajectory sets says nothing about best-response correspondences or subgame-perfect behavior under these different information structures. The affine-form result in part (a) narrows the unknown-game strategy space, but it does not show that every affine history-dependent equilibrium is realizable by a state-feedback equilibrium, nor that no additional history-dependent equilibria exist. This gap is load-bearing for Theorem 3.3, Remark 3.4, and the applications in Theorem 4.2.
- [Theorem 4.2(3) and Appendix D] Theorem 4.2(3) states that \tilde{J}^i - J^i = O(max_i ||K_i^*(T_i)-K_i^*||^2), but the proof's final displayed inequality is |\tilde{J}^i - J^i| <= C max_i ||K_i^*(T_i)-K_i^*||_2, which is linear in the matrix error. The squared rate is therefore not established and is contradicted by the proof as written. The rate must be corrected to O(epsilon) or a separate argument for the quadratic rate must be supplied.
- [Theorem 4.2 and Assumption 1 of [11]] Theorem 4.2 and its proof depend on Assumption 1, Proposition 1, Lemma 2, and Theorem 1 of the companion paper [11], but Assumption 1 is not stated or summarized in this manuscript. Because the convergence claims for the infinite-horizon game are entirely conditional on these unstated premises, the statement is not self-contained and the applicability of those conditions to the present unknown-dynamics setting is not verifiable from the text. Please state the assumption explicitly and confirm that the cited results apply to the games defined here.
minor comments (5)
- [Notation, Section 2.2 and after Theorem 3.7] The symbol B is used both for the behavioral system and for the basis matrix B(B_{Tini}); please use distinct notation, such as \mathcal{B} for the system, to avoid confusion.
- [Equation (1-1a)] The matrix B(B_{Tini}) appears in equation (1-1a) before it is formally introduced in the notation block after Theorem 3.7; please move the definition earlier or add a forward reference.
- [Remark 3.4] Remark 3.4 contains the identity L_i^*_t = L_i^*_t, which appears to be a typo; presumably one side should refer to the known-dynamics constant term and the other to the unknown-dynamics constant term.
- [Figure 1 caption] The caption states 't = 1,...,100' while the horizontal axis of the figure is T = 1,...,50; please clarify which quantity varies in the plot.
- [Section 5, Algorithm 4.1] Algorithm 4.1 uses the same symbol J~i* both as an accumulated sum and as the output total cost; please distinguish the iteration variable from the final output for clarity.
Circularity Check
Infinite-horizon convergence is imported from the authors' companion paper [11]; the finite-horizon data-driven results are self-contained and not circular.
-
self citation load bearing
[Section 4, Theorem 4.2 statement and its proof in Appendix D]
"For the known-dynamics game (2.1, 4.1), suppose that system (2.1) is controllable, Assumption 1 in [11] holds. ... Since Assumption 1 in [11] holds, it follows from Proposition 1 in [11] that the T -stage known-dynamics game has a unique FNE, for T = 1, 2, ... Since Theorem 1 in [11] holds, we have lim_{Tm→+∞} J˜ i(x1) = J i for all i ∈ N."
Theorem 4.2 is the paper's central infinite-horizon claim, but its analytical content is not derived here: the proof imports from [11] (a companion paper by the same authors, Huang, Yang, Mu, Mei) the existence/uniqueness of finite-horizon known-dynamics FNEs (Proposition 1), the convergence of the finite-horizon feedback matrices (Lemma 2), and the cost-convergence theorem for the rolling strategy (Theorem 1). The present paper only supplies the mapping between known- and unknown-dynamics feedback matrices via Theorem 3.3 and Remark 3.4. Thus the main convergence result rests on an unverified self-citation chain rather than on a self-contained derivation. This is not a definitional reduction, so it is scored as load-bearing self-citation rather than full circularity.
full rationale
The finite-horizon part of the paper is genuinely self-contained: Theorem 3.3's affine-form derivation and the coupled equations (1-1)-(1-5) are built from offline data Hankel matrices and the players' cost parameters, with no fitted parameter later renamed as a prediction. The finite-horizon equivalence claim has a proof gap flagged by the skeptic: Appendix A(b) asserts that equality of feasible sets suffices to equate FNEs despite the different information structures (state feedback versus history feedback). That is a correctness or rigor concern, not circularity, because the assertion is not equivalent to its inputs by construction. The principal circularity-adjacent issue is Section 4: the infinite-horizon convergence theorem is delegated to the authors' own companion paper [11], including an unstated Assumption 1, and the proof in Appendix D repeatedly invokes Proposition 1, Lemma 2, and Theorem 1 of [11]. Because that cited result is load-bearing for the advertised infinite-horizon convergence and is not reproduced or independently verified in this manuscript, the circularity score is moderately elevated. The finite-horizon results provide substantial independent content, so a score of 4 is appropriate rather than 6 or higher.
Assumptions & free parameters
free parameters (2)
- epsilon =
0.01
- M =
1000
assumptions (6)
- standard math Behavioral fundamental lemma: if rank(H_L(Wd)) = mL+n and L > l(B), then image(H_L(Wd)) = B_L.
- domain assumption Initial data length satisfies Tini >= l(A,C), so the initial state is uniquely determined by past input/output data (Lemma 2.2).
- domain assumption Offline data satisfy the full-rank/persistence-of-excitation condition rank(H_L(Wd)) = mL+n for the horizons used.
- domain assumption The true plant is an exact noiseless LTI system (2.1), and all objective parameters Qi, Rij, delta_i, l_i^t are known to all players.
- domain assumption Infinite-horizon analysis: system (2.1) is controllable and Assumption 1 in the companion paper [11] holds.
- domain assumption Infinite-horizon unknown-dynamics FNE is restricted to stationary linear strategies ui_t = K^i U0(t).
Cite this review
Pith. "Pith review of Linear-Quadratic Discrete-Time Dynamic Games with Unknown Dynamics." pith.science (2026). https://pith.science/paper/SLGOIRWE
@misc{pith2026250622073,
author = {Pith},
title = {Pith review of: Linear-Quadratic Discrete-Time Dynamic Games with Unknown Dynamics},
year = {2026},
howpublished = {\url{https://pith.science/paper/SLGOIRWE}},
note = {Machine review of arXiv:2506.22073}
}
abstract
Considering linear-quadratic discrete-time games with unknown input/output/state (i/o/s) dynamics and state, we provide necessary and sufficient conditions for the existence and uniqueness of feedback Nash equilibria (FNE) in the finite-horizon game, based entirely on offline input/output data. We prove that the finite-horizon unknown-dynamics game and its corresponding known-dynamics game have the same FNEs, and provide detailed relationships between their respective FNE matrices. To simplify the computation of FNEs, we provide an invertibility condition and a corresponding algorithm that computes one FNE by solving a finite number of linear equation systems using offline data. For the infinite-horizon unknown-dynamics game, limited offline data restricts players to computing optimal strategies only over a finite horizon. We prove that the finite-horizon strategy ``watching $T$ steps into the future and moving one step now,'' which is commonly used in classical optimal control, exhibits convergence in both the FNE matrices and the total costs in the infinite-horizon unknown-dynamics game, and further provide an analysis of the convergence rate of the total cost. The corresponding algorithm for the infinite-horizon game is proposed and its efficacy is demonstrated through a non-scalar numerical example.
Reference graph
Works this paper leans on
-
[11]
S. Huang, X. Yang, Y. Mu, and W. Mei , Finite-horizon strategy in infinite-horizon linear- quadratic discrete-time dynamic games , 2025, https://doi.org/10.48550/arXiv.2506.19565
work page Pith review arXiv doi:10.48550/arxiv.2506.19565 2025
-
[1]
V. Aguirregabiria and A. Magesan , Identification and estimation of dynamic games when players’ beliefs are not in equilibrium , Rev. Econ. Stud., 87 (2020), pp. 582–625, https: //doi.org/10.1093/restud/rdz013
-
[2]
M. Aoyagi, G. R. Fr ´echette, and S. Yuksel , Beliefs in repeated games: An experiment , Am. Econ. Rev., 114 (2024), pp. 3944–3975, https://doi.org/10.1257/aer.20220639
-
[3]
T. Bas ¸ar and G. J. Olsder, Dynamic Noncooperative Game Theory, 2nd Edition , Society for Industrial and Applied Mathematics, 1998, https://doi.org/10.1137/1.9781611971132
-
[4]
J. Coulson, J. Lygeros, and F. D ¨orfler, Data-enabled predictive control: In the shallows of the deepc, in 2019 18th European Control Conference (ECC), 2019, pp. 307–312, https: //doi.org/10.23919/ECC.2019.8795639
-
[5]
F. Delarue and A. V asileiadis , Exploration noise for learning linear-quadratic mean field games, Math. Oper. Res., (2024), https://doi.org/10.1287/moor.2021.0157
arXiv 2024
-
[6]
J. C. Ely, G. Georgiadis, and L. Rayo , Feedback design in dynamic moral hazard , Econo- metrica, 93 (2025), pp. 597–621, https://doi.org/10.3982/ECTA21871
-
[7]
L. Gao, D. Jian, M. Gumus, and B. K. Mishra , Managing channel profits with positive demand externalities , Manage. Sci., (2025)
work page 2025
Show all 36 references
-
[8]
Haarnoja, B
T. Haarnoja, B. Moran, G. Lever, S. H. Huang, D. Tirumala, J. Humplik, and N. Heess, Learning agile soccer skills for a bipedal robot with deep reinforcement learning, Sci. Robot., 9 (2024), p. eadi8022, https://doi.org/10.1126/scirobotics.adi8022
2024 doi
-
[9]
Haghtalab, M
N. Haghtalab, M. Jordan, and E. Zhao , A unifying perspective on multi- calibration: Game dynamics for multi-objective learning , in Proceedings of the Advances in Neural Information Processing Systems, vol. 36, NeurIPS, Decem- ber 2023, pp. 72464–72506, https://proceedings.ne...
2023
-
[10]
L. Han, Q. Zhu, J. Sheng, C. Zhang, T. Li, Y. Zhang, and Z. Zhang , Lifelike agility and play in quadrupedal robots using reinforcement learning and generative pre-trained models, Nat. Mach. Intell., 6 (2024), pp. 787–798, https://doi.org/10.1038/s42256-024-00861-3. 24 S. HUAN...
2024 doi
-
[12]
M. W. Ketelaars, P. Borm, and P. M. Kort , Dynamic stability of cooperative investment under uncertainty, Oper. Res., (2025), https://doi.org/10.1287/opre.2023.0488
2025
-
[13]
Kydland , Decentralized stabilization policies: Optimization and the assignment problem , Ann
F. Kydland , Decentralized stabilization policies: Optimization and the assignment problem , Ann. Econ. Soc. Meas., 5 (1976), pp. 249–261, http://www.nber.org/chapters/c10445
1976
-
[14]
Laine, D
F. Laine, D. Fridovich-Keil, C. Y. Chiu, and C. Tomlin , The computation of approximate generalized feedback nash equilibria, SIAM J. Optim., 33 (2023), pp. 294–318, https://doi. org/10.1137/21M142530
2023 doi
-
[15]
J. Li, Z. Xiao, and P. Li , Discrete-time multi-player games based on off-policy q- learning, IEEE Access, 7 (2019), pp. 134647–134659, https://doi.org/10.1109/ACCESS. 2019.2939384
2019
-
[16]
Y. Li, G. Carboni, F. Gonzalez, D. Campolo, and E. Burdet , Differential game theory for versatile physical human–robot interaction , Nat. Mach. Intell., 1 (2019), pp. 36–43, https://doi.org/10.1038/s42256-018-0010-3
2019 doi
-
[17]
G. H. Liu, T. Chen, and E. Theodorou , Dynamic game theoretic neural optimizer , in Pro- ceedings of the International Conference on Machine Learning, PMLR, July 2021, pp. 6759– 6769, https://proceedings.mlr.press/v139/liu21d.html
2021
-
[18]
Liu and L
N. Liu and L. Guo , Stochastic adaptive linear quadratic differential games , IEEE Trans. Automat. Control, 69 (2023), pp. 1066–1073, https://doi.org/10.1109/TAC.2023.3274863
2023
-
[19]
C. Loch, K. Ladas, and S. Kavadias , Organizational culture, innovation, and competitive performance: A multilevel dynamic model , Manage. Sci., (2025), https://doi.org/10.1287/ mnsc.2021.00008
2025
-
[20]
S. L v, J. Xiong, and X. Zhang, Linear quadratic leader–follower stochastic differential games for mean-field switching diffusions , Automatica, 154 (2023), p. 111072, https://doi.org/ 10.1016/j.automatica.2023.111072
2023
-
[21]
Markovsky and F
I. Markovsky and F. D¨orfler, Identifiability in the behavioral setting, IEEE Trans. Automat. Control, 68 (2022), pp. 1667 – 1677, https://doi.org/10.1109/TAC.2022.3209954
2022
-
[22]
Markovsky and P
I. Markovsky and P. Rapisarda , Data-driven simulation and control , Int. J. Control, 81 (2008), p. 1946–1959, https://doi.org/10.1080/00207170801942170
2008 doi
-
[23]
McClellan, Dynamic outside options and optimal negotiation strategies , Am
A. McClellan, Dynamic outside options and optimal negotiation strategies , Am. Econ. Rev., 114 (2024), pp. 3284–3313, https://doi.org/10.1257/aer.20230015
2024 doi
-
[24]
Y. Meng, S. P. Cornelius, Y. Y. Liu, and A. Li , Dynamics of collective cooperation under personalised strategy updates, Nat. Commun., 15 (2024), p. 3125, https://doi.org/10.1038/ s41467-024-47380-8
2024
-
[25]
Monti, B
A. Monti, B. Nortmann, T. Mylvaganam, and M. Sassano , Feedback and open-loop nash equilibria for lq infinite-horizon discrete-time dynamic games , SIAM J. Control Optim., 62 (2024), pp. 1417–1436, https://doi.org/10.1137/23M1579960
2024 doi
-
[26]
Mylvaganam, M
T. Mylvaganam, M. Sassano, and A. Astolfi , Constructive ϵ-nash equilibria for nonzero- sum differential games , IEEE Trans. Automat. Control, 60 (2014), pp. 950–965, https: //doi.org/10.1109/TAC.2014.2362334
2014
-
[27]
Nortmann, A
B. Nortmann, A. Monti, M. Sassano, and T. Mylvaganam , Nash equilibria for linear quadratic discrete-time dynamic games via iterative and data-driven algorithms , IEEE Trans. Automat. Control, 69 (2024), pp. 6561–6575, https://doi.org/10.1109/TAC.2024. 3375249
2024 doi
-
[28]
Nortmann and T
B. Nortmann and T. Mylvaganam, Approximate nash equilibria for discrete-time linear qua- dratic dynamic games , IF AC Pap. Online, 56 (2023), pp. 1760–1765, https://doi.org/10. 1016/j.ifacol.2023.10.1886
2023
-
[29]
Nortmann, M
B. Nortmann, M. Sassano, and T. Mylvaganam, Feedback nash equilibria for scalar n-player linear quadratic dynamic games , Automatica, 174 (2025), p. 112133, https://doi.org/10. 1016/j.automatica.2025.112133
2025
-
[30]
C. S. Rossetti, O. P. Hauser, and C. Hilbe , Dynamics of cooperation in concurrent games , Nat. Commun., 16 (2025), p. 1524, https://doi.org/10.1038/s41467-025-56083-7
2025 doi
-
[31]
R. Song, Q. Wei, H. Zhang, and F. L. Lewis , Discrete-time non-zero-sum games with completely unknown dynamics , IEEE Trans. Cybern., 51 (2019), pp. 2929–2943, https: //doi.org/10.1109/TCYB.2019.2957406
2019
-
[32]
Sun, Two-person zero-sum stochastic linear-quadratic differential games , SIAM J
J. Sun, Two-person zero-sum stochastic linear-quadratic differential games , SIAM J. Control Optim., 59 (2021), pp. 1804–1829, https://doi.org/10.1137/20M1340368
2021 doi
-
[33]
J. Sun, H. W ang, and J. Wen , Zero-sum stackelberg stochastic linear-quadratic differen- tial games , SIAM J. Control Optim., 61 (2023), pp. 252–284, https://doi.org/10.1137/ 21M1450458
2023
-
[34]
W ang, M
C. W ang, M. Perc, and A. Szolnoki , Evolutionary dynamics of any multiplayer LQ DISCRETE-TIME DYNAMIC GAMES WITH UNKNOWN DYNAMICS 25 game on regular graphs , Nat. Commun., 15 (2024), p. 5349, https://doi.org/10.1038/ s41467-024-49505-5
2024
-
[35]
J. C. Willems, From time series to linear system—part i. finite dimensional linear time invari- ant systems , Automatica, 22 (1986), pp. 561–580, https://doi.org/10.1016/0005-1098(86) 90066-X
1986 doi
-
[36]
Y. Yang, S. Zhang, J. Dong, and Y. Yin , Data-driven nonzero-sum game for discrete-time systems using off-policy reinforcement learning , IEEE Access, 8 (2019), pp. 14074–14088, https://doi.org/10.1109/ACCESS.2019.2960064
2019
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.