Bridging Finite and Infinite-Horizon Nash Equilibria in Linear Quadratic Games

Giulio Salizzoni; Maryam Kamgarpour; Sophie Hall

REVIEW 3 major objections 5 minor 24 references

Bridging Finite and Infinite-Horizon Nash Equilibria in Linear Quadratic Games

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Riccati fixed points equal all stationary Nash equilibria of infinite-horizon LQ games.

desk verdict Clean fixed-point bridge between finite and infinite-horizon LQ Nash equilibria; the cycle-stability lemma has a patchable missing complex-eigenvalue case. read the letter →

arxiv 2508.20675 v1 pith:DHZV76CQ submitted 2025-08-28 cs.MA cs.SYeess.SYmath.DS

Giulio Salizzoni , Sophie Hall , Maryam Kamgarpour This is my paper · ORCID

classification cs.MAcs.SYeess.SYmath.DS MSC 91A2593C5549N10

keywords linearquadraticgamesNashequilibriaRiccatirecursionfixedpointsperiodicterminalcostselectionreceding-horizondynamicalsystems

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper bridges finite-horizon linear quadratic games, which have a unique Nash equilibrium, and infinite-horizon games, which can have many. Treating the finite-horizon backward recursion as a nonlinear dynamical system, the authors prove that its fixed points are exactly the stationary Nash equilibria of the infinite-horizon game, and any such equilibrium can be reproduced in a finite-horizon game by appropriate terminal costs. Cycles of the recursion correspond to periodic non-stationary Nash equilibria. Simulations document three asymptotic regimes: convergence to a stationary equilibrium, convergence to a cycle, and bounded non-convergent trajectories. This matters for tuning finite-horizon approximations, because terminal costs act as equilibrium selectors and stability is not guaranteed.

What carries the argument

The key object is the map f : (S^n_++)^N -> (S^n_++)^N that sends the terminal cost tuple P_{t+1} to the previous-step Riccati solution P_t via (4)-(5). Its fixed points are equated with stationary Nash equilibria; its periodic orbits, satisfying P_l = f(P_{l+1}), are equated with periodic Nash equilibria. The reverse inclusion Fix(f) subset of P_NE_stat relies on positive definiteness of every Q_i, giving detectability of (A_cl^{-i}, Q_i^{1/2}) and a strict Lyapunov decrement; this is the load-bearing assumption.

What would settle it

Find a game with Q_i positive semidefinite (e.g., two-agent scalar, Q_1=0) with a fixed point P=f(P) but the closed-loop matrix has spectral radius >= 1; Proposition 1's reverse inclusion would fail. Alternatively, verify a reported cycle: if a sequence satisfies (13) but the product of its closed-loop matrices has spectral radius >= 1, Lemma 1 fails.

Watch

Extended reading notes

Core claim

The central claim is Proposition 1: for the discrete-time infinite-horizon LQ game with strictly positive definite state-cost matrices Q_i, the set of stationary Nash equilibria P_NE_stat is exactly the fixed point set Fix(f) of the finite-horizon Riccati recursion (10). Corollary 1 states that for any equilibrium P, setting the terminal cost Q_T = P yields a finite-horizon equilibrium K_t = g(P) for all t, recovering the infinite-horizon policy. Theorem 1 extends to cycles: any sequence {P_l} satisfying (13) yields a periodic Nash equilibrium that stabilizes the closed-loop system over one period, though individual policies in the cycle need not be stabilizing. The paper thus shows that the

Load-bearing premise

Every player's state-cost matrix Q_i must be strictly positive definite; if Q_i is only positive semidefinite, a fixed point need not be a Nash equilibrium, so the finite/infinite bridge can break.

Editorial extensions

If this is right

Terminal costs in finite-horizon LQ games act as equilibrium selectors: choosing Q_T = P selects the infinite-horizon equilibrium P for any P in P_NE_stat.
A receding-horizon game that applies policies from a periodic cycle may be stable only over one full period; individual policies can have spectral radius above 1, which can cause transient instability.
If the recursion converges, its limit is automatically an infinite-horizon Nash equilibrium, so finite-horizon approximations are consistent for attractive equilibria.
The existence of multiple attractors means convergence cannot be inferred from the existence of an equilibrium; stability of equilibria determines which, if any, are reached.
Periodic Nash equilibria, previously not identified in time-invariant LQ games, are shown to be possible and are observed numerically.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension is that for Q_i positive semidefinite, the inclusion Fix(f) subset of P_NE_stat can fail; one could search for a fixed point whose induced closed-loop matrix is unstable, which would directly falsify the unregularized version of Proposition 1.
The cycle-length distribution in simulations suggests that low-period cycles dominate, and cycle length appears largely independent of state dimension or number of agents; a theoretical explanation of this insensitivity could be pursued.
The bounded non-convergent regime may correspond to strange attractors or quasi-periodic behavior of the Riccati map; connecting this to known results in nonlinear dynamics could predict when receding-horizon implementations fail.
Incentive design could exploit the selector property: by choosing terminal costs, a planner may steer a population of agents toward a desired equilibrium without altering the game's infinite-horizon structure.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this paper gives a clean conceptual statement—fixed points of the finite-horizon Riccati recursion are exactly the stationary infinite-horizon Nash equilibria, and cycles of that recursion correspond to periodic Nash equilibria. The first claim is correct under Q_i,R_i ≻ 0 and the paper is honest about why that assumption is load-bearing. The second claim is genuinely new but rests on a lemma whose proof only treats real eigenvalues; the complex case is missing. I think it's a quick fix, but as written the proof is incomplete.

What I liked: Proposition 1 is sound. The reverse inclusion relies on positive definite Q_i to get a strict Lyapunov decrement; Remark 1 explicitly concedes that semi-definite Q_i can break it. That's the right way to handle a delicate assumption. Corollary 1 is almost definitional—choose terminal cost equal to the equilibrium value—but it makes the selector role of terminal costs explicit and the paper does not oversell it. The numerical catalog showing three regimes (convergence to stationary, convergence to cycles, bounded non-convergence) is useful, and the observation that individual policies in a cycle need not be stabilizing is a nice practical warning for receding-horizon applications.

Soft spots. The main one is Lemma 1. The proof by contradiction takes a real eigenvector v and derives a contradiction from (lambda^2-1) v^T P v > 0 against a negative right-hand side. Nothing rules out complex eigenvalues with |lambda|>1. The stress-test note has the repair: use v^H and the Hermitian form to get a contradiction. That is standard. But as written, the final 'therefore |lambda|<1 for every eigenvalue' does not follow. Since Theorem 1 leans entirely on Lemma 1 to invoke the periodic Riccati result, this needs to be patched.

Second, the paper does not verify the numerical cycles against the exact algebraic condition (13), and Remark 2 openly says so. So the periodic-Nash existence claim rests on simulation only. That is a limitation, not a deception, but readers should not mistake the numerics for a proof.

Minor: the numerical experiments are described but no code or data are released, so the 10^5 random-games table is hard to reproduce.

Overall: the central fixed-point bridge holds up; the cycle result is promising but formally incomplete in one step. This deserves a serious referee—the fix is easy, the paper is honest, and the conceptual contribution is real. I would suggest accepting conditional on a corrected Lemma 1 and ideally an exact cycle example.

Referee Report

3 major / 5 minor

Summary. The paper studies the backward Riccati recursion of finite-horizon LQ games as a discrete-time dynamical system f. The main theoretical claims are: (i) the fixed points of f coincide with the stationary linear state-feedback Nash equilibria of the infinite-horizon game, under Qi,Ri ≻ 0 (Proposition 1); (ii) every such equilibrium can be reproduced as a stationary finite-horizon equilibrium by setting the terminal cost to the equilibrium matrix P (Corollary 1); and (iii) any cycle of f satisfying (13) is stabilizing (Lemma 1) and defines a periodic Nash equilibrium of the time-invariant infinite-horizon game (Theorem 1). Simulations then classify the recursion's behavior into convergence to stationary equilibria, convergence to cycles, and bounded non-convergent trajectories.

Significance. If the central claims hold, the paper gives a clean algebraic explanation of the finite/infinite-horizon gap in LQ games and identifies a new phenomenon—periodic Nash equilibria in time-invariant LQ games—that is absent in the single-agent LQR case. The fixed-point equivalence in Proposition 1 is simple but useful, and Corollary 1 gives a constructive identity, even if its practical value is limited by the need to know the target equilibrium in advance. The paper is transparent about its main assumptions and limitations: Remark 1 explicitly flags that Qi ≻ 0 is load-bearing, and Remark 2 correctly cautions that the numerical evidence does not by itself prove existence of cycles. The main defect is a proof gap in Lemma 1 that currently leaves Theorem 1 formally incomplete, although the gap appears readily fixable.

major comments (3)

[§IV-B, Lemma 1 (proof following Eq. (14))]
[§IV-B, Theorem 1]
[§IV-A, Proposition 1 and Remark 1]

minor comments (5)

[§IV-B, Lemma 1] Typo: 'i∈[i,N]' should be 'i∈[1,N]'.
[§IV-A, Corollary 1] Duplicate word: 'by setting the the terminal costs'. Also, the interval [0, 1, ..., T−1] should be written [0, T−1].
[Abstract and §II] The abstract says finite-horizon LQ games 'admit a unique Nash equilibrium', but §III-A correctly states that uniqueness holds iff equation (7) has a unique solution. Rephrase the abstract to avoid an unconditional claim.
[§V, Simulations] The numerical criteria for declaring convergence to a cycle, convergence to a fixed point, or non-convergence are not specified. Please state the tolerances, horizon lengths, and random sampling distributions (including seeds) so the results are reproducible. Figure 2 would also benefit from clearer axis labels and a unified color scale.
[§V-B and Fig. 5] The observation that individual policies in a cycle need not be stabilizing is valuable for receding-horizon games. Consider stating this as a formal remark after Theorem 1 rather than only in the simulation discussion.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the fixed-point/equivalence argument is a direct algebraic reformulation with a nontrivial stability proof, and the terminal-cost recovery is an acknowledged construction, not a prediction.

full rationale

The central equality P_NE_stat = Fix(f) (Proposition 1) is not circular: f is defined by the finite-horizon recursion (4)-(5), and at a fixed point P=f(P) these equations coincide with the infinite-horizon algebraic Riccati equations (8)-(9). The reverse inclusion is proved, not assumed, via the Lyapunov decrement (Acl)^T P^i Acl - P^i = -(Q_i + (K^i)^T R^i K^i) < 0 and detectability from Q_i > 0. Corollary 1 is an existence construction, not a disguised fit or prediction: setting QT = P makes the recursion stationary by definition, and the paper explicitly concedes that this requires prior knowledge of P, so no quantity is fitted and then relabelled as a prediction. Lemma 1's restriction to real eigenvalues is a proof gap affecting rigor, but it is not a circular step. Self-citations ([8], [11]) appear only for numerical algorithms and literature context, and the periodic Riccati uniqueness theorem is cited from the external reference [24]. No parameter is fitted to data, no result is forced by normalization, and no central claim reduces to a self-citation chain.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

All central claims rest on strict positive definiteness of cost matrices, stabilizability of the joint system, and the usual linear quadratic game solvability assumptions. The paper transparently flags the Q_i≻0 limitation in Remark 1. No ad hoc entities or fit parameters are introduced.

assumptions (5)

domain assumption For each agent i, Q_i and R_i are positive definite.
Used throughout to guarantee detectability and strict Lyapunov decrease. Remark 1 states that the equivalence Fix(f)=P_NE_stat can fail if Q_i is only positive semidefinite.
domain assumption Equation (7) admits a unique solution for every t (Assumption 1).
Makes the map f well-defined over every finite horizon and ensures the finite-horizon game has a unique feedback Nash equilibrium.
domain assumption The pair (A, [B1 ... BN]) is stabilizable (Assumption 2).
Necessary for the existence of stabilizing closed-loop strategies in the infinite-horizon game.
standard math Bittanti et al. [24, Theorem 6.11] on the discrete periodic Riccati equation
Used in Theorem 1 to conclude that the periodic solution of the periodic Riccati equation corresponds to the unique optimal periodic LQR policy for each agent.
domain assumption Nash equilibrium is restricted to linear state-feedback strategies.
The paper analyzes linear state-feedback equilibria; the definitions of P_NE_stat and the fixed-point set are stated in terms of these strategies, and nonlinear equilibria are explicitly excluded.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Bridging Finite and Infinite-Horizon Nash Equilibria in Linear Quadratic Games." pith.science (2026). https://pith.science/paper/DHZV76CQ

@misc{pith2026250820675,
  author       = {Pith},
  title        = {Pith review of: Bridging Finite and Infinite-Horizon Nash Equilibria in Linear Quadratic Games},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DHZV76CQ}},
  note         = {Machine review of arXiv:2508.20675}
}

read the original abstract

Finite-horizon linear quadratic (LQ) games admit a unique Nash equilibrium, while infinite-horizon settings may have multiple. We clarify the relationship between these two cases by interpreting the finite-horizon equilibrium as a nonlinear dynamical system. Within this framework, we prove that its fixed points are exactly the infinite-horizon equilibria and that any such equilibrium can be recovered by an appropriate choice of terminal costs. We further show that periodic orbits of the dynamical system, when they arise, correspond to periodic Nash equilibria, and we provide numerical evidence of convergence to such cycles. Finally, simulations reveal three asymptotic regimes: convergence to stationary equilibria, convergence to periodic equilibria, and bounded non-convergent trajectories. These findings offer new insights and tools for tuning finite-horizon LQ games using infinite-horizon.

Figures

Figures reproduced from arXiv: 2508.20675 by the authors.

**Figure 2.** For each pair (n, N) we randomly generated 105 games with random terminal costs. The value of d is equal to n, but the results do not change significantly for other values. The three tables show the percentage of cases converging to an equilibrium point (left), to a cycle (center), or not converging (right). Notice that three different scales are used to better highlight variations. which is a contradiction. Therefo… view at source ↗

**Figure 4.** For each case, we kept randomly generating games unti [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

Figures from the paper (1 more)

**Figure 5.** Figure 5: Example of a periodic Nash equilibrium of length [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 23 canonical work pages

[1]

Distributed differenti al games for control of multi-agent systems,

D. Cappello and T. Mylvaganam, “Distributed differenti al games for control of multi-agent systems,” IEEE Transactions on Control of Network Systems , vol. 9, no. 2, pp. 635–646, 2021

work page 2021
[2]

Robots that take advantage of h uman trust,

D. P . Losey and D. Sadigh, “Robots that take advantage of h uman trust,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2019, pp. 7001–7008

work page 2019
[3]

Hierarchical game-theoretic planning for au tonomous vehicles,

J. F. Fisac, E. Bronstein, E. Stefansson, D. Sadigh, S. S. Sastry, and A. D. Dragan, “Hierarchical game-theoretic planning for au tonomous vehicles,” in 2019 International conference on robotics and automation (ICRA). IEEE, 2019, pp. 9590–9596

work page 2019
[4]

C hance- constrained linear quadratic gaussian games for multi-rob ot interaction under uncertainty,

K. Ren, G. Salizzoni, M. E. G¨ ursoy, and M. Kamgarpour, “C hance- constrained linear quadratic gaussian games for multi-rob ot interaction under uncertainty,” IEEE Control System Letters , 2025

work page 2025
[5]

On aggrega tive and mean ﬁeld games with applications to electricity markets,

D. Paccagnan, M. Kamgarpour, and J. Lygeros, “On aggrega tive and mean ﬁeld games with applications to electricity markets,” in 2016 European Control Conference (ECC) . IEEE, 2016, pp. 196–201

work page 2016
[6]

A dis- tributed game theoretic approach for optimal battery use in an energy community,

N. Kirsch, G. Salizzoni, T. Gorecki, and M. Kamgarpour, “ A dis- tributed game theoretic approach for optimal battery use in an energy community,” ACM SIGENERGY Energy Informatics Review , vol. 4, no. 4, pp. 207–213, 2025

work page 2025
[7]

Basar and G

T. Basar and G. Olsder, Dynamic noncooperative game theory . So- ciety for Industrial and Applied Mathematics, 1998

work page 1998
[8]

Nash equil ibria in scalar discrete-time linear quadratic games,

G. Salizzoni, R. Ouhamma, and M. Kamgarpour, “Nash equil ibria in scalar discrete-time linear quadratic games,” 2025 European Control Conference (ECC) , 2025

work page 2025

Show all 24 references

[9]

Fe edback nash equilibria for scalar two-player linear-quadratic di screte-time dynamic games,

B. Nortmann, A. Monti, M. Sassano, and T. Mylvaganam, “Fe edback nash equilibria for scalar two-player linear-quadratic di screte-time dynamic games,” IF AC-PapersOnLine, vol. 56, no. 2, pp. 1772–1777, 2023

2023
[10]

Feedback n ash equilibria for scalar N-player linear quadratic dynamic ga mes,

B. Nortmann, M. Sassano, and T. Mylvaganam, “Feedback n ash equilibria for scalar N-player linear quadratic dynamic ga mes,” arXiv preprint arXiv:2411.19377, 2024

2024 arXiv
[11]

On Linear Quadratic Potential Games,

S. Hosseinirad, A. A. Porzani, G. Salizzoni, and M. Kamg arpour, “On Linear Quadratic Potential Games,” Automatica, 2025

2025
[12]

On the open-loop Nash equilibrium in LQ -games,

J. C. Engwerda, “On the open-loop Nash equilibrium in LQ -games,” Journal of Economic Dynamics and Control , vol. 22, no. 5, pp. 729– 762, 1998

1998
[13]

Asymptotic Analysis of Linear Feedback Nash Equilibria in Nonzero-Sum Linear-Q uadratic Differential Games,

A. Weeren, J. Schumacher, and J. Engwerda, “Asymptotic Analysis of Linear Feedback Nash Equilibria in Nonzero-Sum Linear-Q uadratic Differential Games,” Journal of Optimization Theory and Applications , vol. 101, no. 3, pp. 693–722, 1999

1999
[14]

Receding horizon games with coupling constraints for demand-side ma nage- ment,

S. Hall, G. Belgioioso, D. Liao-McPherson, and F. Dorﬂe r, “Receding horizon games with coupling constraints for demand-side ma nage- ment,” in 2022 IEEE 61st Conference on Decision and Control (CDC) , 2022, pp. 3795–3800

2022
[15]

Reced- ing horizon games for modeling competitive supply chains,

S. Hall, L. Guerrini, F. D¨ orﬂer, and D. Liao-McPherson , “Reced- ing horizon games for modeling competitive supply chains,” IF AC- PapersOnLine, vol. 58, no. 18, pp. 8–14, 2024

2024
[16]

Stability Certiﬁcates for Receding Horizon Games,

S. Hall, D. Liao-McPherson, G. Belgioioso, and F. D¨ orﬂ er, “Stability Certiﬁcates for Receding Horizon Games,” Apr. 2024

2024
[17]

Linear-quadratic dyna mic games as receding-horizon variational inequalities,

E. Benenati and S. Grammatico, “Linear-quadratic dyna mic games as receding-horizon variational inequalities,” arXiv preprint arXiv:2408.15703, 2024

2024 arXiv
[18]

Energy management via pricing in lq dynamic games,

S. Coogan, L. J. Ratliff, D. Calderone, C. Tomlin, and S. S. Sastry, “Energy management via pricing in lq dynamic games,” in 2013 American Control Conference . IEEE, 2013, pp. 443–448

2013
[19]

Linear different ial games for cooperative behavior planning of autonomous vehicles usin g mixed- integer programming,

T. Kessler, K. Esterle, and A. Knoll, “Linear different ial games for cooperative behavior planning of autonomous vehicles usin g mixed- integer programming,” in 2020 59th IEEE Conference on Decision and Control (CDC) . IEEE, 2020, pp. 4060–4066

2020
[20]

Policy Gradient Methods F ind the Nash Equilibrium in N-player General-sum Linear-quadrati c Games,

B. Hambly, R. Xu, and H. Y ang, “Policy Gradient Methods F ind the Nash Equilibrium in N-player General-sum Linear-quadrati c Games,” SSRN Electronic Journal , 1979

1979
[21]

Rein- forcement learning in nonzero-sum linear quadratic deep st ructured games: Global convergence of policy optimization,

Masoud Roudneshin, Jalal Arabneydi and Amir G. Aghdam, “Rein- forcement learning in nonzero-sum linear quadratic deep st ructured games: Global convergence of policy optimization,” arXiv preprint: arXiv:2011.14391, 2020

2011 arXiv
[22]

Katok and B

A. Katok and B. Hasselblatt, Introduction to the modern theory of dynamical systems . Cambridge university press, 1995, no. 54

1995
[23]

The discrete riccati equation of optimal c ontrol,

V . Kuˇ cera, “The discrete riccati equation of optimal c ontrol,” Kyber- netika, vol. 8, no. 5, pp. 430–447, 1972

1972
[24]

The period ic riccati equation,

S. Bittanti, P . Colaneri, and G. De Nicolao, “The period ic riccati equation,” in The Riccati Equation . Springer, 1991, pp. 127–162

1991

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

Distributed differenti al games for control of multi-agent systems,

D. Cappello and T. Mylvaganam, “Distributed differenti al games for control of multi-agent systems,” IEEE Transactions on Control of Network Systems , vol. 9, no. 2, pp. 635–646, 2021

work page 2021

[2] [2]

Robots that take advantage of h uman trust,

D. P . Losey and D. Sadigh, “Robots that take advantage of h uman trust,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2019, pp. 7001–7008

work page 2019

[3] [3]

Hierarchical game-theoretic planning for au tonomous vehicles,

J. F. Fisac, E. Bronstein, E. Stefansson, D. Sadigh, S. S. Sastry, and A. D. Dragan, “Hierarchical game-theoretic planning for au tonomous vehicles,” in 2019 International conference on robotics and automation (ICRA). IEEE, 2019, pp. 9590–9596

work page 2019

[4] [4]

C hance- constrained linear quadratic gaussian games for multi-rob ot interaction under uncertainty,

K. Ren, G. Salizzoni, M. E. G¨ ursoy, and M. Kamgarpour, “C hance- constrained linear quadratic gaussian games for multi-rob ot interaction under uncertainty,” IEEE Control System Letters , 2025

work page 2025

[5] [5]

On aggrega tive and mean ﬁeld games with applications to electricity markets,

D. Paccagnan, M. Kamgarpour, and J. Lygeros, “On aggrega tive and mean ﬁeld games with applications to electricity markets,” in 2016 European Control Conference (ECC) . IEEE, 2016, pp. 196–201

work page 2016

[6] [6]

A dis- tributed game theoretic approach for optimal battery use in an energy community,

N. Kirsch, G. Salizzoni, T. Gorecki, and M. Kamgarpour, “ A dis- tributed game theoretic approach for optimal battery use in an energy community,” ACM SIGENERGY Energy Informatics Review , vol. 4, no. 4, pp. 207–213, 2025

work page 2025

[7] [7]

Basar and G

T. Basar and G. Olsder, Dynamic noncooperative game theory . So- ciety for Industrial and Applied Mathematics, 1998

work page 1998

[8] [8]

Nash equil ibria in scalar discrete-time linear quadratic games,

G. Salizzoni, R. Ouhamma, and M. Kamgarpour, “Nash equil ibria in scalar discrete-time linear quadratic games,” 2025 European Control Conference (ECC) , 2025

work page 2025