Pith. sign in

REVIEW 3 major objections 4 minor 74 references

Well-Posed KL-Regularized Control via Wasserstein and Kalman-Wasserstein KL Divergences

T0 review · 3 major / 4 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Replacing the Fisher-Rao geometry in the KL divergence with transport geometries yields KL analogues that stay finite under support mismatch and low noise, fixing a singularity in KL-regularized LQR control.

desk verdict A real contribution to well-posed KL-style LQR regularization, but the central WKL closed-form theorem degenerates at equal covariances — exactly the case the control section uses. read the letter →

arxiv 2602.02250 v2 pith:M4MXM6EB submitted 2026-02-02 math.OC cs.LG

Viktor Stein , Adwait Datar , Nihat Ay This is my paper
classification math.OCcs.LG MSC 49Q2293E20
keywords state-space-awareKLdivergencesWassersteindivergenceKalman-WassersteinKL-regularizedLQRlow-noisesingularityellipticdistributionsdiscountedRiccatiequationcovarianceinflation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that KL regularization in linear-quadratic control becomes singular as process noise vanishes: the control penalty scales with the inverse noise covariance, driving optimal feedback to zero and degrading closed-loop stability. It introduces two state-space-aware KL analogues — the Wasserstein KL (WKL) and Kalman-Wasserstein KL (KWKL) — by substituting transport-based geometries into the dynamical representation of the KL. For elliptic distributions these divergences have closed forms, and for equal-covariance Gaussians the KWKL equals a KL with covariance Σ+λI, a geometric version of additive covariance inflation. The headline control result is that in the zero-noise limit the KWKL-regularized gain approaches the WKL gain, a generally nonzero policy, while the classical KL gain collapses to zero under a spectral-radius condition. The paper concludes that these new regularizers yield well-posed LQR problems in both high- and low-noise regimes and interpolate smoothly between KL and transport penalties.

What carries the argument

The G-divergence: for a geometry G on the space of smooth probability densities, define DG(µ|ν) as the infimum over G-dual geodesics γ from µ to ν of ∫_0^1 t g^G_γ(t)(γ̇(t),γ̇(t)) dt. With G = Fisher-Rao this recovers the reverse KL; with G = the Wasserstein metric it gives WKL; with G = the Kalman-Wasserstein metric, whose mobility is m(f)=f(C(f)+λI), it gives KWKL. The geodesics are realized as gradient flows of linear expectation functionals, which keeps elliptic distributions invariant and yields closed-form divergence formulas. In the LQR application the divergence contributes a quadratic control cost, so the optimal policy is obtained from a discounted algebraic Riccati equation in whi

What would settle it

Take two commuting positive-definite covariances Σ0, Σ1 with one strictly decreasing eigenvalue, compute the candidate KWKL formula from Theorem 4.11, and numerically evaluate the defining geodesic integral: if the two disagree, the formula fails outside its stated condition. For the control claim, simulate the double-integrator LQR at very small ρ and check whether the KL-regularized closed-loop spectral radius approaches the unit circle while the KWKL-regularized radius stays below it.

Watch

Extended reading notes

Core claim

The central claim is that replacing the Fisher-Rao geometry in the KL's dynamical representation with Wasserstein-type geometries produces KL analogues that remain finite when supports mismatch and when covariances degenerate. For elliptic distributions, including Gaussians, the WKL and KWKL divergences admit closed-form expressions; in the equal-covariance case, DKW(µ|ν) = (1/2)(Δm)^T(κgΣ+λI)^{-1}Δm, which for Gaussians equals the KL between Gaussians with covariance Σ+λI. In the LTI control setting, these divergences add quadratic control penalties with effective control-cost matrices RKL = B^TΣ_w^{-1}B, RWKL = B^TB, and RKW = B^T(Σ_w+λI)^{-1}B. Because RKW stays finite as Σ_w→0, the KWKL-

Load-bearing premise

The closed-form WKL and KWKL formulas are obtained by formally extending the compact-manifold density geodesic framework to R^d, and the paper says it is 'not claiming their well-definedness' there; in particular the KWKL formula is proved only for commuting covariances under a condition that rules out eigenvalues decreasing along the geodesic, so the broad claim of closed-form divergences for elliptic families is narrower than its statement.

Editorial extensions

If this is right

  • In the low-noise limit, KWKL-regularized LQR retains a nonzero optimal feedback gain; classical KL-regularized gain vanishes when the spectral radius of √γA is less than 1.
  • As ρ→∞, the KWKL gain converges to the KL gain, making KWKL a bridge between KL and WKL regularizers.
  • The equal-covariance KWKL formula identifies additive covariance inflation (a noise floor) as the geometric consequence of the Kalman-Wasserstein metric.
  • The closed-form WKL/KWKL expressions extend to elliptic families, so the control construction transfers to non-Gaussian noise models.
  • The regularizer determines the LQR control penalty matrix from the geometry and process noise covariance, replacing an otherwise heuristic design choice.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the control application only needs the equal-covariance KWKL formula, the main conclusions likely survive a rigorous non-compact extension; the open risk is the broader closed-form theorem for unequal commuting covariances, which is proved only under an eigenvalue-nonincreasing condition.
  • The same geometry substitution could define state-space-aware mutual informations or exploration bonuses in reinforcement learning, where KL penalties also blow up under support mismatch.
  • A testable prediction is that a model-based RL agent using KWKL instead of KL regularization should keep exploring near deterministic transitions and show more stable policy updates in low-noise environments.
  • The interpolation behavior of KWKL suggests λ can be tuned as a noise floor matched to model error, making the regularizer less of a free parameter.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces state-space-aware KL divergences (WKL and KWKL) by replacing the Fisher–Rao geometry in the dynamical formulation of the KL divergence with Wasserstein-type geometries. It derives closed-form formulas for elliptic distributions (Theorems 4.10 and 4.11), interprets KWKL as enforcing a covariance noise floor, and applies both divergences as regularizers in discrete-time LTI optimal control. The control section reduces the regularized problem to discounted LQR, proves that the KWKL-regularized optimal gains interpolate between the KL and WKL limits as noise vanishes (Corollary 5.1), and illustrates the behavior on double-integrator and cart-pole examples.

Significance. If the closed-form divergence formulas are correct, the paper provides a useful geometric framework for regularization in control and RL: the KL divergence's low-noise singularity is replaced by well-posed penalties with an explicit noise-floor interpretation, and the control results are concrete enough to test. The Riccati-based analysis (Theorem C.1 and Corollary 5.1) is a genuine strength, as is the explicit identification of the KWKL penalty with additive covariance inflation. However, the stated closed-form Theorem 4.10 has a concrete algebraic gap in the equal-covariance case, and the paper explicitly disclaims well-definedness when passing from compact manifolds to R^d. These issues make the central closed-form claims contingent and need to be repaired before the results can be taken as stated.

major comments (3)
  1. [Theorem 4.10, Eq. (9)] The equal-covariance case of the WKL formula is not supported by the stated theorem. For Σ0=Σ1=Σ and κg=1, one has R=I, log(R)=0, and Q=(R−I)^†(...)(R−I)^†=0, so Eq. (9) returns 0 for every mean shift Δm. But Appendix A.3 gives B=0 and b=m1−m0, and Eq. (8) then yields D=1/2||Δm||^2. The missing piece is a limiting interpretation in which Q tends to 2I on the mean subspace; without this, the theorem is algebraically wrong at equal covariances. Since the control section's D_WKL(pt||p0)=1/2||Bu||^2 is exactly the equal-covariance case, Corollary 5.1 and both numerical examples rely on a limit that Theorem 4.10 neither states nor proves.
  2. [Section 4 preamble] The paper explicitly says, before stating Theorems 4.10 and 4.11, that it will use 'the analogies of the formulas above, not claiming their well-definedness' on M=R^d. Yet these theorems are presented as unconditional closed-form results for elliptic distributions on R^d. This is a load-bearing rigor gap: the entire contribution of explicit WKL/KWKL formulas depends on this non-compact extension. The authors should either supply a limiting/truncation argument or state clearly which parts are formal analogies and which are proven.
  3. [Theorem 4.11 and Corollary 4.12] Theorem 4.11 is stated only under simultaneous diagonalizability and the condition λ<κ_g(s0)_i(s1)_i/((s1)_i−(s0)_i) for all i with (s0)_i≠(s1)_i. For any direction with (s1)_i<(s0)_i, the bound is negative, so the theorem says nothing about covariance decreases. Moreover, the equal-covariance case of Corollary 4.12 violates the theorem's premise (s0)_i≠(s1)_i, so it is not a direct corollary as written. Since Eq. (10) is the formula actually used in Section 5, the paper should either extend Theorem 4.11 to the full commuting case (if true by continuity or analytic continuation) or present an independent proof of Corollary 4.12.
minor comments (4)
  1. [Definition 3.1] The codomain is written as [0,1]; it should presumably be [0,∞).
  2. [Section 5, divergence table] The formulas D_KL(pt||p0), D_WKL(pt||p0), D_KW(pt||p0) are stated without derivation; for D_WKL this is precisely the point where the missing equal-covariance limit in Theorem 4.10 matters. A short derivation or a reference to a corrected theorem would help.
  3. [Figure 4 caption] The phrase 'preserve maintain spectral radii' is ungrammatical; should be 'maintain spectral radii'.
  4. [Remark 5.2] The statement 'if λ<1/√γ is an eigenvalue of A' is slightly confusing because λ is already used as the KW noise-floor parameter; consider renaming or clarifying the eigenvalue notation.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity; the central control claim rests on explicit derivations and standard LQR.

full rationale

The central claim (Corollary 5.1) is not circular. It follows from substituting Gaussian transition densities into the explicit divergence formulas and then solving a standard discounted LQR Riccati equation (Theorem C.1, proved in Appendix C). The divergence formulas themselves are derived in Appendices A.3-A.4 rather than merely assumed from the same-group references [5,17,22]; those citations are provenance, and the equal-covariance WKL/KWKL penalties used in the control section are direct facts (half squared Wasserstein distance and a quadratic form with (Sigma_w + lambda I)^{-1}) that do not depend on an unverified self-citation. No parameter is fitted to data and then called a prediction, and no uniqueness theorem is invoked to forbid alternative regularizers. The notable weaknesses are non-circular correctness gaps: Section 4 explicitly says 'we will use the analogies of the formulas above, not claiming their well-definedness' on R^d; Theorem 4.11 is proven only under (s0)_i != (s1)_i and a one-sided lambda bound, while Corollary 4.12 applies it at equal covariances; and formula (9) formally gives zero when R=I, so the Section 5 value D_WKL = 1/2 ||Bu||^2 requires an unstated limiting interpretation. These affect rigor, not circularity. Score 2 reflects the self-referential lineage of the divergence framework, not a reduction of the derivation to its own inputs by construction.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

Central derivation rests on the chosen G-divergence construction from Ay-Amari/Felice-Ay, a formal transfer of density-manifold calculus to R^d without well-definedness proof, the commuting/eigenvalue-increasing restriction in the KWKL theorem, and standard LQR stabilizability/detectability. No physical entities are invented; λ is a user-chosen smoothing parameter.

free parameters (1)
  • λ (KW noise floor) = λ > 0; examples use λ ∈ {1, 10^{-1}, 10^{-2}, 10^{-4}}
    Introduced in the Kalman-Wasserstein metric (Example 2.4), appears in DKW (Cor 4.12) and RKW (Eq. 12). Not fitted to data but chosen by hand; the claimed finite behavior depends on λ>0.
assumptions (5)
  • domain assumption Dynamical representation of KL as time-weighted Fisher-Rao geodesic energy (Eq. 2), and its replacement by transport geometries (Def. 3.1)
    The whole paper assumes this is the right notion of 'KL analog'; inherited from [22].
  • domain assumption Formal density-manifold calculus extends to non-compact M=R^d
    Section 4 states 'not claiming their well-definedness'; closed-form Theorems 4.10/4.11 use this formal calculus.
  • ad hoc to paper Theorem 4.11 restrictions: simultaneously diagonalizable covariances and λ < κ_g(s0)i(s1)i/((s1)i-(s0)i)
    Restricts KWKL to commuting, variance-non-decreasing cases; not discussed as a limitation.
  • domain assumption Standard LQR conditions: (A,B) stabilizable, (A,Q^{1/2}) detectable, B full column rank, Gaussian process noise
    Theorem C.1 and Corollary 5.1 use these to guarantee a unique DARE solution and optimal linear policy.
  • ad hoc to paper Per-stage additive divergence cost is the correct regularized objective
    Section 5 defines J as discounted sum of stage costs plus per-step D(pt||p0); the relationship to path-space KL-regularized control is not proven.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Well-Posed KL-Regularized Control via Wasserstein and Kalman-Wasserstein KL Divergences." pith.science (2026). https://pith.science/paper/M4MXM6EB

@misc{pith2026260202250,
  author       = {Pith},
  title        = {Pith review of: Well-Posed KL-Regularized Control via Wasserstein and Kalman-Wasserstein KL Divergences},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M4MXM6EB}},
  note         = {Machine review of arXiv:2602.02250}
}
read the original abstract

Kullback-Leibler (KL) divergence regularization is widely used in reinforcement learning, but it becomes infinite under support mismatch and can degenerate in low-noise regimes. Using a unified information-geometric framework, we introduce KL analogs by replacing the Fisher-Rao geometry in the dynamical formulation of the KL with transport-based geometries, and derive closed-form expressions for common distribution families. Between elliptic distributions, these divergences remain finite for degenerating equal covariances and yield a geometric interpretation of regularization heuristics used in Kalman ensemble methods. We demonstrate the utility of these divergences in KL-regularized optimal control. In the fully tractable setting of linear time-invariant systems with Gaussian process noise, the classical KL reduces to a quadratic control penalty that becomes singular as process noise vanishes. Our variants remove this singularity and yield well-posed problems. In both the double integrator and cart-pole examples, the resulting controls preserve nontrivial feedback and achieve better closed-loop performance.

Figures

Figures reproduced from arXiv: 2602.02250 by the authors.

Figure 1
Figure 1. For µ ∼ N (m0, σ2 ) and ν ∼ N (m1, σ2 ), we compare DKW(µ | ν) = 1 2(σ2+λ) (∆m) 2 with KL µ | ν  = 1 2σ2 (∆m) 2 and WKL(µ | ν) = 1 2 (∆m) 2 , where λ = 1 and (∆m) 2 = 1 4 . Remark 4.13 (DKW interpolates between KL and WKL). In [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Optimal feedback gains for KL −, WKL −, and KW − regularized LQR as function ρ. Each feedback gain F ∈ R 1×2 shown component-wise; both entries are shown. KL gains shrink to zero as noise disappears, WKL gains are constant because they do not depend on ρ, and KW gains smoothly interpolate between the two. 0 100 200 300 −4 −2 0 2 4 qt ρ = 1e-01 KL WKL KWλ=1 0 100 200 300 −2 −1 0 1 2 ρ = 1e-02 0 100 200 300 −1 0 1 qt … view at source ↗
Figure 3
Figure 3. Closed-loop trajectories of the double integrator under KL-, WKL-, and KW-regularized controllers for ρ ∈ {10−1 , 10−2 , 10−3 , 10−4}. For small ρ, KL yields weak feedback and large oscil￾lations, whereas WKL- and KW-regularized controllers lead to reduced oscillations. See [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Closed-loop spectral radius as a function of noise ρ for different values of parameter λ. KL-regularized controllers approach the unit circle as ρ → 0, indicating near-instability, while WKL- and KW-regularized controllers preserve maintain spectral radii well below on…
Figure 5
Figure 5. Figure 5: Cart-pole system The cart-pole system is depicted in [PITH_FULL_IMAGE:figures/full_fig_p033_5.png]
Figure 6
Figure 6. Figure 6: Closed-loop cart positions for the nonlinear cart–pole under KL-, WKL-, and KW￾regularized controllers for ρ ∈ {10−3 , 10−4 , 10−5 , 10−6}. At low noise, KL produces near-zero feedback and large oscillations, whereas WKL and KW reduce oscillations. 10−10 10−7 10−4 10−1…
Figure 7
Figure 7. Figure 7: Optimal costs J ◦ as a function of noise ρ for λ ∈ {10−4 , 10−2 , 10−1 , 1} 35 [PITH_FULL_IMAGE:figures/full_fig_p035_7.png]
Figure 8
Figure 8. Figure 8: Closed-loop trajectories of the double integrator under KL-, WKL-, and KW-regularized controllers for noise levels ρ ∈ {10−1 , 10−2 , 10−3 , 10−4}. At low noise, KL produces near-zero feedback and large oscillations, whereas WKL and KW yield more stable trajectories. F…
Figure 9
Figure 9. Figure 9: Comparison of optimal policies for {KL,WKL, KW}-regularized LQR. Each linear policy is represented by a 1 × 2 gain matrix F; both entries are shown. KL gains shrink to zero as noise disappears, WKL gains remain constant since they do not depend on ρ, and KW gains smoot…
Figure 10
Figure 10. Figure 10: Comparison of sub-level sets for KL-divergence, WKL-divergence, and KWKL￾divergence centered at various univariate Gaussian reference distributions N (m0, σ0), indicated by solid markers. Each shaded region represents the set of distributions N (m, σ) whose divergence…
Figure 11
Figure 11. Figure 11: Level sets for KWKL-divergence centered at various univariate Gaussian reference dis￾tributions N (m0, σ0), indicated by solid markers for different values of λ. 38 [PITH_FULL_IMAGE:figures/full_fig_p038_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

74 extracted references · 3 linked inside Pith

  1. [1]

    Abuqrais and D

    M. Abuqrais and D. Pigoli. A Riemannian covariance for manifold-valued data. arXiv preprint arXiv:2410.06164, 2024

    arXiv 2024

  2. [2]

    Altekrüger, J

    F. Altekrüger, J. Hertrich, and G. Steidl. Neural Wasserstein gradient flows for discrepancies with Riesz kernels. In Proceedings of the 40th International Conference on Machine Learning , pages 664–690, 2023

    2023

  3. [3]

    Ambrosio, N

    L. Ambrosio, N. Gigli, and G. Savaré. Gradient flows: in metric spaces and in the space of probability measures. Springer Science & Business Media, 2 edition, 2008

    2008

  4. [4]

    B. D. Anderson and J. B. Moore. Optimal control: linear quadratic methods . Courier Corpo- ration, 2007

    2007

  5. [5]

    N. Ay. Information geometry of the Otto metric. Information Geometry , pages 1–24, 2024

    2024

  6. [6]

    Ay and S.-i

    N. Ay and S.-i. Amari. A novel approach to canonical divergences within information geometry. Entropy, 17(12):8111–8129, 2015

    2015

  7. [7]

    N. Ay, J. Jost, H. Vân Lê, and L. Schwachhöfer. Information geometry , volume 64. Springer, 2017. 15

    2017

  8. [8]

    Ay and L

    N. Ay and L. J. Schwachhöfer. Torsion of α-connections on the density manifold. In Interna- tional Conference on Geometric Science of Information , pages 417–427. Springer, 2025

    2025

Show all 74 references
  1. [9]

    Bauer, M

    M. Bauer, M. Bruveris, and P. W. Michor. Uniqueness of the Fisher–Rao metric on the space of smooth densities. Bulletin of the London Mathematical Society , 48(3):499–506, 2016

    2016

  2. [10]

    Beier, R

    F. Beier, R. Beinert, and G. Steidl. On a linear Gromov–Wasserstein distance. IEEE Trans- actions on Image Processing , 31:7292–7305, 2022

    2022

  3. [11]

    Benamou and Y

    J.-D. Benamou and Y. Brenier. A computational fluid mechanics solution to the Monge- Kantorovich mass transfer problem. Numerische Mathematik , 84(3):375–393, 2000

    2000

  4. [12]

    Bertsekas

    D. Bertsekas. Dynamic programming and optimal control , volume II. Belmont, MA: Athena Scientific, 3rd edition, 2011

    2011

  5. [13]

    K. M. Borgwardt, A. Gretton, M. J. Rasch, H.-P. Kriegel, B. Schölkopf, and A. J. Smola. Integrating structured biological data by Kernel Maximum Mean Discrepancy. Bioinformatics, 22(14):e49–e57, 07 2006

    2006

  6. [14]

    Boufadène and F.-X

    S. Boufadène and F.-X. Vialard. On the global convergence of Wasserstein gradient flow of the Coulomb discrepancy. SIAM Journal on Mathematical Analysis , 57(4):4556–4587, 2025

    2025

  7. [15]

    J. A. Carrillo, S. Lisini, G. Savaré, and D. Slepčev. Nonlinear mobility continuity equations and generalized displacement convexity. Journal of Functional Analysis , 258(4):1273–1309, 2010

    2010

  8. [16]

    M. Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in Neural Information Processing Systems , 26, 2013

    2013

  9. [17]

    Datar and N

    A. Datar and N. Ay. Wasserstein KL-divergence for Gaussian distributions. In F. Nielsen and F. Barbaresco, editors, Geometric Science of Information , pages 91–101, Cham, 2026. Springer Nature Switzerland

    2026

  10. [18]

    Dolbeault, B

    J. Dolbeault, B. Nazaret, and G. Savaré. A new class of transport distances between measures. Calculus of Variations and Partial Differential Equations , 34(2):193–231, 2009

    2009

  11. [19]

    Duncan, N

    A. Duncan, N. Nüsken, and L. Szpruch. On the geometry of Stein variational gradient descent. Journal of Machine Learning Research , 24(56):1–39, 2023

    2023

  12. [20]

    Duong, N

    R. Duong, N. Rux, V. Stein, and G. Steidl. Wasserstein gradient flows of maximum mean dis- crepancy functionals with distance kernels under Sobolev regularization. Philosophical trans- actions. Series A, Mathematical, physical, and engineering sciences , 383(2298):20240243, 2025

    2025

  13. [21]

    Duong, V

    R. Duong, V. Stein, R. Beinert, J. Hertrich, and G. Steidl. Wasserstein gradient flows of MMD functionals with distance kernel and Cauchy problems on quantile functions. ESAIM: Control, Optimisation and Calculus of Variations , 2026

    2026

  14. [22]

    Felice and N

    D. Felice and N. Ay. Towards a canonical divergence within information geometry. Information geometry, 4(1):65–130, 2021

    2021

  15. [23]

    Frahm, M

    G. Frahm, M. Junker, and A. Szimayer. Elliptical copulas: applicability and limitations. Statistics & Probability Letters , 63(3):275–286, 2003. 16

    2003

  16. [24]

    Friedrich

    T. Friedrich. Die Fisher-Information und symplektische Strukturen. Mathematische Nachrichten, 153(1):273–296, 1991

    1991

  17. [25]

    Garbuno-Inigo, F

    A. Garbuno-Inigo, F. Hoffmann, W. Li, and A. M. Stuart. Interacting Langevin diffusions: Gra- dient structure and ensemble Kalman sampler. SIAM Journal on Applied Dynamical Systems , 19(1):412–441, 2020

    2020

  18. [26]

    Glaser, M

    P. Glaser, M. Arbel, and A. Gretton. KALE flow: A relaxed KL gradient flow for probabilities with disjoint support. In Advances in Neural Information Processing Systems, volume 34, pages 8018–8031, Virtual event, 6–14 Dec 2021

    2021

  19. [27]

    Gretton, K

    A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Schölkopf, and A. Smola. A kernel two-sample test. Journal of Machine Learning Research , 13(25):723–773, 2012

    2012

  20. [28]

    Hardion and H

    M. Hardion and H. Lavenant. Gradient flows of potential energies in the geometry of Sinkhorn divergences. arXiv preprint arXiv:2511.14278, 2025

    2025

  21. [29]

    Hashizume, K

    Y. Hashizume, K. Oishi, and K. Kashima. Tsallis entropy regularization for linearly solvable MDP and linear quadratic regulator. arXiv preprint arXiv:2403.01805 , 2024

    2024 arXiv

  22. [30]

    H. J. Kappen. Path integrals and symmetry breaking for optimal control theory. Journal of Statistical Mechanics: Theory and Experiment , 2005(11):P11011, 2005

    2005

  23. [31]

    V. Kučera. A review of the matrix Riccati equation. Kybernetika, 9(1):42–61, 1973

    1973

  24. [32]

    Kullback and R

    S. Kullback and R. A. Leibler. On information and sufficiency. The Annals of Mathematical Statistics, 22(1):79–86, 1951

    1951

  25. [33]

    J. D. Lafferty. The density manifold and configuration space quantization. Transactions of the American Mathematical Society , 305(2):699–741, 1988

    1988

  26. [34]

    W. Lee, L. Wang, and W. Li. Deep JKO: time-implicit particle methods for general nonlinear gradient flows. Journal of Computational Physics , page 113187, 2024

    2024

  27. [35]

    W. Li. Geometric calculations on density manifolds from reciprocal relations in hydrodynamics. arXiv preprint arXiv:2501.16479, 2025

    2025

  28. [36]

    Liero and A

    M. Liero and A. Mielke. Gradient structures and geodesic convexity for reaction–diffusion systems. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engi- neering Sciences, 371(2005):20120346, 2013

    2005

  29. [37]

    Q. Liu. Stein variational gradient descent as gradient flow. Advances in Neural Information Processing Systems, 30, 2017

    2017

  30. [38]

    Liu and D

    Q. Liu and D. Wang. Stein variational gradient descent: A general purpose Bayesian inference algorithm. Advances in Neural Information Processing Systems , 29, 2016

    2016

  31. [39]

    T. Liu, P. Ghosal, K. Balasubramanian, and N. Pillai. Towards understanding the dynamics of Gaussian-Stein variational gradient descent. Advances in Neural Information Processing Systems, 36, 2024. 17

    2024

  32. [40]

    Liutkus, U

    A. Liutkus, U. Simsekli, S. Majewski, A. Durmus, and F.-R. Stöter. Sliced-Wasserstein flows: Nonparametric generative modeling via optimal transport and diffusions. In International Conference on Machine Learning , pages 4104–4113. PMLR, 2019

    2019

  33. [41]

    J. Lott. Some geometric calculations on Wasserstein space. Communications in Mathematical Physics, 277(2):423–437, 2008

    2008

  34. [42]

    Nielsen and K

    F. Nielsen and K. Okamura. On the f -divergences between densities of a multivariate location or scale family. Statistics and Computing , 34(1), jan 2024

    2024

  35. [43]

    Nüsken and D

    N. Nüsken and D. Renger. Stein variational gradient descent: Many-particle and long-time asymptotics. Foundations of Data Science , 5(3):286–320, 2023

    2023

  36. [44]

    F. Otto. The geometry of dissipative evolution equations: the porous medium equation. Com- munications in Partial Differential Equations , 26:101–174, 2001

    2001

  37. [45]

    X. Pennec. Intrinsic statistics on Riemannian manifolds: Basic tools for geometric measure- ments. Journal of Mathematical Imaging and Vision , 25(1):127–154, 2006

    2006

  38. [46]

    R. Peyre. Comparison between W2 distance and ˙H −1 norm, and localization of Wasserstein distance. ESAIM: Control, Optimisation and Calculus of Variations , 24(4):1489–1501, 2018

    2018

  39. [47]

    Rabin, G

    J. Rabin, G. Peyré, J. Delon, and M. Bernot. Wasserstein barycenter and its application to texture mixing. In International Conference on Scale Space and Variational Methods in Computer Vision , pages 435–446. Springer, 2011

    2011

  40. [48]

    B. Recht. A tour of reinforcement learning: The view from continuous control. Annual Review of Control, Robotics, and Autonomous Systems , 2(1):253–279, 2019

    2019

  41. [49]

    R. Schmidt. Tail dependence for elliptically contoured distributions. Mathematical Methods of Operations Research, 55(2):301–327, 2002

    2002

  42. [50]

    Schulman, S

    J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz. Trust region policy optimization. In International Conference on Machine Learning , pages 1889–1897. PMLR, 2015

    2015

  43. [51]

    Stein and W

    V. Stein and W. Li. Towards understanding accelerated Stein variational gradient flow – analysis of generalized bilinear kernels for Gaussian target distributions. arXiv preprint arXiv:2509.04008, 2025

    2025

  44. [52]

    Stein and W

    V. Stein and W. Li. Accelerated Stein variational gradient flow. In F. Nielsen and F. Barbaresco, editors, Geometric Science of Information. 7th International Conference, GSI 2025, Saint- Malo, France, October 29–31, 2025, Proceedings , volume 16033 of Lecture Notes in Compute...

    2025

  45. [53]

    Stein, S

    V. Stein, S. Neumayer, N. Rux, and G. Steidl. Wasserstein gradient flows for Moreau envelopes of f -divergences in reproducing kernel Hilbert spaces. Analysis and Applications, 24(01):21–65, 2026

    2026

  46. [54]

    J. Sun. Sensitivity analysis of the discrete-time algebraic Riccati equation. Linear algebra and its Applications, 275:595–615, 1998. 18

    1998

  47. [55]

    Takeda and T

    K. Takeda and T. Sakajo. Uniform error bounds of the ensemble transform Kalman filter for chaotic dynamics with multiplicative covariance inflation. SIAM/ASA Journal on Uncertainty Quantification, 12(4):1315–1335, 2024

    2024

  48. [56]

    Theodorou, J

    E. Theodorou, J. Buchli, and S. Schaal. Learning policy improvements with path integrals. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics , pages 828–835. JMLR Workshop and Conference Proceedings, 2010

    2010

  49. [57]

    E. Todorov. Linearly-solvable Markov decision problems. Advances in Neural Information Processing Systems, 19, 2006

    2006

  50. [58]

    Tolstikhin, O

    I. Tolstikhin, O. Bousquet, S. Gelly, and B. Schoelkopf. Wasserstein auto-encoders. arXiv preprint arXiv:1711.01558, 2017

    2017 arXiv

  51. [59]

    C. Villani. Topics in optimal transportation , volume 58 of Graduate Studies in Mathematics . American Mathematical Society, 2003

    2003

  52. [60]

    C. Villani. Optimal transport: old and new , volume 338 of Grundlehren der mathematischen Wissenschaften. Springer Berlin, Heidelberg, 1 edition, 2008

    2008

  53. [61]

    Wang and W

    Y. Wang and W. Li. Accelerated information gradient flow. Journal of Scientific Computing , 90(1):11, 2022

    2022

  54. [62]

    L. Xu, A. Korba, and D. Slepcev. Accurate quantization of measures via interacting particle- based optimization. In International Conference on Machine Learning , pages 24576–24595. PMLR, 2022. 19 Appendix First, in section A, we present the proofs for section 4, which are cum...

    2022

  55. [63]

    We now derive the ODE ( 7) for the mean and covariance parameters mt and Σt. Using integration by parts, we have ˙mt = Z Rd x ˙ρt(x) dx = Z Rd xr ρt(x)vt(x) dx = Z Rd ρt(x)(Stx + st) dx = Stmt + st, 20 and, similarly, ˙Σt = ∂t Z Rd (x mt)(x mt)Tρt(x) dx = Z Rd (x mt)(x mt)T∂tρ...

  56. [64]

    Solving for B yields Σ1 = λΣ0 (λI + κgΣ0)e−2λB κgΣ0 −1 ( ) e2λB = λΣ−1 0 + κgI λΣ−1 1 + κgI −1

    Now, we solve for B and b. Solving for B yields Σ1 = λΣ0 (λI + κgΣ0)e−2λB κgΣ0 −1 ( ) e2λB = λΣ−1 0 + κgI λΣ−1 1 + κgI −1 . Since Σ0 and Σ1 are symmetric positive definite matrices and λ, κg > 0, the principal matrix logarithm of the right-hand side exists and is unique, and w...

  57. [65]

    Now, we can evaluate the divergence formula from ( 8): DKW(µ j ν) = κg 2 tr(BΣ1) + 1 2 mT 1 (Bm1 + 2b) Z 1 0 κg 2 tr(BΣt) + 1 2 mT t (Bmt + 2b) dt. Splitting the sum into the βi = 0 and βi 6= 0 contributions and plugging in ( 18) yields κg 2 tr(BΣt) + 1 2 mT t (Bmt + 2b) = 1 2...

  58. [66]

    dy = m π d 2 Γ d 2 Z ∞ 0 r d 2 −1g(r) dr = m, where the first integral vanishes because the integrand is odd, as well as V[X] = Z Rd (x m)(x m)Tpm,Σ,g(x) dx = Σ 1 2 Z Rd yy Tg(kyk2

  59. [67]

    Let ΣY := R Rd yy Tg(kyk2

    dy Σ 1 2 . Let ΣY := R Rd yy Tg(kyk2

  60. [68]

    For any orthogonal matrix O 2 O(d), we have OΣY OT = Σ Y

    dy. For any orthogonal matrix O 2 O(d), we have OΣY OT = Σ Y . Hence, there exists a κg 2 R with ΣY = κg idd. We have κgd = tr(Σ Y ) = R Rd kyk2g(kyk2

  61. [69]

    whitening

    dy, so V[X] = κgΣ with κg = 1 d Z Rd kyk2g(kyk2) dy = 1 d π d 2 Γ d 2 Z ∞ 0 r d 2 g(r) dr. Lemma B.2 (Potential energy of elliptic distribution) . If µ E(m, Σ, g) and f (x) = 1 2 xTBx + bTx + c, then Z Rd f dµ = κg 2 tr(BΣ) + 1 2 mTBm + bTm + c. Proof. Using the “whitening” su...

  62. [70]

    dy + bTm + c = 1 2 tr Σ 1 2 BΣ 1 2 Z Rd yy Tg(kyk2

  63. [71]

    dy + 1 2 mTBm + bTm + c = κg 2 tr(BΣ) + 1 2 mTBm + bTm + c, using that R Rd yg(kyk2

  64. [72]

    normalized Stein metric

    dy = 1 and the linearity and cyclic permutation property of the trace together with yTAy = tr(Ayy T) and lemma B.1. 25 B.1 Calculating the vector fields for elliptic distributions We now verify the calculations from example 4.7 Lemma B.3 (Kalman-Wasserstein vector field for el...

  65. [73]

    E[x0] = µ0 and E[(x0 µ0)(x0 µ0)⊤] = Σ 0 for Σ0 0 and

  66. [74]

    ∞X k=0 γkℓ(xk, uk) x0 = x # = ℓ(x, π0(x)) + γE

    wk are i.i.d. with E[wk] = 0 and E[wkw⊤ k ] = Σ w for Σw 0. 4See also the expository blog post by Stephen Tu for an intuitive derivation: https://stephentu.github.io/ blog/control-theory/optimal-control/2017/12/09/discounted-infinite-horizon-lqr.html . 30 Consider the value fu...

    2017

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.