Continuous Policy and Value Iteration for Stochastic Control Problems and Its Convergence

Gu Wang; Qi Feng

REVIEW 4 major objections 7 minor 6 references

Continuous Policy and Value Iteration for Stochastic Control Problems and Its Convergence

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A continuous policy-value iteration updates control and value together and converges to the optimal stochastic control.

desk verdict Genuinely new continuous policy-value iteration framework, but the headline Wasserstein rate theorem has a dimensional bug and the monotonicity assumptions are largely unverified. read the letter →

arxiv 2506.08121 v2 pith:4CT2BJIX submitted 2025-06-09 math.OC cs.LG

Qi Feng , Gu Wang This is my paper

classification math.OCcs.LG MSC 93E2093E3560H10

keywords stochasticcontrolrelaxedpolicyiterationvalueLangevindynamicsHamilton-Jacobi-BellmanequationimprovementWassersteindistance

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a continuous policy-value iteration algorithm for infinite-horizon stochastic control problems, in which the value-function approximation and the policy are updated simultaneously along an iteration variable $\tau$, rather than in discrete rounds. The policy update is a Langevin-type stochastic differential equation in the control variable, and the value update is a coupled differential equation driven by the current Hamiltonian. The authors prove that the value sequence increases along $\tau$ (policy improvement), that the limiting policy is the optimal relaxed control, and that the limiting value solves the Hamilton-Jacobi-Bellman equation. Under a sign condition called the monotonicity condition on the Hamiltonian, they prove exponential convergence of the policy distribution to its Gibbs invariant distribution in Wasserstein distance. If correct, the method removes the need to evaluate the value function of each policy iterate and connects stochastic control to Langevin sampling.

What carries the argument

The load-bearing object is the coupled dynamics (28) for relaxed control and (37) for classical control: the value $v^\tau$, its derivatives $v^\tau_x$ and $v^\tau_{xx}$ follow ODEs whose drifts are the expected Hamiltonian minus $\beta v^\tau$, while the control $u_\tau$ follows $du_\tau = \nabla_u H(x,u_\tau,v^\tau_x,v^\tau_{xx})d\tau+\sqrt{2\lambda}dB_\tau$. The invariant distribution of this Langevin diffusion at the optimal value is the Gibbs measure $e^{H/\lambda}/\int e^{H/\lambda}du$, which is precisely the optimal relaxed control. The proof of policy improvement uses Itô's formula along the coupled processes; the exponential convergence proof couples two copies of the dynamics with the same Brownian motion and uses the monotonicity conditions to obtain a contraction with rate $\beta$ on value differences and rate $\kappa(x)$ on control differences.

What would settle it

Run the coupled dynamics on a linear-quadratic model with small values of $|A|$, $M$, or $N$, where the paper's verification of Assumption 20 does not apply, and measure the Wasserstein distance between $\pi_\tau$ and $\pi^*$ together with $v^\tau$ along $\tau$. If the distance does not decay exponentially, or if $v^\tau$ ever decreases, the monotonicity condition is violated and the advertised convergence rate fails.

Watch

Extended reading notes

Core claim

The central claim is that value iteration and policy iteration can be run simultaneously as continuous dynamics: an ODE for the value function and an overdamped Langevin SDE for the control, coupled through the Hamiltonian. For relaxed (entropy-regularized) control, the limiting policy is the Gibbs distribution $e^{H/\lambda}/\int e^{H/\lambda}du$, where $H$ is the Hamiltonian evaluated at the limiting value function, and the limiting value solves the HJB equation (Proposition 10). For classical control, under concavity or an additional condition, the expected value is nondecreasing in $\tau$, and in the deterministic $\lambda=0$ case the control converges to the maximizer of the limiting Hamiltonian (Proposition 17). Under Assumptions 20 and 25 the authors prove the exponential Wasserstein convergence rate $W_2(\pi_\tau,\tilde\pi_\tau)\le e^{-2\kappa(x)\tau}\mathbb{E}[\|u_0-\tilde u_0\|^2]+\frac{e^{-2\beta\tau}-e^{-2\kappa(x)\tau}}{2(\kappa(x)-\beta)}C_0$ using synchronous coupling. In the linear-quadratic example, monotonicity conditions (II) and (III) are verified for sufficiently large values of $|A|$, $M$, and $N$.

Load-bearing premise

The exponential convergence results rest on the monotonicity condition that the expected change in the Hamiltonian and the expected change in the value function (or its derivatives) always have opposite signs; the paper verifies this only for a linear-quadratic example and only when $|A|$, $M$, and $N$ are sufficiently large.

Editorial extensions

If this is right

No PDE solve per iteration: the algorithm updates $v$ and $\pi$ simultaneously, so each step only needs current derivatives, making high-dimensional implementation plausible.
The same continuous iteration handles classical controls with controlled diffusion coefficients, which discrete relaxed-control policy iteration cannot easily handle.
Under the monotonicity condition, value differences decay at rate $e^{-2\beta\tau}$ and the policy's Wasserstein distance decays at a rate governed by $\kappa(x)$.
The Gibbs invariant structure connects stochastic control to Langevin-based distribution sampling, enabling sampling and non-convex-learning tools.
With $\lambda\to 0$ annealing, the classical-control dynamics form a continuous bridge to non-convex optimization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The monotonicity conditions (MC II and MC III) are stated abstractly and are verified only for a linear-quadratic model in a large-parameter regime; whether they hold for broader classes, such as non-quadratic costs or state-dependent diffusion, remains open.
The analysis treats the continuous iteration directly; any practical discrete-time Euler-Maruyama discretization of (28) or (37) will introduce discretization error, and the paper leaves that error analysis for future work.
The authors flag the policy-inhomogeneous nature of the diffusion (the Hamiltonian changes with $\tau$) as an open mathematical problem; proving convergence for general time-inhomogeneous Langevin dynamics would directly strengthen this framework.
A testable extension is to run the classical dynamics with an annealing schedule $\lambda(\tau)\to 0$ and check whether the coupled system exits suboptimal wells and reaches the global optimum, connecting the method to simulated-annealing theory.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The core idea is worth taking seriously: a genuinely new iteration-continuous coupled dynamics for value and policy, with Langevin-type updates. It is not in the cited discrete policy-iteration or gradient-flow literature. The policy improvement result (Proposition 5) is coherent, and the connection to mean-field Langevin sampling is a real bridge. Credit where due: the framework is new, the relaxed-control-to-sampling link is plausible, and the authors are honest that the convergence analysis relies on monotonicity conditions they do not derive.

Now the soft spots. The stress-test note is right. Proposition 22 states W2(πτ,π̃τ) ≤ e^{-2κτ}E||u0−ũ0||² + (e^{-2βτ}−e^{-2κτ})/(2(κ−β))C0. The proof bounds ||uτ−ũτ||² via Gronwall and then relabels the left side as W2 without taking a square root. At τ=0 the claimed inequality reduces to W2(π0,π̃0) ≤ E||u0−ũ0||², which fails for nearby Gaussians with the synchronous coupling: W2=ε while E||u0−ũ0||²=ε². What the proof actually gives is a squared-Wasserstein bound, with a rate governed by min(κ,β). That is still a usable rate, but Proposition 22 as displayed is false. Fixable, but it is the main advertised convergence claim.

Second, Proposition 9 has an unjustified step from L2-type dissipation to pointwise convergence of ∇u ln πτ. The argument may be repairable with more regularity, but as written it does not go through.

Third, the monotonicity Assumptions 20 and 25 are load-bearing. They are assumed, not derived from model data. The LQ example verifies only conditions (II) and (III), for large parameter regimes, and does not verify (I) or the structural condition (30) used in Proposition 22. This limits the paper's reach. To be fair, the assumptions are stated openly, and the policy improvement and HJB verification provide some independent support for the framework.

Overall: the central framework is novel; the convergence-rate section is not yet sound. A serious referee should engage with it because the bugs look fixable and the idea deserves a proper treatment. The paper is primarily for stochastic control and RL theorists interested in continuous-time policy iteration. I would not cite the current rate theorem in my own work until it is corrected.

Recommendation: send to peer review, with a request to fix the Wasserstein/square-root issue, tighten Proposition 9, and either verify or weaken the monotonicity conditions.

Referee Report

4 major / 7 minor

Summary. The paper proposes a continuous-time coupled policy-value iteration for infinite-horizon stochastic control problems. The policy is updated by a Langevin-type SDE in the control variable while the value function, together with its derivatives, is updated simultaneously by an ODE in the iteration parameter. The framework is developed for entropy-regularized relaxed control problems in Section 3 and for classical control problems in Section 4. Under monotonicity conditions on the Hamiltonian, the paper claims exponential Wasserstein convergence of the policy distribution (Propositions 22 and 27) and optimality of the limiting policy via HJB verification (Proposition 10). A linear-quadratic example is analyzed in Section 6 as a partial verification of the assumptions.

Significance. If the main results were rigorously established, this paper would make a useful contribution to the continuous-time reinforcement-learning and stochastic-control literature: it offers a genuinely simultaneous continuous update of value and policy, connects relaxed control to mean-field Langevin sampling, and provides a rate statement for the coupled dynamics. The policy-improvement derivations in Section 3 are elaborate, and the linear-quadratic example is instructive. However, the central convergence and optimality claims are currently not established as written: Proposition 22 contains a false Wasserstein bound, Proposition 9 has a pointwise-convergence gap, and the monotonicity assumptions are not verified from primitive model data except in a special example. No code or numerical experiments are provided, and the authors are honest that annealing, discretization, and the policy-inhomogeneous diffusion analysis are left to future work.

major comments (4)

[Section 5.1, Proposition 22] The displayed Wasserstein bound is false as stated. The proof applies Gronwall to d/dτ E||uτ−ũτ||² and obtains a bound on the squared distance, and then the last line substitutes W2(πτ,π̃τ) on the left-hand side without extracting a square root. At τ=0 the bound reduces to W2(π0,π̃0) ≤ E||u0−ũ0||², which fails, for example, in the scalar LQ setting with π0=N(0,σ²), π̃0=N(ε,σ²) and the synchronous coupling Ũ0=U0+ε, where W2(π0,π̃0)=ε while E||u0−ũ0||²=ε² for ε∈(0,1). The correct conclusion of the proof is a squared-Wasserstein bound, and the exponential rate that follows is min(κ,β), not 2κ. Since Proposition 22 is the main advertised rate theorem of Section 5, this is a load-bearing error that must be repaired.
[Section 3, Proposition 9] The proof asserts that because the expectation gτ converges and the dissipation term is nonnegative, ∇_u ln πτ converges pointwise to ∇_u H(x,u,v*_x,v*_xx)/λ. Convergence of gτ gives only an L2-type dissipation estimate along the trajectory after (19); it does not imply pointwise convergence of the score ∇_u ln πτ. The subsequent representation of πτ and the identification of π* as the Gibbs density therefore lack proof. Since Proposition 10 uses Proposition 9 to verify the HJB equation, the optimality conclusion is not established as written. Closing this gap requires additional compactness or stronger uniform estimates, which are not present in the manuscript.
[Assumptions 20 and 25; Section 6] The monotonicity conditions MC I–MC VI are assumed directly on the coupled iterates and are not derived from primitive conditions on f, b, and σ. They are load-bearing: Lemma 21 and Lemma 26 use them to contract vτ, vτx, and vτxx, and those contractions feed into Propositions 22 and 27. For a general model the paper gives no criterion by which a user can check these conditions. In the LQ example, Proposition 33 verifies only conditions (II) and (III), and the argument relies on the auxiliary ODE system for Yτ and Zτ with boundedness claims that are asserted rather than proved. The text also states that 'the first term is negative for sufficiently large τ as shown above' although the preceding computation establishes only that ∂xxE[Hτ]−∂xxE[H̃τ]=0. The example therefore does not provide a complete verification of the monotonicity hypotheses.
[Section 6, first paragraph] The LQ example explicitly violates the boundedness Assumption 2, since f, b, and σ are unbounded in x, while the propositions of Section 3 are proved under that assumption. The text says that 'all the results in that section, in particular the policy improvement and convergence results, still hold while we omit the proofs.' Omitting the verification of the standing assumptions for the example leaves the example's support of the general claims incomplete. The paper should either extend the proofs to the unbounded-coefficient case or clearly present the example as heuristic rather than as a full verification.

minor comments (7)

[Notation throughout] The superscript and subscript conventions are inconsistent, especially for ρτ, πτ−s, and vτ(t,x); please harmonize the notation so that the time direction of the state SDE and the iteration direction of the policy are unambiguous.
[Assumption 8] Assumption 8(ii) assumes local boundedness of ∇_u ln πτ uniformly in τ, but the proof of Proposition 9 needs a stronger control to obtain pointwise convergence; the exact regularity requirement should be stated explicitly.
[Definition 13] In the classical control case, the value function in (23) depends on a realization of the control path, but the probability space and the conditional expectation are not fully specified; the statement should clarify the dependence on the Brownian motion driving the policy iterations.
[Proposition 17, footnote] The footnote contains a stray superscript and an unproved implication about concavity of H; please correct the typo and provide the argument or remove the footnote.
[Remark 24] The remark claims that the continuous dynamics provide theoretical guarantees for discrete algorithm design, but no discretization or finite-step convergence analysis is provided; this should be rephrased as a research program rather than a guarantee.
[Proof of Proposition 22] The symbol ε(x) is used first as the maximum of the Young inequality constants and later as the minimum; this inconsistent notation should be fixed, and the dimensions in the matrix inequality (30) should be specified precisely.
[Typos] There are numerous typographical errors (e.g., 'sovle', 'startegies', '2000 AMS Mathematics subject classification') that should be corrected in a revision.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation chain: the main convergence results are conditional on explicitly stated monotonicity assumptions and are derived by Itô-calculus and Gronwall arguments, not by renaming inputs; the few self-citations are contextual rather than load-bearing.

full rationale

The paper's central derivation is self-contained and does not reduce to its own inputs. Proposition 5 derives the policy improvement inequality v^τ(0,x) ≤ v^{τ+h}(0,x) by an explicit Itô/Fokker-Planck computation ending in a non-positive squared-integral term; the conclusion is not assumed. Lemma 6 derives the continuous value dynamics from the definition of ρ^τ, using the identity v^{τ+h}(h,x)=v^τ(0,x), which is a consequence of Definition 3 rather than a hidden assumption. Propositions 9 and 10 pass to the limit and verify the HJB equation using the explicit Gibbs form and standard verification arguments; optimality is not presupposed. The exponential convergence estimates in Lemma 21 and Lemma 26 are conditional on the monotonicity conditions in Assumptions 20 and 25, which are stated as explicit hypotheses, not as fitted parameters or as renamed conclusions. Proposition 22 and Proposition 27 additionally assume the strong-convexity-type condition (30), again as an explicit hypothesis. These assumptions are restrictive and may fail for general models, but that is a scope limitation, not circularity. The self-citations (Feng et al. 2024; Deng et al. 2020; Bayraktar et al. 2024; Ma et al. 2024a,b) appear in the introduction and in connections to Langevin MCMC literature; they are not used to justify the main theorems. One mathematical concern is that the last step of Proposition 22 replaces E||u^τ−ũ^τ||² with W2(π^τ,π̃^τ) without taking a square root, so the displayed bound is dimensionally inconsistent; however, this is a correctness issue in the proof rather than a circular reduction of the claimed result to its assumptions.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The central results rest on a growing set of assumptions: bounded smooth coefficients, finite value, smoothness of v^tau, exponential integrability of Gibbs weights, uniform control of nabla ln pi^tau, and especially the Hamiltonian monotonicity inequalities (MC I-VI). These are not established from the model data; the LQ example verifies only a subset for large parameters. The paper introduces no new particles, forces, dimensions, or conserved quantities.

free parameters (2)

kappa(x) (dissipativity rate)
Introduced in condition (30) as an assumed positive constant, kappa != beta, controlling the contraction of the coupled control difference. No method to compute it from model data is given.
epsilon(x) (Young's inequality slack)
Introduced in condition (30) and in the proof of Proposition 22 as a positive constant used to split cross terms. Its existence is assumed, not constructed.

assumptions (6)

domain assumption Assumption 2: f, b, sigma sigma^T are bounded, C^2, with bounded derivatives, and sigma sigma^T is bounded away from zero; the value function V is finite.
Used throughout for Ito calculations, martingale arguments, and convergence proofs in Sections 3 and 5.
domain assumption Admissible relaxed controls are continuous probability densities on R^n (Definition 1).
The policy pi^tau is required to have a density with respect to Lebesgue measure, which supports the Gibbs-form representation and Fokker-Planck calculations.
ad hoc to paper Assumption 20 / 25: Monotonicity conditions (MC I)-(MC VI) on the Hamiltonian cross-terms.
These are the load-bearing inequalities used in Lemma 21 and Lemma 26 to get exponential contraction of value functions. They are not derived from the problem data and are only partially verified in the LQ example, where Proposition 33 verifies (II) and (III) for large parameters.
ad hoc to paper Assumption 8: exponential integrability of e^{H/lambda} and local boundedness of nabla_u ln pi^tau uniformly in tau.
Needed in Proposition 9 to identify the limit of pi^tau as a Gibbs distribution. Stated without verification for general models.
ad hoc to paper Condition (30): uniform negative-definiteness of the Hessian-type matrix with constants kappa(x), epsilon(x), and the uniform bound L in (31).
Used in Proposition 22 and Proposition 27 to obtain the Wasserstein contraction rate. Existence of such constants is assumed.
standard math Standard Ito calculus, Fokker-Planck equation, Schauder interior estimates, Arzela-Ascoli, and Gronwall's inequality.
Background results used in the proofs without formalization.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Continuous Policy and Value Iteration for Stochastic Control Problems and Its Convergence." pith.science (2026). https://pith.science/paper/4CT2BJIX

@misc{pith2026250608121,
  author       = {Pith},
  title        = {Pith review of: Continuous Policy and Value Iteration for Stochastic Control Problems and Its Convergence},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4CT2BJIX}},
  note         = {Machine review of arXiv:2506.08121}
}

read the original abstract

We introduce a continuous policy-value iteration algorithm where the approximations of the value function of a stochastic control problem and the optimal control are simultaneously updated through Langevin-type dynamics. This framework applies to both the entropy-regularized relaxed control problems and the classical control problems, with infinite horizon. We establish policy improvement and demonstrate convergence to the optimal control under the monotonicity condition of the Hamiltonian. By utilizing Langevin-type stochastic differential equations for continuous updates along the policy iteration direction, our approach enables the use of distribution sampling and non-convex learning techniques in machine learning to optimize the value function and identify the optimal control simultaneously.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

6 extracted references · 4 canonical work pages

[1]

and Souganidis, P

Barles, G. and Souganidis, P. E. (1991), ‘Convergence of approximation schemes for fully nonlinear second order equations’,Asymptotic analysis4(3), 271–283. Bayraktar, E., Feng, Q. and Li, W. (2024), ‘Exponential entropy dissipation for weakly self-consistent vlasov–fokker–planck equations’,Journal of Nonlinear science34(1),

work page 1991
[7]

(1954), ‘The theory of dynamic programming’,Bulletin of the American Math- ematical Society60(6), 503–515

Bellman, R. (1954), ‘The theory of dynamic programming’,Bulletin of the American Math- ematical Society60(6), 503–515. Bellman, R. (1966), ‘Dynamic programming’,science153(3731), 34–37. Bertsekas, D. (2012),Dynamic programming and optimal control: Volume I, Vol. 4, Athena scientific. Bertsekas, D. P. (2015), ‘Value and policy iterations in optimal control...

work page 1954
[28]

and Wang, S

Chen, F., Ren, Z. and Wang, S. (2022), ‘Uniform-in-time propagation of chaos for mean field langevin dynamics’,arXiv preprint arXiv:2212.03050. Cheng, X., Chatterji, N. S., Bartlett, P. L. and Jordan, M. I. (2018), Underdamped langevin mcmc: A non-asymptotic analysis,in‘Conference on learning theory’, PMLR, pp. 300–

arXiv 2022
[323]

(2022), ‘Mean-field langevin dynamics: Exponential convergence and annealing’, arXiv preprint arXiv:2202.01009

Chizat, L. (2022), ‘Mean-field langevin dynamics: Exponential convergence and annealing’, arXiv preprint arXiv:2202.01009. Dai, M., Dong, Y. and Jia, Y. (2023), ‘Learning equilibrium mean-variance strategy’,Math- ematical Finance33(4), 1166–1212. 34 Dalalyan, A. S. and Riou-Durand, L. (2020), ‘On sampling from a log-concave density using kinetic langevin ...

arXiv 2022
[509]

Bertsekas, D. P. et al. (2011), ‘Dynamic programming and optimal control 3rd edition, volume ii’,Belmont, MA: Athena Scientific1. Boltyanski, V., Gamkrelidze, R., Mishchenko, E. and Pontryagin, L. (1960), ‘The maxi- mum principle in the theory of optimal processes of control’,IF AC Proceedings Volumes 1(1), 464–469. ˇCern` y, V. (1985), ‘Thermodynamical a...

work page Pith review arXiv 2011
[8510]

Convergence of Policy Iteration for Entropy-Regularized Stochastic Control Problems

Hu, K., Ren, Z., ˇSiˇ ska, D. and Szpruch, L. (2021), ‘Mean-field langevin dynamics and energy landscape of neural networks’,Annales de l’Institut Henri Poincare (B) Probabilites et statistiques57(4), 2043–2065. Huang, Y., Wang, Z. and Zhou, Z. (2022), ‘Convergence of policy improvement for entropy- regularized stochastic control problems’,arXiv preprint ...

work page Pith review arXiv 2021

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

and Souganidis, P

Barles, G. and Souganidis, P. E. (1991), ‘Convergence of approximation schemes for fully nonlinear second order equations’,Asymptotic analysis4(3), 271–283. Bayraktar, E., Feng, Q. and Li, W. (2024), ‘Exponential entropy dissipation for weakly self-consistent vlasov–fokker–planck equations’,Journal of Nonlinear science34(1),

work page 1991

[2] [7]

(1954), ‘The theory of dynamic programming’,Bulletin of the American Math- ematical Society60(6), 503–515

Bellman, R. (1954), ‘The theory of dynamic programming’,Bulletin of the American Math- ematical Society60(6), 503–515. Bellman, R. (1966), ‘Dynamic programming’,science153(3731), 34–37. Bertsekas, D. (2012),Dynamic programming and optimal control: Volume I, Vol. 4, Athena scientific. Bertsekas, D. P. (2015), ‘Value and policy iterations in optimal control...

work page 1954

[3] [28]

and Wang, S

Chen, F., Ren, Z. and Wang, S. (2022), ‘Uniform-in-time propagation of chaos for mean field langevin dynamics’,arXiv preprint arXiv:2212.03050. Cheng, X., Chatterji, N. S., Bartlett, P. L. and Jordan, M. I. (2018), Underdamped langevin mcmc: A non-asymptotic analysis,in‘Conference on learning theory’, PMLR, pp. 300–

arXiv 2022

[4] [323]

(2022), ‘Mean-field langevin dynamics: Exponential convergence and annealing’, arXiv preprint arXiv:2202.01009

Chizat, L. (2022), ‘Mean-field langevin dynamics: Exponential convergence and annealing’, arXiv preprint arXiv:2202.01009. Dai, M., Dong, Y. and Jia, Y. (2023), ‘Learning equilibrium mean-variance strategy’,Math- ematical Finance33(4), 1166–1212. 34 Dalalyan, A. S. and Riou-Durand, L. (2020), ‘On sampling from a log-concave density using kinetic langevin ...

arXiv 2022

[5] [509]

Bertsekas, D. P. et al. (2011), ‘Dynamic programming and optimal control 3rd edition, volume ii’,Belmont, MA: Athena Scientific1. Boltyanski, V., Gamkrelidze, R., Mishchenko, E. and Pontryagin, L. (1960), ‘The maxi- mum principle in the theory of optimal processes of control’,IF AC Proceedings Volumes 1(1), 464–469. ˇCern` y, V. (1985), ‘Thermodynamical a...

work page Pith review arXiv 2011

[6] [8510]

Convergence of Policy Iteration for Entropy-Regularized Stochastic Control Problems

Hu, K., Ren, Z., ˇSiˇ ska, D. and Szpruch, L. (2021), ‘Mean-field langevin dynamics and energy landscape of neural networks’,Annales de l’Institut Henri Poincare (B) Probabilites et statistiques57(4), 2043–2065. Huang, Y., Wang, Z. and Zhou, Z. (2022), ‘Convergence of policy improvement for entropy- regularized stochastic control problems’,arXiv preprint ...

work page Pith review arXiv 2021