pith. sign in

arxiv: 1907.04209 · v1 · pith:5TLFU3Y7new · submitted 2019-07-06 · 🧮 math.OC · math.PR

Maximum principle for stochastic optimal control problem of finite state forward-backward stochastic difference systems

Shailin Ji , Haodong Liu This is my paper

Pith reviewed 2026-05-25 02:06 UTC · model grok-4.3

classification 🧮 math.OC math.PR
keywords stochastic optimal controlforward-backward stochastic difference equationsmaximum principlefinite state processadjoint difference equationconvex control domain
0
0 comments X

The pith

A maximum principle is established for optimal control of forward-backward stochastic difference systems driven by finite-state processes.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper derives necessary conditions for optimality in stochastic control problems governed by forward-backward stochastic difference equations with uncertainty from a discrete-time finite-state process. It treats both partially coupled and fully coupled cases. An adjoint difference equation is obtained through a tailored product rule representation together with a suitable backward stochastic difference equation formulation. The resulting maximum principle applies when the control domain is convex. This supplies a tool for characterizing optimal controls without continuous white-noise assumptions.

Core claim

For forward-backward stochastic difference systems driven by a finite-state discrete-time process, an adjoint difference equation can be deduced by means of an appropriate product rule representation and an appropriate formulation of the backward stochastic difference equation; the maximum principle for the optimal control problem then follows when the control domain is convex. The result covers both the partially coupled and the fully coupled cases.

What carries the argument

The adjoint difference equation, obtained from the product rule representation and the backward stochastic difference equation formulation, which encodes the necessary optimality condition.

If this is right

  • The optimal control at each time maximizes a Hamiltonian expression that incorporates the adjoint process.
  • The same adjoint construction yields the maximum principle in both the partially coupled and the fully coupled settings.
  • The convexity of the control domain permits the use of first-order variational arguments to obtain the necessary condition.
  • The maximum principle supplies a verifiable necessary condition that any candidate optimal control must satisfy.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same derivation route may extend to other discrete-time stochastic systems whose product rule can be expressed in analogous form.
  • Because the driving process takes only finitely many values, the adjoint equation may admit direct recursive computation for small state spaces.
  • Taking a suitable scaling limit of the time step could recover known continuous-time maximum principles for forward-backward stochastic differential equations.

Load-bearing premise

The chosen product rule representation and backward stochastic difference equation formulation remain valid and permit derivation of the adjoint for the partially or fully coupled systems driven by the finite-state process.

What would settle it

A concrete convex-control example of a fully coupled forward-backward stochastic difference system in which a candidate optimal control fails to satisfy the maximum condition stated by the derived adjoint equation.

read the original abstract

In this paper, we study the maximum principle for stochastic optimal control problems of forward-backward stochastic difference systems (FBS{\Delta}Ss) where the uncertainty is modeled by a discrete time, finite state process, rather than white noises. Two types of FBS{\Delta}Ss are investigated. The first one is described by a partially coupled forward-backward stochastic difference equation (FBS{\Delta}E) and the second one is described by a fully coupled FBS{\Delta}E. By adopting an appropriate representation of the product rule and an appropriate formulation of the backward stochastic difference equation (BS{\Delta}E), we deduce the adjoint difference equation. Finally, the maximum principle for this optimal control problem with the control domain being convex is established.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 2 minor

Summary. The paper derives the stochastic maximum principle for optimal control problems governed by forward-backward stochastic difference systems (FBSΔSs) driven by a discrete-time finite-state process. It treats both partially coupled and fully coupled FBSΔEs, obtains the adjoint difference equation via a chosen product-rule representation and backward stochastic difference equation (BSΔE) formulation, and establishes the maximum principle when the control domain is convex.

Significance. If the derivations are correct, the work extends the stochastic maximum principle to discrete-time settings with finite-state uncertainty rather than white noise, covering both partial and full coupling. This could support applications in discrete stochastic systems where continuous-time or Gaussian noise models are inappropriate. The explicit handling of the product rule and BSΔE formulation for the adjoint is a technical contribution if the steps are fully rigorous.

minor comments (2)
  1. The abstract states the main results but does not preview any key equations or the structure of the adjoint equation; adding a brief indication of the form of the adjoint (e.g., the dependence on the Hamiltonian or the terminal condition) would improve readability without lengthening the abstract substantially.
  2. Notation for the finite-state process and the coupling between forward and backward equations should be introduced with a short table or explicit list of processes and their dimensions in §2 to aid readers unfamiliar with FBSΔE literature.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the careful summary of our manuscript and the positive recommendation of minor revision. No specific major comments were raised in the report.

Circularity Check

0 steps flagged

No significant circularity; derivation is self-contained standard adaptation

full rationale

The paper derives a maximum principle for convex-control stochastic optimal control of partially and fully coupled FBSΔSs driven by finite-state discrete-time processes. The key steps—selecting a product-rule representation and BSΔE formulation to obtain the adjoint equation—are presented as methodological choices within established stochastic control techniques, not as fitted parameters or self-referential definitions. No load-bearing step reduces by construction to the paper's own inputs, no self-citation chain is invoked to justify uniqueness or ansatz, and the result is not a renaming of a known empirical pattern. The derivation remains independent of the target maximum principle itself.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies no explicit free parameters, axioms, or invented entities; ledger left empty.

pith-pipeline@v0.9.0 · 5655 in / 1004 out tokens · 21084 ms · 2026-05-25T02:06:16.272280+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 1 canonical work pages · 1 internal anchor

  1. [1]

    for any (x, y, ˜z, u) ∈ Rm× Rn× Rn×(d−1)× Rr, ϕ (·, ·, x, y, ˜z, u) is {Ft}-adapted process

    ϕ is an adapted map, i.e. for any (x, y, ˜z, u) ∈ Rm× Rn× Rn×(d−1)× Rr, ϕ (·, ·, x, y, ˜z, u) is {Ft}-adapted process

  2. [2]

    Also, for t = T , f is independent of ˜z at time T

    for any t ∈ { 0, 1, ..., T} and ω ∈ Ω, ϕ (ω, t, ·, ·, ·, ·) is continuously differentiable with respect to x, y, ˜z, u, and ϕx, ϕy, ϕ˜zi, ϕu are uniformly bounded. Also, for t = T , f is independent of ˜z at time T . Set λ = (x, y, z) , A (t, λ; u) = ( − f (t, λ; u) , b (t, λ; u) , σ (t, λ; u) E [ Mt+1M ∗ t+1|Ft ]) and |λ|= |x|+ |y|+ ⏐ ⏐ ⏐z ˜I ⏐ ⏐ ⏐ , |A (...

  3. [3]

    It is easy to check that   ˜bx (t − 1) − bx (t − 1)    → 0 and   [˜σix (t − 1) − σix (t − 1)] ˜I    → 0 as ε → 0

    ⏐ ⏐X ε t−1 − ¯Xt−1 − ξt−1 ⏐ ⏐2 +   ˜bx (t − 1) − bx (t − 1)    2 |ξt−1|2 + m∑ i=1   [˜σix (t − 1) − σix (t − 1)] ˜I    2 |ξt−1|2 ] . It is easy to check that   ˜bx (t − 1) − bx (t − 1)    → 0 and   [˜σix (t − 1) − σix (t − 1)] ˜I    → 0 as ε → 0. Since ˜bx (t − 1) and ˜σix (t − 1) are bounded, by the estimation (3.5), we have lim ε...

  4. [4]

    Bensoussan, A. (1982). Lectures on stochastic control. In N onlinear filtering and stochastic control (pp. 1-62). Springer, Berlin, Heidelberg. 24

    1982

  5. [5]

    R., Cialenco, I., & Chen, T

    Bielecki, T. R., Cialenco, I., & Chen, T. (2015). Dynamic conic financ e via backward stochastic difference equations. SIAM Journal on Financial Mathematics, 6(1), 1068-1 122

    2015

  6. [6]

    Bismut, J. M. (1978). An introductory approach to duality in opt imal stochastic control. SIAM review, 20(1), 62-78

    1978

  7. [7]

    N., & Elliott, R

    Cohen, S. N., & Elliott, R. J. (2008). Solutions of backward stoch astic differential equations on Markov chains. Communications on stochastic analysis, 2(2), 251-262

    2008

  8. [8]

    N., & Elliott, R

    Cohen, S. N., & Elliott, R. J. (2010). A general theory of finite st ate backward stochastic difference equations. Stochastic Processes and their Applications, 120(4), 442-466

    2010

  9. [9]

    N., & Elliott, R

    Cohen, S. N., & Elliott, R. J. (2011). Backward stochastic differe nce equations and nearly time-consistent nonlinear expectations. SIAM Journal on Control and Optimization , 49(1), 125-139

    2011

  10. [10]

    Dokuchaev, N., & Zhou, X. Y. (1999). Stochastic controls with t erminal contingent conditions. Journal of Mathematical Analysis and Applications, 238(1), 143-165

    1999

  11. [11]

    Eberlein, E., Gehrig, T., & Madan, D. B. (2011). Pricing to accepta bility: With applications to valuing one’s own credit risk

    2011

  12. [12]

    El Karoui, N., & Huang, S. J. (1997). A general result of existen ce and uniqueness of backward stochastic differential equations. Pitman Research Notes in Mathematics Serie s, 27-38

    1997

  13. [13]

    Hu, M., Ji, S., & Xue, X. (2018). A global stochastic maximum princ iple for fully coupled forward- backward stochastic systems. arXiv preprint arXiv:1803.02109

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  14. [14]

    Hu, M., Ji, S., & Xue, X. (2018). A note on the global stochastic m aximum principle for fully coupled forward-backward stochastic systems. arXiv preprint arXiv:181 2.10469

    2018

  15. [15]

    & Liu, H

    Ji, S. & Liu, H. (2018). Fully coupled finite state forward-backw ard stochastic difference equations, arXiv

    2018

  16. [16]

    Kushner, H. J. (1972). Necessary conditions for continuous parameter stochastic optimization problems. SIAM Journal on Control, 10(3), 550-565

    1972

  17. [17]

    E., & Zhou, X

    Lim, A. E., & Zhou, X. Y. (2001). Linear-quadratic control of b ackward stochastic differential equations. SIAM journal on control and optimization, 40(2), 450-474

    2001

  18. [18]

    Lin, Y., & Yang, H. (2014). Discrete-Time BSDEs with Random Ter minal Horizon. Stochastic Analysis and Applications, 32(1), 110-127

    2014

  19. [19]

    Lin, X., & Zhang, W. (2015). A maximum principle for optimal contr ol of discrete-time stochastic systems with multiplicative noise. IEEE Transactions on Automatic Co ntrol, 60(4), 1121-1126

    2015

  20. [20]

    Madan, D. B. (2010). Conserving capital by adjusting deltas f or gamma in the presence of skewness. Journal of Risk and Financial Management, 3(1), 1-25. 25

    2010

  21. [21]

    Peng, S. (1990). A general stochastic maximum principle for op timal control problems. SIAM Journal on control and optimization, 28(4), 966-979

    1990

  22. [22]

    Peng, S. (1993). Backward stochastic differential equations and applications to optimal control. Applied Mathematics and Optimization, 27(2), 125-144

    1993

  23. [23]

    Pardoux, E., & Peng, S. (1990). Adapted solution of a backwar d stochastic differential equation. Systems & Control Letters, 14(1), 55-61

    1990

  24. [24]

    Schroder, M., & Skiadas, C. (1999). Optimal consumption and p ortfolio selection with stochastic dif- ferential utility. Journal of Economic Theory, 89(1), 68-126

    1999

  25. [25]

    Shi, Y., & Zhu, Q. (2013). Partially observed optimal controls of forward-backward doubly stochastic systems. ESAIM: Control, Optimisation and Calculus of Variations, 1 9(3), 828-843

    2013

  26. [26]

    Stadje, M. (2010). Extending dynamic convex risk measures f rom discrete time to continuous time: A convergence approach. Insurance: Mathematics and Economics , 47(3), 391-404

    2010

  27. [27]

    Williams, N. (2009). On dynamic principal-agent problems in continu ous time. University of Wisconsin, Madison

    2009

  28. [28]

    Wu, Z. (1998). Maximum principle for optimal control problem of fully coupled forward-backward stochastic systems. Systems Science and Mathematical sciences , 3, 249-259

    1998

  29. [29]

    Xu, W. (1995). Stochastic maximum principle for optimal contro l problem of forward and backward system. The ANZIAM Journal, 37(2), 172-185

    1995

  30. [30]

    Yong, J. (2010). Optimality variational principle for controlled f orward-backward stochastic differential equations with mixed initial-terminal conditions. SIAM Journal on Con trol and Optimization, 48(6), 4119-4156

    2010

  31. [31]

    Zhang, L., & Shi, Y. (2011). Maximum principle for forward-back ward doubly stochastic control systems and applications. ESAIM: Control, Optimisation and Calculus of Variat ions, 17(4), 1174-1197. 26

    2011