Pith. sign in

REVIEW 3 major objections 4 minor 56 references

Stability Analysis of an Integrated Multistage Stochastic Programming and Markov Decision Process Problem

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proves that perturbing either the endogenous or exogenous randomness in an integrated MSP-MDP model changes the optimal value and optimal solution set only by weighted sums of stagewise probability-metric distances.

desk verdict New integrated MSP-MDP model with careful stability bounds, but the 'WLOG' independence of endogenous noise is false and breaks Theorem 4.1 as stated. read the letter →

arxiv 2509.22194 v2 pith:ZS7EPTOT submitted 2025-09-26 math.OC

Zhiyao Yang , Zhiping Chen , Huifu Xu This is my paper · ORCID
classification math.OC MSC 90C1590C40
keywords multistagestochasticprogrammingMarkovdecisionprocessstabilityanalysisKantorovichmetricFortet-Mourierendogenousuncertaintyexogenousnestedreformulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that an integrated model combining Markov decision processes and multistage stochastic programming is quantitatively stable under distributional perturbations of both types of randomness. It derives a nested reformulation, then proves that the optimal value and optimal solution set change by amounts controlled by stagewise Kantorovich (or Fortet-Mourier) distances between original and perturbed distributions. This matters because when the environment shifts or the data contain estimation error, a model user wants a guarantee that small distribution errors cannot cause large changes in the recommended policy. The results offer explicit error bounds without relying on filtration distance or nested distance, making them easier to compute in practice.

What carries the argument

The central machinery is the dynamic nested reformulation (3.2)-(3.4), combined with a Lipschitz-continuity result for the stagewise value functions (Theorem 3.2) obtained under Slater's condition and Lipschitz assumptions on costs, transitions, and constraints. The Kantorovich metric, the supremum expectation gap over 1-Lipschitz test functions, converts these Lipschitz moduli into metric bounds through its dual representation. For exogenous perturbations, an additional conditional-Lipschitz condition on the kernels $Q_t(\xi_t\mid\xi_{[t-1]})$ lets the effect of a stagewise perturbation propagate recursively through the value functions.

What would settle it

Build a two-stage instance satisfying all Lipschitz and Slater assumptions, set $\zeta_1=\zeta_0+\varepsilon$ with a coupling that preserves each marginal, and perturb only the joint distribution. If the resulting change in optimal value exceeds the sum of stagewise Kantorovich terms in (4.5), the paper's independence claim fails; if it stays within the bound, the independence assumption is not the limiting premise.

Watch

Extended reading notes

Core claim

For the integrated model, with endogenous uncertainties $\zeta_0,\dots,\zeta_T$ independent across stages, Theorem 4.1 establishes $|\vartheta(\zeta)-\vartheta(\tilde\zeta)|\le \sum_{t=0}^{T-1}\hat L_{t+1}d_K(P_t,\tilde P_t)+L_C d_K(P_T,\tilde P_T)$, together with a corresponding Hausdorff bound for the optimal solution sets. For exogenous uncertainty, Theorems 4.5 and 4.6 give analogous bounds through conditional Kantorovich metrics, while Theorems 4.2 and 4.4 provide whole-process Fortet-Mourier bounds under Lipschitz conditions on conditional distributions. The stability theory is built on the nested dynamic reformulation and does not require the relatively complete recourse condition used in earlier filtration-distance results.

Load-bearing premise

The claim that $\zeta_0,\dots,\zeta_T$ can be taken mutually independent without loss of generality is unproved; if the endogenous uncertainties are correlated across stages, the additive stagewise decomposition behind the main endogenous-stability bound collapses.

Editorial extensions

If this is right

  • Small estimation error in a single stage's endogenous distribution yields value error bounded by that stage's Kantorovich distance times a constant, without modeling the whole process's filtration distance.
  • The bounds decompose perturbation effects stage by stage, so a decision maker can identify which stage's distribution error contributes most to the total value error.
  • For exogenous uncertainty, the conditional Lipschitz condition implies that perturbations at early stages propagate to later stages at a rate controlled by $L_{Q}$ products.
  • When a growth condition holds, similar quantitative Hausdorff bounds apply to the optimal solution sets, not only to optimal values.
  • In the one-stage case the solution-set bound yields a linear dependence on the Kantorovich distance, strengthening earlier square-root-type estimates.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper treats mutual independence of $\zeta_0,\dots,\zeta_T$ as 'without loss of generality' but supplies no reduction argument; if the endogenous shocks are correlated across stages, the additive stagewise sum in Theorem 4.1 is not guaranteed, and a nested or joint-distance formulation would likely be needed.
  • The same dual-representation argument could be adapted to distributionally robust versions of the integrated model: the Lipschitz moduli computed here directly supply worst-case gaps for ambiguity sets measured in Kantorovich or Fortet-Mourier metrics.
  • For infinite-horizon problems, the recursive constants $\hat L_t$ would need to be shown contractive; the paper stops at a finite horizon, so whether the stability bounds survive as $T\to\infty$ remains open.
  • A testable extension would be numerical validation of bound tightness on inventory or energy-dispatch instances with misspecified demand and loss-rate distributions.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a finite-horizon integrated MSP-MDP model in which endogenous noises ζ_t drive state transitions and an exogenous process ξ_t generates history-dependent costs and constraints. It derives a nested dynamic reformulation (Theorem 3.1), establishes continuity, convexity and Lipschitz properties of the stagewise value functions (Proposition 3.2, Theorem 3.2), and then develops quantitative stability bounds for the optimal value and optimal solution sets under perturbations of the endogenous distributions in Kantorovich metric (Theorem 4.1) and of the exogenous process in Fortet-Mourier and conditional Kantorovich metrics (Theorems 4.2–4.6). The paper also includes several worked examples intended to compare the new bounds with the filtration-distance and nested-distance bounds of Heitsch–Römisch and Pflug–Pichler.

Significance. If the stability theorems were correct, they would supply the first explicit distributional-perturbation bounds for this integrated MSP-MDP class, with constants built from Lipschitz moduli, feasible-set diameters and the Slater constant, and they would complement the existing filtration/nested-distance results. The paper contains detailed proofs, explicit constants, and worked examples including a nonlinear example for which the new bound is computable. However, two load-bearing technical gaps—the unjustified 'without loss of generality' independence claim for the endogenous noise and the incorrect inequality direction in the Fortet-Mourier estimates—mean that the central stability claims are not yet established as stated.

major comments (3)
  1. [§2, after Eq. (2.1d); Theorem 4.1, proof around (4.17)] The assertion that ζ0,...,ζT can be assumed mutually independent 'without loss of generality' is false for model (2.1). Since ζ_t enters the state transition S_t^M and the cost C_t, cross-stage dependence of (ζ0,...,ζT) affects the value even when all stagewise marginals are fixed. Concretely, take T=2, s0=0, s1=ζ0, s2=s1·ζ1, C0=C1=0, C2(s2)=−s2, with ζ0 and ζ1 Bernoulli(1/2). Under independence the optimal value is −1/4; under perfect correlation ζ1=ζ0 the value is −1/2, while all stagewise marginal distributions are identical. Applying Theorem 4.1 to this pair would give d_K(P_t,P̃_t)=0 for all t but |ϑ(ζ)−ϑ(ζ̃)|=1/4, contradicting (4.5). The telescoping proof at (4.17) relies precisely on inter-stage independence. The manuscript must either add an explicit assumption that ζ0,...,ζT are independent and treat this as a modeling restriction, or generalize Theorem 4.1 to a metric on the joint distribution of ζ; the current WLOG claim and the theorem as stated are not valid.
  2. [§4.2, Proposition 4.1 Eq. (4.30); Theorem 4.2 Eqs. (4.45), (4.52); Theorem 4.4 Eq. (4.61); Theorem 4.6 Eq. (4.85)] The passage from E_{ξ,ξ̃}[max{1,‖ξ‖^r,‖ξ̃‖^r}‖ξ−ξ̃‖] to the Fortet-Mourier metric d_FM,r+1 has the wrong inequality direction. With the definition (4.1)–(4.2), d_FM,p is a supremum over test functions and, by Kantorovich-Rubinstein duality, the infimum over couplings of the weighted L1 cost max{1,‖ξ‖^{p−1},‖ξ̃‖^{p−1}}‖ξ−ξ̃‖. Therefore, for any fixed coupling—in particular the product coupling implicitly used by writing E_{ξ,ξ̃}—the expectation of the cost is ≥ d_FM,p, not ≤. For instance, in (4.52) the chain 'E[...] ≤ Lϑ d_FM,3T+1' is only valid if (ξ,ξ̃) is chosen as an optimal coupling for the Fortet-Mourier cost; otherwise the inequality goes the wrong way. The same incorrect direction appears in (4.30), (4.61), and in (4.85), where E‖ξ_T−ξ̃_T‖ ≤ d_K is asserted despite d_K being the infimum over couplings. The proofs can likely be repaired by fixing, for each pair of measures, a coupling attaining the infimum in the relevant Kantorovich/Fortet-Mourier metric and performing all estimates under that coupling, or by using the dual representation directly on the value functions, but as written the Fortet-Mourier bounds are not established.
  3. [Theorem 4.3, proof leading to (4.53)] The theorem statement concludes E_{ξ,ξ̃}[H(X*(ξ),X*(ξ̃))] ≤ ϵ, but the proof ends with '<3ϵ' after accumulating the distance through the orthogonal projection and the ϵ-neighborhood argument. The constants in the definition of δ(ξ,ξ̃) should be rescaled (e.g., by replacing ϵ with ϵ/3 throughout) to match the stated conclusion, or the statement should be changed to a 3ϵ bound. This is a quantitative mismatch in one of the main stability theorems and needs to be corrected in revision.
minor comments (4)
  1. [§1 and the paragraph after Theorem 4.2] The claim that the obtained results 'subsume the main conclusions of [18], [29] and [36]' is stronger than what the paper itself demonstrates; Example 4.2 shows regimes where the new bound is looser than the nested-distance bound, and the authors later write that 'theoretical evidence is yet to be established'. The wording should be softened to 'complement'.
  2. [Example 4.2, around Eqs. (4.95)–(4.97)] The displayed identity d_K(Q1,Q̃1)=∫‖ξ1‖dQ1−∫‖ξ̃1‖dQ̃1 uses only the particular 1-Lipschitz test function ‖·‖; this gives a lower bound on the Kantorovich metric, not its value. The tightness comparison in the example should be reworked by computing or bounding the actual supremum over all 1-Lipschitz functions.
  3. [Theorem 4.1 statement, Eq. (4.5)] The constants L_{X,t} appearing in the definition of L̂_t are not defined in the theorem statement; they should be defined before the statement. In the proof they are introduced as L_{X,t}=A L_g L_S/ρ, so this is only a presentation issue, but it should be fixed for readability.
  4. [References] Reference [55], 'David Wozabal. Stability of Markovian stochastic programming', is incomplete: no publication venue or year is given. Please complete the citation or replace it with a published version.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: bounds are derived from explicit Lipschitz/Slater assumptions and Kantorovich duality; the only self-citation is not load-bearing.

full rationale

The paper's stability results are not circular. Theorem 4.1 bounds |ϑ(ζ)−ϑ(ζ̃)| by a weighted sum of Kantorovich metrics by (i) deriving Lipschitz continuity of the stagewise value functions from Assumptions 3.6/3.7/4.1 (Theorem 3.2 and (4.15)), and (ii) applying the Kantorovich duality (4.16) to the resulting Lipschitz integrand. No parameter is fitted and no bound is defined as the quantity it predicts; the constants L̂_t, H_t, LX,t are explicit functions of problem data (L_C, L_S, L_g, A, ρ). Theorems 4.5-4.6 similarly use the conditional-distribution Lipschitz assumption (4.71) as an input, not as the conclusion. The only self-citation that appears in a proof is [32, Lemma 3] for the interchangeability principle in Theorem 3.1; the paper verifies the lemma's conditions in place and the lemma is a standard measurable-selection result, so it is not load-bearing. One flagged limitation is the assertion after (2.1d): 'As shown in [19] and other related references, we assume without loss of generality that ζ0,ζ1,ζ2,···,ζT are mutually independent, thereby facilitating subsequent analysis.' This independence is used in the stagewise summation (4.17), and the marginal Kantorovich metrics in (4.5) do not determine joint dependence, so the WLOG claim is a correctness/missing-assumption gap rather than circularity; per the hard rules it does not raise the circularity score.

Assumptions & free parameters 0 free parameters · 9 assumptions · 0 invented entities

No free parameters are fitted; all constants in the bounds are explicit functions of problem data such as Lipschitz moduli, the feasible-set diameter A, and the Slater constant ρ. The main assumptions are standard in MSP and MDP stability analysis, with the mutual independence of ζt and the conditional-distribution Lipschitz condition being the most restrictive and least standard.

assumptions (9)
  • domain assumption Integrability of cost functions (Assumption 3.1)
    Ensures expected costs are finite and the value functions are well-defined.
  • domain assumption Continuity of cost, constraint, and transition mappings (Assumption 3.2)
    Needed for Berge's maximum theorem and for the nested reformulation in Theorem 3.1.
  • domain assumption Uniform boundedness of stage feasible sets (Assumption 3.3)
    Provides compactness, which is used for existence of optimal solutions and for Hausdorff-distance bounds.
  • domain assumption Uniform Slater condition with constant ρ (Assumption 3.6)
    Load-bearing for the Lipschitz continuity of the feasible set-valued mappings; the error bounds contain 1/ρ and so degrade as ρ approaches zero.
  • domain assumption Lipschitz continuity of cost, transition, and constraint functions (Assumptions 3.7, 4.1, 4.2)
    Provides the moduli L_C, L_S, L_g that appear explicitly in every quantitative bound; these are assumed rather than derived.
  • domain assumption Convexity and monotonicity of relevant functions (Assumptions 3.4 and 3.5)
    Needed for convexity of the value functions and for the specific form of the feasible-set Lipschitz constants.
  • domain assumption Mutual independence of endogenous uncertainties ζ0,...,ζT
    Stated as WLOG in Section 2; used in Theorem 4.1 to decompose perturbations into a sum of stagewise Kantorovich metrics. Restrictive for correlated endogenous noise and unproven.
  • ad hoc to paper Lipschitz continuity of conditional distributions (inequalities (4.71a) and (4.71b))
    Assumed in Theorem 4.5 for stagewise exogenous perturbation bounds; restricts the class of exogenous stochastic processes to those whose conditional laws vary Lipschitz-continuously with the history.
  • ad hoc to paper Growth conditions (4.6), (4.59), and (4.82)
    Needed to convert value-function stability into optimal-solution-set stability; not satisfied in general without strong convexity or similar conditions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Stability Analysis of an Integrated Multistage Stochastic Programming and Markov Decision Process Problem." pith.science (2026). https://pith.science/paper/ZS7EPTOT

@misc{pith2026250922194,
  author       = {Pith},
  title        = {Pith review of: Stability Analysis of an Integrated Multistage Stochastic Programming and Markov Decision Process Problem},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZS7EPTOT}},
  note         = {Machine review of arXiv:2509.22194}
}
read the original abstract

In this paper, we consider an integrated MSP-MDP framework which captures features of Markov decision process (MDP) and multistage stochastic programming (MSP). The integrated framework allows one to study a dynamic decision-making process that involves both transition of system states and dynamic change of the stochastic environment affected respectively by potential endogenous uncertainties and exogenous uncertainties. The integrated model differs from classical MDP models by taking into account the effect of history-dependent exogenous uncertainty and distinguishes itself from standard MSP models by explicitly considering transition of states between stages. We begin by deriving dynamic nested reformulation of the problem and the Lipschitz continuity and convexity of the stage-wise optimal value functions. We then move on to investigate stability of the problem in terms of the optimal value and the set of optimal solutions under the perturbations of the probability distributions of the endogenous uncertainty and the exogenous uncertainty. Specifically, we quantify the effects of the perturbation of the two uncertainties on the optimal values and optimal solutions by deriving the error bounds in terms of Kantorovich metric and Fortet-Mourier metric of the probability distributions of the respective uncertainties. These results differ from the existing stability results established in terms of the filtration distance \cite{heitsch2009scenario} or the nested distance \cite{pflug2012distance}. We use some examples to explain the differences via tightness of the error bounds and applicability of the stability results. The results complement the existing stability results and provide new theoretical grounding for emerging integrated MSP-MDP models.

Figures

Figures reproduced from arXiv: 2509.22194 by the authors.

Figure 1
Figure 1. Chronology of states, random variables, and decision variables [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 47 canonical work pages

  1. [1]

    Springer Science & Business Media, 2006

    Charalambos D Aliprantis and Kim C Border.Infinite dimensional analysis: a hitchhiker’s guide. Springer Science & Business Media, 2006

    work page 2006

  2. [2]

    Distributionally robust markov decision processes and their connection to risk measures.Mathematics of Operations Research, 47(3):1757–1780, 2022

    Nicole B¨ auerle and Alexander Glauner. Distributionally robust markov decision processes and their connection to risk measures.Mathematics of Operations Research, 47(3):1757–1780, 2022

    work page 2022

  3. [3]

    Athena scientific, 2012

    Dimitri Bertsekas.Dynamic Programming and Optimal Control: Volume I. Athena scientific, 2012

    work page 2012

  4. [4]

    Dynamic optimization with side information.European Journal of Operational Research, 304(2):634–651, 2023

    Dimitris Bertsimas, Christopher McCord, and Bradley Sturt. Dynamic optimization with side information.European Journal of Operational Research, 304(2):634–651, 2023

    work page 2023

  5. [5]

    Springer Science & Business Media, 2011

    John R Birge and Francois Louveaux.Introduction to stochastic programming. Springer Science & Business Media, 2011

    2011

  6. [6]

    Weak continuity of risk functionals with applications to stochastic programming.SIAM Journal on Optimization, 27(1):91–109, 2017

    Matthias Claus, Volker Kratschmer, and R¨ udiger Schultz. Weak continuity of risk functionals with applications to stochastic programming.SIAM Journal on Optimization, 27(1):91–109, 2017

    work page 2017

  7. [7]

    Springer Science & Business Media, 2005

    Carlos Daganzo.Logistics systems analysis. Springer Science & Business Media, 2005

    work page 2005

  8. [8]

    Dellacherie and P.A

    C. Dellacherie and P.A. Meyer.Probabilities and Potential. North-Holland mathematics studies. Hermann, London, 1978

    work page 1978

Show all 56 references
  1. [9]

    Subregular recourse in nonlinear multistage stochastic optimization.Mathematical Programming, 189(1):249–270, 2021

    Darinka Dentcheva and Andrzej Ruszczy´ nski. Subregular recourse in nonlinear multistage stochastic optimization.Mathematical Programming, 189(1):249–270, 2021

    2021

  2. [10]

    Single item periodic review inventory control with sales dependent stochastic return flows.International Journal of Production Economics, 255:108699, 2023

    Esra G¨ okbayrak and Enis Kayı¸ s. Single item periodic review inventory control with sales dependent stochastic return flows.International Journal of Production Economics, 255:108699, 2023

    2023

  3. [11]

    Reinforcement learning: A tutorial survey and recent advances.INFORMS Journal on Computing, 21(2):178–192, 2009

    Abhijit Gosavi. Reinforcement learning: A tutorial survey and recent advances.INFORMS Journal on Computing, 21(2):178–192, 2009

    2009

  4. [12]

    Springer, 2006

    A Hitchhiker’s Guide.Infinite dimensional analysis. Springer, 2006

    2006

  5. [13]

    Statistical robustness in utility preference robust optimization models.Mathematical Programming, 190(1):679–720, 2021

    Shaoyan Guo and Huifu Xu. Statistical robustness in utility preference robust optimization models.Mathematical Programming, 190(1):679–720, 2021

    2021

  6. [14]

    Robust microgrid capacity investment with endogenous and exogenous uncertainties.IEEE Transactions on Smart Grid, 15(3):2480– 2492, 2023

    Hossein Haghighat, Wei Wang, and Bo Zeng. Robust microgrid capacity investment with endogenous and exogenous uncertainties.IEEE Transactions on Smart Grid, 15(3):2480– 2492, 2023

    2023

  7. [15]

    Contextual markov decision processes

    Assaf Hallak, Dotan Di Castro, and Shie Mannor. Contextual markov decision processes. arXiv preprint arXiv:1502.02259, 2015

    2015 arXiv

  8. [16]

    Online reinforcement learning in non-stationary context-driven environments

    Pouya Hamadanian, Arash Nasr-Esfahany, Malte Schwarzkopf, Siddartha Sen, and Moham- mad Alizadeh. Online reinforcement learning in non-stationary context-driven environments. arXiv preprint arXiv:2302.02182, 2023

    2023 arXiv

  9. [17]

    Scenario tree modeling for multistage stochastic programs.Mathematical Programming, 118:371–406, 2009

    Holger Heitsch and Werner R¨ omisch. Scenario tree modeling for multistage stochastic programs.Mathematical Programming, 118:371–406, 2009

    2009

  10. [18]

    Stability of multistage stochastic programs.SIAM Journal on Optimization, 17(2):511–525, 2006

    Holger Heitsch, Werner R¨ omisch, and Cyrille Strugarek. Stability of multistage stochastic programs.SIAM Journal on Optimization, 17(2):511–525, 2006. 56

    2006

  11. [19]

    Springer Science & Business Media, 2012

    On´ esimo Hern´ andez-Lerma and Jean B Lasserre.Discrete-Time Markov Control Processes: Basic Optimality criteria, volume 30. Springer Science & Business Media, 2012

    2012

  12. [20]

    Lipschitz continuity of value functions in markovian decision processes

    Karl Hinderer. Lipschitz continuity of value functions in markovian decision processes. Mathematical Methods of Operations Research, 62:3–22, 2005

    2005

  13. [21]

    Reinforcement learning and stochastic optimisation.Finance and Stochastics, 26(1):103–129, 2022

    Sebastian Jaimungal. Reinforcement learning and stochastic optimisation.Finance and Stochastics, 26(1):103–129, 2022

    2022

  14. [22]

    Amir Jamali, Amirhossein Ranjbar, Jafar Heydari, and Sina Nayeri. A multi-objective stochastic programming model to configure a sustainable humanitarian logistics considering deprivation cost and patient severity.Annals of operations research, pages 1–36, 2022

    2022

  15. [23]

    Quantitative stability of multistage stochastic programs via calm modifications.Operations Research Letters, 46(5):543–547, 2018

    Jie Jiang and Zhiping Chen. Quantitative stability of multistage stochastic programs via calm modifications.Operations Research Letters, 46(5):543–547, 2018

    2018

  16. [24]

    Springer, 1994

    Peter Kall, Stein W Wallace, and Peter Kall.Stochastic programming, volume 5. Springer, 1994

    1994

  17. [25]

    First-order sensitivity of the optimal value in a markov decision model with respect to deviations in the transition probability function

    Patrick Kern, Axel Simroth, and Henryk Z¨ ahle. First-order sensitivity of the optimal value in a markov decision model with respect to deviations in the transition probability function. Mathematical Methods of Operations Research, 92(1):165–197, 2020

    2020

  18. [26]

    Convexity and monotonicity in nonlinear optimal control under uncertainty.arXiv preprint arXiv:1904.00987, 2019

    Kevin J Kircher and K Max Zhang. Convexity and monotonicity in nonlinear optimal control under uncertainty.arXiv preprint arXiv:1904.00987, 2019

    1904 arXiv

  19. [27]

    A stability result for linear markovian stochastic optimization problems.Mathematical Programming, 191(2):871–906, 2022

    Adriana Kiszka and David Wozabal. A stability result for linear markovian stochastic optimization problems.Mathematical Programming, 191(2):871–906, 2022

    2022

  20. [28]

    Detection of discomfort in automated driving via stochastic approximation.Human Factors in Transportation, 60:87, 2022

    Florian Kretzschmar, Alois Pichler, and Matthias Beggiato. Detection of discomfort in automated driving via stochastic approximation.Human Factors in Transportation, 60:87, 2022

    2022

  21. [29]

    On stability of multistage stochastic programs.SIAM Journal on Optimization, 19(2):952–968, 2008

    Christian K¨ uchler. On stability of multistage stochastic programs.SIAM Journal on Optimization, 19(2):952–968, 2008

    2008

  22. [30]

    Metrics and continuity in reinforcement learning

    Charline Le Lan, Marc G Bellemare, and Pablo Samuel Castro. Metrics and continuity in reinforcement learning. InProceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 8261–8269, 2021

    2021

  23. [31]

    Non-stationary markov decision processes, a worst-case approach using model-based reinforcement learning.Advances in neural information processing systems, 32, 2019

    Erwan Lecarpentier and Emmanuel Rachelson. Non-stationary markov decision processes, a worst-case approach using model-based reinforcement learning.Advances in neural information processing systems, 32, 2019

    2019

  24. [32]

    Preference ambiguity and robustness in multistage decision making.Mathematical Programming, pages 1–93, 2025

    Jia Liu, Zhiping Chen, and Huifu Xu. Preference ambiguity and robustness in multistage decision making.Mathematical Programming, pages 1–93, 2025

    2025

  25. [33]

    Xiangguo Liu, Neda Masoud, Qi Zhu, and Anahita Khojandi. A markov decision process framework to incorporate network-level data in motion planning for connected and au- tomated vehicles.Transportation Research Part C: Emerging Technologies, 136:103550, 2022

    2022

  26. [34]

    A bayesian composite risk approach for stochastic optimal control and markov decision processes.arXiv preprint arXiv:2412.16488, 2024

    Wentao Ma, Zhiping Chen, and Huifu Xu. A bayesian composite risk approach for stochastic optimal control and markov decision processes.arXiv preprint arXiv:2412.16488, 2024

    2024 arXiv

  27. [35]

    Robust stochastic approximation approach to stochastic programming.SIAM Journal on Optimiza- tion, 19(4):1574–1609, 2009

    Arkadi Nemirovski, Anatoli Juditsky, Guanghui Lan, and Alexander Shapiro. Robust stochastic approximation approach to stochastic programming.SIAM Journal on Optimiza- tion, 19(4):1574–1609, 2009. 57

    2009

  28. [36]

    A distance for multistage stochastic optimization models

    Georg Ch Pflug and Alois Pichler. A distance for multistage stochastic optimization models. SIAM Journal on Optimization, 22(1):1–23, 2012

    2012

  29. [37]

    Springer, 2014

    Georg Ch Pflug and Alois Pichler.Multistage stochastic optimization, volume 1104. Springer, 2014

    2014

  30. [38]

    Quantitative stability analysis for minimax distributionally robust risk optimization.Mathematical Programming, 191(1):47–77, 2022

    Alois Pichler and Huifu Xu. Quantitative stability analysis for minimax distributionally robust risk optimization.Mathematical Programming, 191(1):47–77, 2022

    2022

  31. [39]

    Tutorial on stochastic optimization in energy—part ii: An energy storage illustration.IEEE Transactions on Power Systems, 31(2):1468–1475, 2015

    Warren B Powell and Stephan Meisel. Tutorial on stochastic optimization in energy—part ii: An energy storage illustration.IEEE Transactions on Power Systems, 31(2):1468–1475, 2015

    2015

  32. [40]

    John Wiley & Sons, 2014

    Martin L Puterman.Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, 2014

    2014

  33. [41]

    Quantitative stability in stochastic programming: The method of probability metrics.Mathematics of Operations Research, 27(4):792–818, 2002

    Svetlozar T Rachev and Werner R¨ omisch. Quantitative stability in stochastic programming: The method of probability metrics.Mathematics of Operations Research, 27(4):792–818, 2002

    2002

  34. [42]

    Stability of stochastic programming problems.Handbooks in operations research and management science, 10:483–554, 2003

    Werner R¨ omisch. Stability of stochastic programming problems.Handbooks in operations research and management science, 10:483–554, 2003

    2003

  35. [43]

    Stability of solutions for stochastic programs with complete recourse.Mathematics of Operations Research, 18(3):590–609, 1993

    Werner R¨ omisch and R¨ udiger Schultz. Stability of solutions for stochastic programs with complete recourse.Mathematics of Operations Research, 18(3):590–609, 1993

    1993

  36. [44]

    Distributionally robust optimal control and mdp modeling.Operations Research Letters, 49(5):809–814, 2021

    Alexander Shapiro. Distributionally robust optimal control and mdp modeling.Operations Research Letters, 49(5):809–814, 2021

    2021

  37. [45]

    SIAM, 2021

    Alexander Shapiro, Darinka Dentcheva, and Andrzej Ruszczynski.Lectures on stochastic programming: modeling and theory. SIAM, 2021

    2021

  38. [46]

    Hindsight learning for mdps with exogenous inputs

    Sean R Sinclair, Felipe Vieira Frujeri, Ching-An Cheng, Luke Marshall, Hugo De Oliveira Barbalho, Jingling Li, Jennifer Neville, Ishai Menache, and Adith Swaminathan. Hindsight learning for mdps with exogenous inputs. InInternational Conference on Machine Learning, pages 31877...

    2023

  39. [47]

    MIT press, 2018

    Richard S Sutton and Andrew G Barto.Reinforcement learning: An introduction. MIT press, 2018

    2018

  40. [48]

    Production and energy mode control of a production-inventory system.European Journal of Operational Research, 308(3):1176–1187, 2023

    Barı¸ s Tan, Oktay Karaba˘ g, and Siamak Khayyati. Production and energy mode control of a production-inventory system.European Journal of Operational Research, 308(3):1176–1187, 2023

    2023

  41. [49]

    Generalized bayesian nash equilibrium with continuous type and action spaces.SIAM Journal on Optimization, 35(2):789–817, 2025

    Yuan Tao and Huifu Xu. Generalized bayesian nash equilibrium with continuous type and action spaces.SIAM Journal on Optimization, 35(2):789–817, 2025

    2025

  42. [50]

    Inventory management of remanufacturable products.Management science, 46(11):1412–1426, 2000

    L Beril Toktay, Lawrence M Wein, and Stefanos A Zenios. Inventory management of remanufacturable products.Management science, 46(11):1412–1426, 2000

    2000

  43. [51]

    Multistage energy management of coordinated smart buildings: A multiagent markov decision process approach.IEEE Transactions on Smart Grid, 13(4):2788–2797, 2022

    Georgios Tsaousoglou, Nikolaos Efthymiopoulos, Prodromos Makris, and Emmanouel Varvarigos. Multistage energy management of coordinated smart buildings: A multiagent markov decision process approach.IEEE Transactions on Smart Grid, 13(4):2788–2797, 2022

    2022

  44. [52]

    Aleatoric and epistemic discrimination: Fundamental limits of fairness interventions.Advances in Neural Information Processing Systems, 36:27040–27062, 2023

    Hao Wang, Luxi He, Rui Gao, and Flavio Calmon. Aleatoric and epistemic discrimination: Fundamental limits of fairness interventions.Advances in Neural Information Processing Systems, 36:27040–27062, 2023. 58

    2023

  45. [53]

    Lizhi Wang and Nan Kong. Security constrained economic dispatch: A markov decision pro- cess approach with embedded stochastic programming.International Journal of Operations Research and Information Systems (IJORIS), 1(2):1–16, 2010

    2010

  46. [54]

    Robust spectral risk optimization when information on risk spectrum is incomplete.SIAM Journal on Optimization, 30(4):3198–3229, 2020

    Wei Wang and Huifu Xu. Robust spectral risk optimization when information on risk spectrum is incomplete.SIAM Journal on Optimization, 30(4):3198–3229, 2020

    2020

  47. [55]

    Stability of markovian stochastic programming

    David Wozabal. Stability of markovian stochastic programming

  48. [56]

    A two-phase optimization model com- bining markov decision process and stochastic programming for advance surgery scheduling

    Jian Zhang, Mahjoub Dridi, and Abdellah El Moudni. A two-phase optimization model com- bining markov decision process and stochastic programming for advance surgery scheduling. Computers & Industrial Engineering, 160:107548, 2021. A Supplementary results and proofs Proposition...

    2021

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.