Pith. sign in

REVIEW 3 major objections 4 minor 29 references

On the Modelling of Impulse Control with Random Effects for Continuous Markov Processes

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single path-space model puts randomized impulse control of continuous Markov processes on one probability space.

desk verdict Solid modelling paper with a genuine but fixable gap: Theorem 3.3 only realizes cycle-local policies, not the full class of admissible stopping times. read the letter →

arxiv 1908.08357 v5 pith:HYNYX2Z2 submitted 2019-08-22 math.OC math.PR

K.L. Helmes , R.H. Stockbridge , C. Zhu This is my paper · ORCID
classification math.OCmath.PR MSC 93E2060H30
keywords impulsecontrolrandomizedrandomeffectsMarkovpolicyfamilypathspaceconstructionstrongprocess(sS)ordering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper establishes that impulse control with random effects for continuous Markov processes can be modelled on the single path space of càdlàg functions, rather than on the customary countable product of path spaces. For every admissible nominal impulse policy and initial distribution, it constructs a probability measure under which the coordinate process evolves as the underlying strong Markov process between interventions, and at each intervention the next state is drawn from a prescribed distribution conditioned on the pre-intervention state and the nominal impulse. The construction also identifies a class of Markov nominal policies for which the controlled process is Markov and forms a Markov family, and a class of independent-cycles policies for which cycles after the first are independent and identically distributed, making classical renewal arguments available for long-term average cost problems. This matters because randomized impulses arise naturally in applications such as inventory management, and a single natural filtration gives the intervention times a simpler, more direct interpretation as stopping times.

What carries the argument

The load-bearing object is the pasting map from the countable product of path spaces to the single path space: after building a product measure through transition kernels that integrate the fundamental process against the random-effect distribution for the next cycle, the map concatenates the coordinate paths at the random intervention times, and its push-forward is the desired single-space measure. Two supporting mechanisms make this work: the path-locality characterization of left-continuous stopping times ensures that evaluating the next intervention time on the new coordinate path does not change the values of earlier intervention times, and the terminal-time condition restricts Markov policies to decisions based on the current cycle, which is what lets the Markov property be proved. The continuity of the fundamental process guarantees that the pre-intervention state appears in the natural filtration and that the pasted paths are càdlàg.

What would settle it

Take E=R with a continuous strong Markov process, set the first intervention at τ1≡1, and define the second intervention time by τ2(ω)=2+1_{X(0)∈A}, which is a left-continuous stopping time under the paper's Definition 2.3. Under the constructed measure, the second intervention time is computed from the first-cycle path after shifting, so it depends on the state after the first impulse rather than on X(0). A path with X(0)∈A but different post-impulse state would receive different second intervention times under the original policy and under the constructed process, showing that the theorem as stated does not realize this admissible policy.

Watch

Extended reading notes

Core claim

The central discovery is Theorem 3.3: for a strong Markov process with continuous paths and a given family of post-impulse distributions, any admissible nominal impulse policy determines a family of probability measures on the single space of càdlàg paths. Under the resulting measure, the coordinate process runs as the fundamental Markov process on the first interval, and on each later interval the process again follows the fundamental dynamics, with the post-impulse state conditionally distributed according to the random-effect distribution given the information before the intervention. The measure is built on the countable product of path spaces by applying the Ionescu Tulcea extension theorem to transition kernels that select each new cycle's starting distribution, then pushing forward the product measure through a càdlàg pasting map that splices the coordinate paths at the intervention times. The paper then proves that when the stopping times are terminal times depending only on the current cycle and the impulses depend only on the pre-intervention state, the controlled process is Markov and, together with time-shifted policies, forms a Markov family. For policies whose intervention times are determined within each cycle and whose effect distributions do not depend on the path, the cycles are independent; for the (s,S) ordering policy they are identically distributed after the first cycle.

Load-bearing premise

The construction assumes that every intervention time can be evaluated from the current cycle's path alone, because the next stopping time is applied to the newly pasted coordinate path; policies whose stopping decisions use absolute clock time or the history of earlier cycles are not realized by the model.

Editorial extensions

If this is right

  • Randomized impulses can be handled rigorously on one path space: the controller's nominal impulse and the observed pre-impulse state only need to specify a distribution for the new state, not a deterministic value.
  • For continuous strong Markov processes, any admissible policy's controlled process has a concrete single-space representation, so optimal control questions can be posed with the natural filtration of the controlled process.
  • Markov nominal policies make the controlled process Markov and the time-shifted family a Markov family, opening dynamic programming and Markov-process tools to randomized impulse control.
  • Independent-cycles policies, including (s,S) inventory ordering, produce independent cycles; identical stopping rules and effect distributions make cycles after the first identically distributed, allowing renewal theorems for average-cost criteria.
  • The construction is restricted to continuous paths; extending it to jump processes would require a different treatment of the left-limit state at intervention times.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The cycle-local interpretation implicit in the construction suggests that admissible policies using absolute clock time or the full history of earlier cycles are not realized by the theorem as stated; one testable extension is to enlarge the state with a clock or finite memory so that such policies become cycle-local.
  • Because the random-effect mechanism is just another transition kernel, the same construction should extend to randomized intervention times or relaxed controls by making the next-cycle kernel itself random.
  • The Markov family is parameterized by time-shifted policies, which points toward viewing randomized impulse control as a family of Markov decision problems on the path space, one for each starting time and state.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper develops a single-path-space model for impulse control of continuous strong Markov processes with random effects. The main construction (Theorem 3.3) builds, for a nominal impulse policy and an initial distribution, a probability measure on the path space under which the coordinate process evolves as the fundamental Markov process between interventions, with the post-intervention state drawn from a policy-dependent kernel. The paper then isolates a class of Markov nominal impulse policies for which the controlled process is Markov and forms a Markov family, and a class of independent-cycles policies for which renewal arguments can be applied. The (s,S) inventory ordering policy is given as an example.

Significance. If the central construction is repaired, the paper addresses a genuine modelling problem: impulse control with randomized post-intervention states, formulated on a single path space rather than on a countable product of path spaces. The proposed subclasses (Markov policies and independent-cycles policies) are natural and potentially useful for long-term average control problems, and the (s,S) example illustrates a practically relevant setting. The paper is also careful in its use of strong Markov families, Galmarino/Courrège–Priouret characterizations, and the Ionescu Tulcea extension theorem. However, the main existence theorem is currently proved only for a narrower class of policies than the one stated in Definition 2.3, and a few supporting arguments need tightening before the results can be accepted as stated.

major comments (3)
  1. [Theorem 3.3 proof, transition kernel measurability] The construction defines the second intervention by ~τ2(e1) = τ2(ω1), i.e., the original policy functional τ2 is evaluated on the fresh coordinate path ω1, not on the pasted path T1(e1); the induction step repeats this by evaluating τ_{k+1} on ω_k. Equality ~τ_k = τ_k(~X) is asserted in the final paragraph of the proof but is not proved for k ≥ 2, and it is false in general under Definition 2.3. For example, if τ1 ≡ 5 and τ2 ≡ 10 (both are admissible {F_{t−}}-stopping times), the construction gives ~τ2 = 10 in the time scale of the second coordinate, so the pasted process intervenes at absolute time 15 rather than 10. The observation after (3.5) and Corollary 3.2 only compare ω1 with ω0 on [0, τ1(ω0)); they do not compare T1(e1) with ω1 on the second cycle. The theorem can be repaired either by restricting Definition 2.3 to cycle-local policies, as is effectively done in Definitions 4.1 and 5.1, or, for the general class, by defining ~τ_{k+1}(e_k) = τ_{k+1}(T_k(e_k)) and using Corollary 3.2 to prove consistency with the final pasted path. As written, the central claim of Theorem 3.3(a) is not established for the stated class of admissible policies.
  2. [Theorem 3.3 proof, transition kernel measurability] In the verification that P1 is a transition kernel, the proof asserts that ω0 ↦ ∫_E P_{v1}(F1) Q1(ω0, dv1) is F0-measurable because v1 ↦ P_{v1}(F1) is universally measurable and can be uniformly approximated by simple functions. Universal measurability does not imply Borel measurability, and composing a non-Borel universally measurable function with the F0-measurable maps (~Y1, ~Z1) need not produce an F0-measurable function (a Dirac-type kernel makes this explicit). The Ionescu Tulcea step therefore requires either a Borel-measurable transition family, for example under a Feller-type assumption, or a completed σ-algebra with a correspondingly amended statement of the theorem. This is a technical gap, but it affects the proof of existence in Theorem 3.3.
  3. [Theorem 4.9] The theorem states that {P^{θ_s(τ,Z)}_x : s ≥ 0, x ∈ E} is a Markov family. Under the standard definition cited from Karatzas and Shreve (Definition 2.5.11), a member P_{s,x} must satisfy P_{s,x}(X(s) = x) = 1. The measures constructed here satisfy P^{θ_s(τ,Z)}_x(X(0) = x) = 1, since they are measures on paths indexed from time 0. Equation (4.19) is a useful policy-shift relation, but it does not by itself place the collection in the cited definition. The proof should either define the measures on time-shifted paths, e.g. by composing with the shift operator θ_s, or reformulate the claim as a time-inhomogeneous Markov family with policy shifts.
minor comments (4)
  1. [Theorem 3.1] The proof of Theorem 3.1 is deferred with the statement that the proof of Courrège and Priouret 'remains valid with the slight modification'. Since Corollary 3.2 is used in the main construction, the authors should either give a complete proof of the left-continuous version or cite a source where it is proved.
  2. [Theorem 3.3 proof, notation] The notation in the induction step overloads ω_k, which is used both for the coordinate path in Ω_k and for the pasted path T_k(e_k). In particular, equations (3.4)–(3.6) are hard to parse because ω1 is used in both senses. Introducing separate notation for the pasted path would clarify the argument and help expose the consistency issue raised above.
  3. [Induction step, text around Eq. (3.9)] In the induction step, the phrase '~Y_{k+1} ∈ E_k corresponds to ω_k(τ_{k+1}−) ∈ E' appears to contain a typo: ~Y_{k+1} should take values in E, not in E_k.
  4. [Theorem 4.6 proof] The proof of Theorem 4.6 repeatedly uses the phrase that the collections {F_{j−1}} and {G_j(e_{j−1})} are 'sufficiently large' to conclude equality of conditional expectations. This should be replaced by an explicit monotone-class or π–λ argument, since the current wording is too terse for a proof of this complexity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: derivation is self-contained; flagged cycle-locality gap is a correctness issue, not a circular reduction.

full rationale

The paper's derivation is self-contained against standard measure-theoretic inputs: Theorem 3.3 constructs P^{(tau,Z)}_nu from the given strong Markov family {P_x}, the random-effects kernel Q, and the Ionescu-Tulcea extension theorem, with no fitted parameters and no quantity predicted from a subset of data. The Galmarino/Courrege-Priouret characterization (Theorem 3.1, Corollary 3.2) is an external classical result, not a self-citation, and is used only to check consistency at the first intervention. The Markov property in Theorems 4.6 and 4.7 is proved from the terminal-time condition (4.1) and the Markov property of the fundamental process; the terminal-time condition is a policy restriction, not the conclusion being derived. The independent-cycles result in Proposition 5.2 follows from the path-independence conditions in Definition 5.1 through the product structure of the transition kernels, so it is a derived consequence rather than a renamed assumption. The only self-citation, Helmes et al. (2015), is contextual and not load-bearing. A genuine limitation is present but it is not circular: equations (3.4)-(3.6) and the induction step of Theorem 3.3 evaluate tau_{k+1} on the current coordinate path omega_k, while Definition 2.3 permits arbitrary {F_{t-}}-stopping times, and the paper does not prove that tau_k of the pasted path equals tau_tilde_k for k >= 2. This flagged gap affects the stated domain of the theorem, not the independence of the derivation from its own conclusions.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The paper introduces no new entities and no fitted parameters. The central claim rests on standard measure-theoretic extension theorems, the strong Markov and continuity properties of the fundamental process, and an implicit cycle-local structure of policies.

assumptions (6)
  • domain assumption Strong Markov property of the fundamental process {P_x}
    Invoked in Theorem 3.3 for measurability of P_v(F_1), and in Theorems 4.7 and 4.9 for the Markov family property. Remark 2.2 notes this property is required for the Markov results.
  • domain assumption Continuity of paths under each P_x (Condition 2.1)
    Ensures X(tau_k-) = X(tau_k) lies in the natural filtration and that the constructed paths are cadlag. The paper states the construction fails for jump processes.
  • standard math Ionescu Tulcea extension theorem (Neveu 1965)
    Used in Theorem 3.3 to pass from finite-dimensional transition kernels to a measure on the countable product space.
  • standard math Galmarino characterization of {Ft-}-stopping times (Courrege and Priouret 1965)
    Theorem 3.1 is stated as a modification of this cited result; the proof is only sketched as 'remains valid with slight modification'.
  • domain assumption Measurability of the random-effect kernel Q (B(E) tensor Z to B(E) measurable)
    Needed for the transition functions Q_k in equations (3.2), (3.7), and (3.9) to be measurable.
  • ad hoc to paper Cycle-local interpretation of policies
    The construction evaluates tau_{k+1} on the coordinate path omega_k (equations (3.4)-(3.6)), which realizes policies whose decisions depend on the current cycle path, not arbitrary history or absolute clock time. This restriction is not stated in Definition 2.3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On the Modelling of Impulse Control with Random Effects for Continuous Markov Processes." pith.science (2026). https://pith.science/paper/HYNYX2Z2

@misc{pith2026190808357,
  author       = {Pith},
  title        = {Pith review of: On the Modelling of Impulse Control with Random Effects for Continuous Markov Processes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HYNYX2Z2}},
  note         = {Machine review of arXiv:1908.08357}
}
abstract

The use of coordinate processes for the modelling of impulse control for general Markov processes typically involves the construction of a probability measure on a countable product of copies of the path space. In addition, admissibility of an impulse control policy requires that the random times of the interventions be stopping times with respect to different filtrations arising from the different component coordinate processes. When the underlying Markov process has continuous paths, however, a simpler model can be developed which takes the single path space as its probability space and uses the natural filtration with respect to which the intervention times must be stopping times. Moreover, this model construction allows for impulse control with random effects whereby the decision maker selects a distribution of the new state. This paper gives the construction of the probability measure on the path space for an admissible intervention policy subject to a randomized impulse mechanism. In addition, a class of polices is defined for which the paths between interventions are independent and a further subclass for which the cycles following the initial cycle are identically distributed. A benefit of this smaller subclass of policies is that one is allowed to use classical renewal arguments to analyze long-term average control problems. Further, the paper defines a class of {\em stationary}\/ impulse policies for which the family of models gives a Markov family. The decision to use an $(s,S)$ ordering policy in inventory management provides an example of an impulse policy for which the process has i.i.d.~cycles and the family of models forms a Markov family.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 29 canonical work pages

  1. [1]

    Alvarez, L. H. R. (2004) A Class of Solvable Impulse Control Problem s. Appl. Math. Optim. 49:265-295

    work page 2004

  2. [2]

    and Lions, J

    Bensoussan, A. and Lions, J. L. (1975). Nouvelle Methodes en Con trˆ ole Impulsionnel.Appl. Math Optim. 1:289-312

    work page 1975

  3. [3]

    and Lions, J

    Bensoussan, A. and Lions, J. L. (1984). Impulse Control and Quasi-Variational Inequalities ,

    work page 1984

  4. [4]

    and Zapatero, F

    Cadenillas, A. and Zapatero, F. (2000) Classical and Impulse Stoch astic Control of the Exchange Rate using Interest Rates and Reserves Math. Fin. 10:141-156

    work page 2000

  5. [5]

    Christensen, S. (2014). On the solution of general impulse contro l problems using superhar- monic functions. Stochastic Process. Appl. 124:709–729

    work page 2014

  6. [6]

    Chung, K. L. and Walsh, J. B. (2005) Markov processes, Brownian motion, and time symme- try, second ed., vol. 249 of Grundlehren der Mathematischen Wissenschaften [Fundamen tal Principles of Mathematical Sciences] . Springer, New York. Courr` ege, P. and Priouret, P, (1965). Temps d’arrˆ et d’une fon ction al´ eatoire: Relations d’´ equivalence associ´ ee...

    work page 2005

  7. [7]

    Davis, M. H. A. (1993) Markov models and optimization, Monographs on Statistics and Applied Probability 49, Chapman & Hall, London

    work page 1993

  8. [8]

    Ethier, S. N. and Kurtz, T. G. (1986). Markov processes: Characterization and Convergence . Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. John Wiley & Sons Inc., New York

    work page 1986

Show all 29 references
  1. [9]

    Friedman, A. (1975). Stochastic differential equations and applications. Vol. 1 . Probability and Mathematical Statistics, Vol. 28, Academic Press, New York-L ondon

    1975

  2. [10]

    and Seydel, R

    Frey, R. and Seydel, R. C. (2010) Optimal securitization of credit p ortfolios via impulse control. Math Finan Econ 4:128

    2010

  3. [11]

    Galmarino, A. R. (1963) A test for Markov times. Rev. Un. Mat. Argentina 21:173-178

    1963

  4. [12]

    M., Sellke, T

    Harrison, J. M., Sellke, T. M. and Taylor, A. J. (1983). Impulse Cont rol of Brownian Motion Math. Oper. Res. 8:454-466

    1983

  5. [13]

    L., Stockbridge, R

    Helmes, K. L., Stockbridge, R. H. and Zhu, C. (2015). A measure ap proach for continuous inventory models: discounted cost criterion. SIAM J. Control Optim. , 53:4, 2100–2140

    2015

  6. [14]

    and Watanabe, S

    Ikeda, N., Nagasawa, M. and Watanabe, S. (1966). A constructio n of Markov processes by piecing out Proc. Japan Acad. 42:370-375

    1966

  7. [15]

    and Zervos, M

    Jack, A. and Zervos, M. (2006) Impulse Control of One-Dimension al Itˆ o Diffusions with an Expected and a Pathwise Ergodic Criterion. Appl. Math. Optim. 54:7193. 31

    2006

  8. [16]

    and Shreve, S

    Karatzas, I. and Shreve, S. E. (1988). Brownian motion and stoc hastic calculus, Graduate Texts in Mathematics 113, Springer-Verlag, New York

    1988

  9. [17]

    Korn, R. (1997). Optimal Impulse Control when Control Actions h ave Random Conse- quences. Math. Oper. Res. 22:639-667

    1997

  10. [18]

    Lepeltier, J. P. and Marchal, B. (1984) Theorie Generale du Contro le Impulsionnel Markovien. SIAM J. Control Optim. 22:645-665

    1984

  11. [19]

    Menaldi, J. L. (1980). On the Optimal Impulse Control Problem for D egenerate Diffusions. SIAM J. Control Optim. 18:729-739

    1980

  12. [20]

    Menaldi, J. L. and Robin, M. (2017). On Some Impulse Control Proble ms with Constraint SIAM J Control Optim. 55:3204-3325

    2017

  13. [21]

    Meyer, P.-A. (1975). Renaissance, recollements, m´ elanges, rale ntissement de processus de Markov Ann. Inst. Fourier, 25:465-497

    1975

  14. [22]

    Neveu, J. (1965). Mathematical foundations of the calculus of probability , Holden-Day, Inc., San Francisco, Calif.-London-Amsterdam. Øksendal, B. and Sulem, A. (2005). Applied stochastic control of j ump diffusions, Springer-

    1965

  15. [23]

    Ormeci, M., Dai, J. G. and Vande Vate, J. (2008). Impulse Control o f Brownian Motion: The Average Cost Case. Oper. Res. 56:618-629

    2008

  16. [24]

    and Stettner, L

    Palczewski, J. and Stettner, L. (2017). Impulse Control Maximizin g Average Cost per Unit Time: A Nonuniformly Ergodic Case. SIAM J. Control Optim. 55:936-960

    2017

  17. [25]

    (1977) Optimal Impulse Control of a Diffusion Process w ith Both Fixed and Proportional Costs SIAM J

    Richard, S. (1977) Optimal Impulse Control of a Diffusion Process w ith Both Fixed and Proportional Costs SIAM J. Control Optim. 15:79-91

    1977

  18. [26]

    Runggaldier, W. J. and Yashudo, K. (2018) Classical and restricte d impulse control for the exchange rate under a stochastic trend model J. Econ. Dyn. Control 91:361-390

    2018

  19. [27]

    (1978) Contrˆ ole Impulsionnel des Processus de Markov

    Robin, M. (1978) Contrˆ ole Impulsionnel des Processus de Markov . Th` ese d’´ etat, Paris Dauphine University, Paris

    1978

  20. [28]

    Stettner, L. (1983). On impulsive control with long run average co st criterion Studia Math 76:279-298

    1983

  21. [29]

    Stroock, D. W. and Varadhan, S. R. S. (1979). Multidimensional diffusion processes , vol- ume 233 of Grundlehren der Mathematischen Wissenschaften [Fundamen tal Principles of Mathematical Sciences]. Springer-Verlag, Berlin. 32

    1979

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.