Pith. sign in

REVIEW 2 major objections 5 minor 26 references

Markov Chain Approximation of Sticky Diffusions and Hamilton-Jacobi-Bellman Equations on Networks

T0 review · 2 major / 5 minor · reviewed 2026-07-30 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An edge-adapted Markov chain converges to sticky and Kirchhoff network diffusions and yields a convergent semi-Lagrangian scheme for network HJB equations.

desk verdict Solid, usable Markov-chain + SL package for sticky/Kirchhoff network diffusions; novelty is the edge-adapted lattice and geometric residence, proofs are standard Ethier–Kurtz / Barles–Souganidis done carefully on a star. read the letter →

arxiv 2607.26709 v1 pith:EUWVAZRK submitted 2026-07-29 math.NA cs.NA

Alessio Basti (Ud'A) , Jules Berry (L2S) , Fabio Camilli (Ud'A) This is my paper · ORCID
classification math.NAcs.NA MSC 60J6049L2549N8065C30
keywords stickydiffusionsnetworksMarkovchainapproximationHamilton–Jacobi–Bellmanequationssemi-LagrangianschemesKirchhoffconditionsviscositysolutions
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Diffusions on networks need junction rules that either conserve flux (Kirchhoff) or hold the process at the vertex for a positive time (sticky). This paper builds a discrete Markov chain on a star network whose spatial steps are scaled to each edge's diffusion coefficient so that a downward step lands exactly on the vertex, with no overshoot or truncation. Stickiness is a geometric holding probability at the vertex, after which the chain is re-emitted along an edge with the right weights. The piecewise-constant interpolation of the chain is shown to converge in distribution, in the Skorokhod space, to the continuous sticky or non-sticky diffusion. The same transition mechanism produces a fully discrete semi-Lagrangian scheme for second-order Hamilton–Jacobi–Bellman equations on the network; the scheme converges locally uniformly to the unique bounded viscosity solution under both Kirchhoff and sticky vertex conditions. A sympathetic reader cares because the construction turns an abstract generator and transmission condition into a concrete, simulable random walk and a practical numerical method for controlled network dynamics with delays at junctions.

What carries the argument

The edge-adapted lattice with spacing σ_ι√h together with the geometric residence-and-re-emission rule at the vertex. Exact hitting of the vertex makes the discrete generator uniformly consistent in the sticky case and only integrably consistent (via vanishing occupation time) in the non-sticky case, so Ethier–Kurtz martingale-problem arguments close.

What would settle it

On a three-edge star with known stickiness and weights, check whether the discrete occupation time at the vertex tends to zero like √h when stickiness is zero and stays positive and increasing with stickiness when it is positive, and whether empirical exit probabilities from a ball match the predicted redistribution weights; if either fails under grid refinement, the claimed weak convergence fails.

Watch

Extended reading notes

Core claim

The piecewise-constant interpolation of the edge-adapted Markov chain converges in distribution in the Skorokhod space to the sticky diffusion when the stickiness parameter is positive and to the non-sticky Walsh-type diffusion when it is zero; the associated fully discrete semi-Lagrangian scheme converges locally uniformly to the unique bounded viscosity solution of the network HJB equation under both junction regimes.

Load-bearing premise

The continuous sticky generator's martingale problem is already well-posed and the network HJB equation already has a comparison principle; both facts are taken from earlier work rather than proved here, and the construction is limited to a star of half-lines with constant diffusion coefficient on each edge.

Editorial extensions

If this is right

  • Sticky and Kirchhoff network diffusions become simulable by a nearest-neighbour random walk with exact vertex hits and a simple geometric hold.
  • Second-order HJB equations on networks with either junction condition admit a monotone, fully discrete semi-Lagrangian scheme that converges to the viscosity solution.
  • When edge lengths divided by diffusion coefficients are commensurable, the same local construction extends to finite metric graphs.
  • Numerical experiments recover the predicted sticky occupation time, exit redistribution, and the approach of the vertex value toward the pure holding cost over the discount rate as stickiness grows.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The exact-hit lattice idea should transfer to other singular boundaries (skew reflections, sticky points on intervals) where classical Euler–Maruyama needs artificial buffers.
  • Once a common time step exists, the same scheme can serve as a building block for mean-field games or control problems on larger networks with sticky hubs.
  • Empirical slope ~0.64 for the HJB error under refinement suggests a concrete rate question that the present consistency-plus-comparison argument does not yet answer.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper constructs an edge-adapted Markov-chain approximation of sticky and non-sticky (Kirchhoff/Walsh) diffusions on a star network. On each edge the chain is a nearest-neighbor walk on the lattice with spacing σ_ι√h; at the vertex a geometric residence/re-emission mechanism encodes stickiness η and redistribution weights γ_ι. The piecewise-constant interpolation is shown to converge in D([0,T];Γ) to the target diffusion via Ethier–Kurtz (tightness by Lyapunov+Aldous; identification by uniform generator consistency for η>0 and integrated consistency plus vanishing occupation time for η=0). From the same transitions the authors derive a fully discrete semi-Lagrangian scheme for the discounted network HJB with sticky/Kirchhoff vertex condition and prove local uniform convergence to the unique bounded viscosity solution by monotonicity, contraction, and Barles–Souganidis. Numerical tests illustrate occupation time, exit probabilities, and value-function behavior.

Significance. The contribution is solid and useful for numerical stochastic control on networks. The edge-adapted lattice that hits the vertex exactly (no overshoot/truncation) and the geometric sticky mechanism are clean design choices that make generator consistency tractable. Treating both η>0 and η=0 in one framework, and transferring the construction to a convergent HJB scheme, fills a genuine gap relative to Euclidean semi-Lagrangian theory and to existing sticky-walk literature. Strengths include fully written Ethier–Kurtz arguments (with appendix proofs), an explicit discrete reflection estimate for the non-sticky case, and a standard but carefully adapted Barles–Souganidis argument at the vertex. Scope is limited to stars with constant per-edge σ (Remarks 3.6–3.7) and well-posedness/comparison are imported from [9] and an adaptation of [6]; within that scope the central claims are well supported.

major comments (2)
  1. [Section 4, Theorem 4.2] Theorem 4.2 asserts comparison for bounded USC/LSC viscosity sub- and supersolutions on the unbounded network by a “straightforward adaptation” of [6]. The manuscript does not spell out which steps of [6] change under unbounded edges, linear growth of the Hamiltonian, or the sticky vertex operator K. Since uniqueness of the HJB limit (Theorem 4.6) rests entirely on this comparison, a short self-contained sketch (or a precise pointer to the modified doubling-variables/penalization argument) should be added so the reader can verify the adaptation covers the stated coefficient class (H1)–(H2) and the sticky condition.
  2. [Section 3.3, Theorem 3.9 and (3.21)–(3.25)] For η=0, identification (Theorem 3.9) uses L² vanishing of the discrete occupation time O^h_T (display (3.21)) together with the bound |G^h φ(O)−Gφ(O)|≤C. The argument is correct under the stated core of C³_b test functions, but the passage from (3.25) to the martingale problem only controls the integrated discrepancy. It would strengthen the result to record explicitly that the same vanishing occupation time also yields the Kirchhoff condition in the limit (so the limit lies in D(G) rather than only solving the edge martingale problem). A one-paragraph clarification would remove any ambiguity about domain membership of the limit.
minor comments (5)
  1. [Section 3.1, Remark 3.1] In (3.2) the residence probability uses √h/(η+√h); the calibration Remark 3.1 then gives expected occupation ηρ∑γ_κσ_κ+O(√h). A brief sentence linking this scaling to the continuous identity (2.5) would help readers unfamiliar with sticky local time.
  2. [Section 5.1, Figure 1] Figure 1 caption says η=0.35 while the panel label writes “= 0.7”; align caption and figure labels.
  3. [Section 5.2] The empirical slope ~0.64 for E_h (Figure 3) is reported without a theoretical rate. Stating that no rate is claimed (already partly done) and, if possible, noting the expected O(√h) consistency barrier from Lemma 3.5 would frame the numerics more clearly.
  4. [Throughout] Typos/notation: “ash→0” missing space (Thm 3.2); “Fully discretesemi-Lagrangian” (p.3); “recalibrated weights” vs later γ_ι without recalibration explanation; AMS class has a stray space in “49L25 ,”.
  5. [References] References [2] and [6] are preprints; ensure arXiv identifiers remain stable or update if published before final version.

Circularity Check

1 steps flagged · score 1.0 of 10

No significant circularity: discrete scheme is built from local consistency and proved to converge to an independently characterized limit; self-citations supply background well-posedness only.

  1. uniqueness imported from authors [Proof of Theorem 3.2 (and analogously Thm 3.9); citation [9]]
    "By the well-posedness of the martingale problem for (G, D(G)), established in [9], every weak limit has the same law. The standard Ethier–Kurtz convergence theorem then implies convergence of the whole family."

    Identification of the weak limit’s law uses uniqueness of the continuous martingale problem from prior work coauthored by Berry. This is load-bearing for the convergence statement but is not circular construction: the discrete generator and scheme are defined independently, and [9] is external theory the approximation converges to, not a fit or self-definition of the discrete objects.

full rationale

The Markov chain (3.3)–(3.4) is defined by edge Euler–Maruyama transitions and a geometric vertex residence/re-emission mechanism calibrated to (η, γ_ι, σ_ι), not by fitting to a target path law or value function. Convergence (Thms 3.2, 3.9) follows the standard Ethier–Kurtz route: Lyapunov compact containment, Aldous tightness, uniform (η>0) or integrated (η=0, via vanishing occupation time O^h_T) generator consistency, then identification of every weak limit as a solution of the martingale problem for (G, D(G)). Uniqueness of that martingale problem is cited from [9] (Berry–Colantoni; overlapping author), and HJB comparison is an adaptation of external [6]; both are ordinary background inputs, not definitions that force the discrete objects equal to their limits. The semi-Lagrangian scheme (4.9)–(4.10) inherits monotonicity/consistency from the controlled chain and converges by Barles–Souganidis (Thm 4.6). Numerical checks illustrate occupation time and exit weights against the same calibration formulas; they are not fitted-then-predicted. No self-definitional loop, no fitted-input-as-prediction, and no ansatz smuggled in as a forced uniqueness claim.

Assumptions & free parameters 0 free parameters · 6 assumptions · 2 invented entities

The work sits on standard stochastic-process and viscosity theory plus domain modeling choices for networks. No empirical free parameters are fitted to force the main theorems; η, γ_ι, σ_ι are part of the model. Continuous well-posedness and comparison are external inputs. The main invented device is the specific discrete residence/re-emission mechanism on the edge-adapted lattice.

assumptions (6)
  • standard math Ethier–Kurtz tightness/martingale-problem convergence framework applies to the piecewise-constant interpolants in D([0,T];Γ).
    Used as the backbone of Theorems 3.2 and 3.9 (Section 3.2).
  • domain assumption The martingale problem for the sticky/non-sticky generator (G,D(G)) is well-posed (uniqueness in law).
    Imported from [9]; required to identify every weak limit of the chain.
  • domain assumption Bounded viscosity sub- and supersolutions of the network HJB satisfy comparison on the unbounded star (Thm 4.2).
    Stated as a straightforward adaptation of [6]; uniqueness of the HJB limit depends on it.
  • domain assumption (H1)–(H2): continuous locally Lipschitz bounded drifts with finite b_ι(0); positive constant diffusion coefficients per edge; γ_ι>0 sum to 1; η≥0.
    Section 2; needed for admissible transition probabilities and generator domain.
  • standard math Barles–Souganidis monotone-consistency-stability theorem yields local uniform convergence once half-relaxed limits are sub/super solutions.
    Invoked in the proof of Theorem 4.6.
  • ad hoc to paper For finite metric graphs, a common time step exists only when intrinsic lengths L_ι/σ_ι are commensurable.
    Remark 3.6; mesh-compatibility restriction not needed on the infinite star but limits claimed generality.
invented entities (2)
  • Edge-adapted lattice Γ_h with spacing σ_ι√h and geometric vertex residence/re-emission (R_n, I_n)
    purpose: Approximate sticky and Kirchhoff network diffusions without overshoot and with correct occupation time and excursion weights.
    Core discrete construction in §3.1; calibrated in Remark 3.1 so limiting exit probabilities match (2.3).
  • Fully discrete semi-Lagrangian operators S^ι_h and S^0_h for sticky/Kirchhoff HJB
    purpose: Turn the controlled chain into a computable dynamic-programming scheme on Γ_h.
    Defined in (4.11)–(4.12); convergence proved via viscosity techniques in §4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Markov Chain Approximation of Sticky Diffusions and Hamilton-Jacobi-Bellman Equations on Networks." pith.science (2026). https://pith.science/paper/EUWVAZRK

@misc{pith2026260726709,
  author       = {Pith},
  title        = {Pith review of: Markov Chain Approximation of Sticky Diffusions and Hamilton-Jacobi-Bellman Equations on Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EUWVAZRK}},
  note         = {Machine review of arXiv:2607.26709}
}
read the original abstract

We propose a discrete Markov-chain approximation of diffusion processes on networks with both Kirchhoff and sticky vertex conditions. Stickiness is modeled by a probabilistic residence mechanism at the vertex, while the motion along the edges follows an Euler-Maruyama-type update at the diffusive scale. We prove that the associated time-interpolated chain converges in distribution to the limiting diffusion in the Skorokhod space using the Ethier-Kurtz framework. Based on this construction, we derive a fully discrete semi-Lagrangian scheme for Hamilton-Jacobi-Bellman equations on networks and establish its convergence using viscosity solution techniques.

Figures

Figures reproduced from arXiv: 2607.26709 by the authors.

Figure 1
Figure 1. Sample paths of the radial component of the approximating chain for [PITH_FULL_IMAGE:figures/full_fig_p020_1.png] view at source ↗
Figure 2
Figure 2. Left: empirical mean of the discrete occupation time at the vertex for different values of [PITH_FULL_IMAGE:figures/full_fig_p021_2.png] view at source ↗
Figure 3
Figure 3. Left: numerical value function on the three edges for [PITH_FULL_IMAGE:figures/full_fig_p022_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Left: numerical value function on the first edge for [PITH_FULL_IMAGE:figures/full_fig_p023_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 1 linked inside Pith

  1. [9]

    Berry and F

    J. Berry and F. Colantoni, Sticky diffusions on star graphs: characterization and Itô formula, Stochastic Process. Appl. 192 (2026), Paper No. 104795, 21 pp

    2026

  2. [6]

    Barles, O

    G. Barles, O. Ley, and E. Topp, Degenerate elliptic PDEs on a network with Kirchhoff conditions, preprint, arXiv:2509.12848, 2025

    arXiv 2025

  3. [1]

    Achdou, M.-K

    Y. Achdou, M.-K. Dao, O. Ley, and N. Tchou, A class of infinite horizon mean field games on networks, Networks and Heterogeneous Media 14 (2019), no. 3, 537–566

    2019

  4. [2]

    Anagnostakis, General diffusions on metric graphs as limits of time-space Markov Chains, preprint, arXiv:2507.23724, 2025

    A. Anagnostakis, General diffusions on metric graphs as limits of time-space Markov Chains, preprint, arXiv:2507.23724, 2025

    arXiv 2025

  5. [3]

    Anagnostakis, A

    A. Anagnostakis, A. Lejay and D. Villemonais, General diffusion processes as limit of time- space Markov chains, Ann. Appl. Probab.33(2023), no. 5, 3620–3651

    2023

  6. [4]

    Aleandri, M

    M. Aleandri, M. Colangeli and D. Gabrielli, A combinatorial representation for the invariant measure of diffusion processes on metric graphs, ALEA Lat. Am. J. Probab. Math. Stat. 18 (2021), no. 2, 1773–1799

    2021

  7. [5]

    Barles and E

    G. Barles and E. Chasseigne,On modern approaches of Hamilton-Jacobi equations and control problems with discontinuities–a guide to theory, applications, and some open problems, Progress in Nonlinear Differential Equations and their Applications PNLDE Subseries in Control, 104 , Birkhäuser/Springer, Cham 2024

    2024

  8. [7]

    Barles and P

    G. Barles and P. E. Souganidis, Convergence of approximation schemes for fully nonlinear second order equations, Asymptot. Anal. 4 (1991), no. 3, 271–283

    1991

Show all 26 references
  1. [8]

    Berry and F

    J. Berry and F. Camilli, Stationary mean field games on networks with sticky transition conditions, ESAIM, Control Optim. Calc. Var. 31, Paper No. 18, 23 p

  2. [10]

    Bonaccorsi and M

    S. Bonaccorsi and M. D’Ovidio, Sticky Brownian motions on star graphs, Fract. Calc. Appl. Anal. 27 (2024), no. 6, 2859–2891

    2024

  3. [11]

    N. M. Bou-Rabee and M. C. Holmes-Cerfon, Sticky Brownian motion and its numerical solution, SIAM Rev. 62 (2020), no. 1, 164–195 28

    2020

  4. [12]

    Camilli and M.Falcone, An approximation scheme for the optimal control of diffusion processes

    F. Camilli and M.Falcone, An approximation scheme for the optimal control of diffusion processes. RAIRO Modél. Math. Anal. Numér. 29 (1995), no. 1, 97–122

    1995

  5. [13]

    Camilli, A

    F. Camilli, A. Festa and D. Schieborn, An approximation scheme for a Hamilton-Jacobi equation defined on a network, Appl. Numer. Math. 73 (2013), 33–47

    2013

  6. [14]

    Calzola, E

    E. Calzola, E. Carlini, X. Dupuis and F. Silva, A semi-Lagrangian scheme for Hamilton-Jacobi- Bellman equations with oblique derivatives boundary conditions, Numer. Math. 153 (2023), no. 1, 49–84

    2023

  7. [15]

    Carlini, A

    E. Carlini, A. Festa and N. Forcadel, A semi-Lagrangian scheme for Hamilton-Jacobi-Bellman equations on networks, SIAM J. Numer. Anal. 59 (2020), no. 6, 3165–3196

    2020

  8. [16]

    S. N. Ethier and T. G. Kurtz,Markov processes, Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics, Wiley, New York, 1986

    1986

  9. [17]

    H. J. Engelbert and G. Peskir, Stochastic differential equations for sticky Brownian motion, Stochastics 86 (2014), no. 6, 993–1021

    2014

  10. [18]

    M. I. Freidlin; A. D. Wentzell, Diffusion processes on graphs and the averaging principle, Ann. Probab. 21 (1993), no. 4, 2215–2245

    1993

  11. [19]

    Freidlin; S.-J

    M. Freidlin; S.-J. Sheu. Diffusion processes on graphs: stochastic differential equations, large deviation principle. Probab. Theory Related Fields 116 (2000), no. 2, 181–220

    2000

  12. [20]

    P. E. Kloeden and E. Platen,Numerical solution of stochastic differential equations, Applications of Mathematics (New York), 23, Springer, Berlin, 1992

    1992

  13. [21]

    V. V. Kostrykin, J. Potthoff and R. Schrader, Brownian motions on metric graphs, J. Math. Phys. 53 (2012), no. 9, 095206, 36 pp

    2012

  14. [22]

    H. J. Kushner and P. G. Dupuis,Numerical methods for stochastic control problems in continuous time, secondedition, ApplicationsofMathematics(NewYork)StochasticModelling and Applied Probability, 24 , Springer, New York, 2001

    2001

  15. [23]

    Martínez and I

    M. Martínez and I. Ohavi, Martingale problem for a Walsh spider process with spinning measure selected from its own local time, Electron. J. Probab.30(2025), Paper No. 22, 39 pp

    2025

  16. [24]

    Ohavi, Quasi linear parabolic pde posed on a network with non linear Neumann boundary condition at vertices, J

    I. Ohavi, Quasi linear parabolic pde posed on a network with non linear Neumann boundary condition at vertices, J. Math. Anal. Appl. 500 (2021), no. 1, Paper No. 125154, 29 pp

    2021

  17. [25]

    Salins and K

    M. Salins and K. Spiliopoulos, Markov processes with spatial delay: path space characterization, occupation time and properties, Stoch. Dyn. 17 (2017), no. 6, 1750042, 21 pp

    2017

  18. [26]

    Touhami, On skew sticky Brownian motion, Statist

    W. Touhami, On skew sticky Brownian motion, Statist. Probab. Lett.173(2021), Paper No. 109086, 9 pp. 29

    2021

Pith tools

Reviewed July 30, 2026 · model on record in the stance chip above.