pith. sign in

arxiv: 2606.25716 · v1 · pith:CPB35UVDnew · submitted 2026-06-24 · 📊 stat.ME · math.PR

Imprecise Transition Matrices for Markov Cohort Models: Lower and Upper Expectations with a Practical Health Economic Application

Rowan Iskandar This is my paper

Pith reviewed 2026-06-25 20:28 UTC · model grok-4.3

classification 📊 stat.ME math.PR
keywords imprecise probabilityMarkov cohort modelstransition matriceslower expectationhealth economicsImprecise Dirichlet Modelcost-effectiveness analysis
0
0 comments X

The pith

Markov cohort models with sets of admissible transition matrices allow exact computation of lower and upper accumulated outcomes via recursive operators.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper shows that when transition counts and other evidence identify a set of admissible matrices rather than one unique matrix, the lower and upper expectations of finite-horizon accumulated outcomes can be obtained by applying lower and upper transition operators in a Bellman-style recursion. This handles the common situation in health economics where evidence does not justify a single precise matrix. The approach proves that the operators satisfy coherence and reduce to the classical case when the set shrinks to a singleton. In a cost-effectiveness analysis of patent foramen ovale closure, the single empirical matrix slightly favors the procedure while the set-based calculation produces an interval for incremental net monetary benefit that includes zero.

Core claim

Under non-empty compact separately specified outgoing-row sets, the lower and upper accumulated outcomes are computed exactly by Bellman-style lower and upper transition operators. Multinomial transition counts induce such sets through the Imprecise Dirichlet Model, and the resulting interval for incremental net monetary benefit in the stroke example crosses zero.

What carries the argument

Bellman-style lower and upper transition operators that act on evidence-induced sets of admissible transition matrices.

If this is right

  • The lower and upper expectations satisfy an envelope theorem and coherence properties of the lower transition operator.
  • Algebraic conditions on the admissible set identify when a single selected matrix produces a non-robust decision.
  • The formulation reduces exactly to the classical precise Markov cohort model when the admissible set is a singleton.
  • The method supplies both a lower-expectation formulation and a diagnostic for decisions sensitive to unresolved transition probabilities.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same operator construction could be tested on other finite-horizon cohort models to see how often the decision interval changes sign.
  • Incorporating additional structural constraints or treatment-effect data directly into the admissible sets would be a direct next step.
  • The crossing-zero result in the example suggests the method can flag cases where current evidence is insufficient to support a firm policy choice.

Load-bearing premise

The admissible set of transition matrices is non-empty, compact, and separately specified row by row, and is generated from multinomial counts by the Imprecise Dirichlet Model.

What would settle it

Apply the lower and upper operators to the Imprecise Dirichlet Model sets derived from the transition counts in the patent foramen ovale example and check whether the resulting interval for incremental net monetary benefit actually contains zero.

Figures

Figures reproduced from arXiv: 2606.25716 by Rowan Iskandar.

Figure 1
Figure 1. Figure 1: Empirical point incremental net monetary benefit (INMB) and Imprecise Dirich [PITH_FULL_IMAGE:figures/full_fig_p016_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Decision outcomes under increasing evidence, for a ground truth in which medical [PITH_FULL_IMAGE:figures/full_fig_p019_2.png] view at source ↗
read the original abstract

In applied health research, Markov cohort models are built on a precisely specified transition probability matrix. However, in many applications, the available evidence -- transition counts, structural constraints, and treatment-effect data -- identifies a set of admissible matrices rather than one uniquely justified matrix. This paper formulates an imprecise-probability extension in which inference yields lower and upper expectations over an evidence-compatible set of precise Markov cohort models. The contribution differs from existing imprecise Markov-chain work by focusing on finite-horizon cohort trajectories, additive accumulated outcomes, and transition matrices constructed from empirical transition counts. Under non-empty compact separately specified outgoing-row sets, the lower and upper accumulated outcomes are computed exactly by Bellman-style lower and upper transition operators. We prove the envelope theorem, reduction to the classical model, coherence properties of the lower transition operator, and algebraic conditions under which a single selected matrix yields a non-robust decision. We then show how multinomial transition counts induce admissible matrix sets through the Imprecise Dirichlet Model. A real-world cost-effectiveness example of patent foramen ovale closure after cryptogenic stroke illustrates the practical consequence: the empirical transition matrix slightly favors closure, whereas the imprecise analysis yields an incremental net monetary benefit interval crossing zero. The method provides both a rigorous lower-expectation formulation and a practical diagnostic for decisions that depend on transition probabilities not fully resolved by the evidence.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 3 minor

Summary. The paper develops an imprecise-probability extension of finite-horizon Markov cohort models for health-economic evaluation. Transition matrices are replaced by credal sets of admissible matrices induced row-wise from multinomial transition counts via the Imprecise Dirichlet Model. Under the maintained assumptions of non-empty, compact, separately specified outgoing-row credal sets, the paper proves that lower and upper expectations of additive accumulated outcomes are obtained exactly by iterating the corresponding lower and upper transition operators (envelope theorem, reduction to the precise case, and coherence of the lower operator). It further derives algebraic conditions under which a single matrix yields a non-robust decision and illustrates the method on a patent-foramen-ovale-closure cost-effectiveness analysis, where the point estimate slightly favors closure while the imprecise interval for incremental net monetary benefit crosses zero.

Significance. If the envelope theorem and coherence results hold, the work supplies a rigorous, computationally tractable route to propagating epistemic uncertainty in transition probabilities through cohort models with additive outcomes. The exact operator iteration, the explicit reduction to the classical model, and the diagnostic for decisions that hinge on unresolved transition probabilities are concrete strengths. The real-data example shows that the framework can alter the robustness assessment of a policy conclusion, which is directly relevant to applied health economics.

minor comments (3)
  1. [theoretical development of lower/upper operators] The statement of the envelope theorem (presumably in the theoretical section following the operator definitions) would benefit from an explicit display of the induction step that shows the lower accumulated expectation equals the iterated lower operator; the current high-level description leaves the precise inductive hypothesis unclear.
  2. [IDM construction from multinomial counts] In the IDM construction section, the paper asserts that the row-wise credal sets remain separately specified and compact; a short explicit verification that the lower and upper probability bounds induced by the IDM satisfy the separate-specification condition for the outgoing rows would strengthen the claim.
  3. [patent foramen ovale closure example] The application section reports that the imprecise INMB interval crosses zero while the empirical matrix does not; tabulating the lower and upper INMB values alongside the precise point estimate and the width of the interval would make the practical consequence easier to assess.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the detailed and positive summary of our paper, the recognition of its contributions regarding the envelope theorem, coherence results, and the practical health-economic application, and the recommendation for minor revision. No specific major comments were listed in the report, so we have no individual points to address point-by-point. We remain available to incorporate any minor suggestions once they are provided.

Circularity Check

0 steps flagged

No significant circularity identified

full rationale

The derivation relies on standard imprecise-probability concepts (credal sets that are non-empty, compact, and separately specified) and the externally defined Imprecise Dirichlet Model to induce admissible transition matrices from multinomial counts. The central result—that lower/upper finite-horizon accumulated expectations are obtained exactly by iterating lower/upper transition operators—is supported by explicit proofs of the envelope theorem, reduction to the precise case, and coherence properties. These steps are self-contained mathematical arguments that do not reduce the claimed operators or the IDM-induced sets to any fitted parameter or self-citation defined inside the paper. No load-bearing premise collapses to a renaming, ansatz smuggling, or self-referential definition.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim rests on the assumption that admissible matrix sets are non-empty, compact, and separately specified, plus the use of the Imprecise Dirichlet Model to generate those sets from counts. No free parameters or invented entities are explicitly named in the abstract.

axioms (2)
  • domain assumption The set of admissible transition matrices is non-empty, compact, and separately specified for outgoing rows.
    Required for the exact computation by lower and upper transition operators.
  • domain assumption Multinomial transition counts induce admissible matrix sets through the Imprecise Dirichlet Model.
    Stated as the mechanism that produces the evidence-compatible set.

pith-pipeline@v0.9.1-grok · 5768 in / 1403 out tokens · 37968 ms · 2026-06-25T20:28:06.814673+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

18 extracted references

  1. [1]

    J. R. Beck, S. G. Pauker, The Markov process in medical prognosis, Medical Decision Making 3 (4) (1983) 419–458

    1983

  2. [2]

    F. A. Sonnenberg, J. R. Beck, Markov models in medical decision mak- ing: A practical guide, Medical Decision Making 13 (4) (1993) 322–338

    1993

  3. [3]

    Iskandar, A theoretical foundation for state-transition cohort models in health decision analysis, PLOS ONE 13 (12) (2018) e0205543

    R. Iskandar, A theoretical foundation for state-transition cohort models in health decision analysis, PLOS ONE 13 (12) (2018) e0205543

    2018

  4. [4]

    Walley, Statistical Reasoning with Imprecise Probabilities, Chapman and Hall, London, 1991

    P. Walley, Statistical Reasoning with Imprecise Probabilities, Chapman and Hall, London, 1991. 25

    1991

  5. [5]

    Augustin, F

    T. Augustin, F. P. A. Coolen, G. de Cooman, M. C. M. Troffaes (Eds.), Introduction to Imprecise Probabilities, John Wiley & Sons, Chichester, UK, 2014

    2014

  6. [6]

    M. C. M. Troffaes, G. de Cooman, Lower Previsions, Wiley, Chichester, 2014

    2014

  7. [7]

    de Cooman, F

    G. de Cooman, F. Hermans, E. Quaeghebeur, Imprecise Markov chains and their limit behavior, Probability in the Engineering and Informa- tional Sciences 23 (4) (2009) 597–635

    2009

  8. [8]

    Hermans, D

    F. Hermans, D. Škulj, Stochastic processes, in: T. Augustin, F. P. A. Coolen, G. de Cooman, M. C. M. Troffaes (Eds.), Introduction to Im- precise Probabilities, John Wiley & Sons, 2014, pp. 258–278

    2014

  9. [9]

    T. Krak, J. De Bock, A. Siebes, Imprecise continuous-time Markov chains, International Journal of Approximate Reasoning 88 (2017) 452– 528

    2017

  10. [10]

    Škulj, Efficient computation of the bounds of continuous time impre- cise Markov chains, Applied Mathematics and Computation 250 (2015) 165–180

    D. Škulj, Efficient computation of the bounds of continuous time impre- cise Markov chains, Applied Mathematics and Computation 250 (2015) 165–180

    2015

  11. [11]

    de Cooman, J

    G. de Cooman, J. De Bock, S. Lopatatzidis, Imprecise stochastic pro- cesses in discrete time: Global models, imprecise Markov chains, and ergodic theorems, International Journal of Approximate Reasoning 76 (2016) 18–46

    2016

  12. [12]

    T’Joens, J

    N. T’Joens, J. De Bock, Average behaviour in discrete-time imprecise Markov chains: A study of weak ergodicity, International Journal of Approximate Reasoning 132 (2021) 181–205

    2021

  13. [13]

    Nilim, L

    A. Nilim, L. El Ghaoui, Robust control of Markov decision processes with uncertain transition matrices, Operations Research 53 (5) (2005) 780–798

    2005

  14. [14]

    G. N. Iyengar, Robust dynamic programming, Mathematics of Opera- tions Research 30 (2) (2005) 257–280

    2005

  15. [15]

    Wiesemann, D

    W. Wiesemann, D. Kuhn, B. Rustem, Robust Markov decision pro- cesses, Mathematics of Operations Research 38 (1) (2013) 153–183. 26

    2013

  16. [16]

    M. H. Leppert, S. N. Poisson, J. D. Carroll, D. E. Thaler, C. H. Kim, K. D. Orjuela, B. S. Wiggins, M. Berman, C. A. Velez, Cost-effectiveness of patent foramen ovale closure versus medical therapy for secondary stroke prevention, Stroke 49 (2018) 1443–1450

    2018

  17. [17]

    J.-L. Mas, G. Derumeaux, B. Guillon, E. Massardier, H. Hos- seini, L. Mechtouff, C. Arquizan, Y. Béjot, F. Vuillier, O. Detante, C. Guidoux, S. Canaple, C. Vaduva, N. Dequatre-Ponchelle, I. Sibon, P. Garnier, A. Ferrier, S. Timsit, E. Robinet-Borgomano, D. Sablot, J.-C. Lacour, M. Zuber, P. Favrole, J.-F. Pinel, M. Apoil, P. Reiner, G. Leftheriotis, P. Gu...

    2017

  18. [18]

    P. H. Lee, J.-K. Song, J. S. Kim, J. H. Heo, S.-W. Lee, D.-H. Kim, J.-H. Song, D.-W. Kang, S.-W. Kwon, D.-H. Kang, C. W. Lee, M.-K. Hong, J.-J. Kim, S.-J. Park, Cryptogenic stroke and high-risk patent foramen ovale: The DEFENSE-PFO trial, Journal of the American College of Cardiology 71 (2018) 2335–2342. 27

    2018