REVIEW 2 major objections 4 minor 43 references
Value of Information under Imprecise Probabilities: Decision-Rule-Specific Values and Fixed-Measure Envelopes on a Credal Set
T0 review · 2 major / 4 minor · reviewed 2026-07-12 · grok-4.5
Pith's one-line read When evidence leaves a whole set of probabilities, the value of information splits into a rule-specific number and a classical envelope that can disagree.
desk verdict Clean, checkable split between rule-specific VOI and fixed-measure envelopes, with real EVPI structure (concavity, generator lower endpoint, LP upper, Gamma-maximin overshoot) that the literature had not packaged this way. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Concavity of EVPI(P) = min_a E_P[M - NB_a] (a pointwise minimum of affine maps), which yields exact lower-endpoint evaluation on the generators of a finitely generated credal set, an LP for the upper endpoint, and an explicit duality-gap example showing that Gamma-maximin EVPI can overshoot the classical envelope.
What would settle it
Construct any decision model and finitely generated convex credal set in which the classical EVPI lower envelope is not equal to the minimum generator EVPI, or in which the Gamma-maximin EVPI cannot exceed the classical upper envelope; either counter-example would refute the structural claims of Theorem 1.
Extended reading notes
Core claim
Under a credal set the value of information must be split into a rule-specific quantity (what the information is worth once a decision rule for imprecision is fixed) and a fixed-measure envelope (the range of classical VOI over every admissible precise measure). EVPI is concave on the set, so its lower envelope endpoint equals the minimum over extreme generators while its upper endpoint may be interior and is given by a finite linear program; the Gamma-maximin EVPI can strictly exceed the entire envelope.
Load-bearing premise
The argument needs a finite action set, uniformly bounded net benefits, and a nonempty compact convex set of measures in a topology that makes the relevant expectations continuous; if the real admissible set is nonconvex or unbounded the exact generator and linear-program endpoints need not hold.
Editorial extensions
If this is right
- A single reference VOI that lies on one side of a research threshold while the envelope straddles that threshold is assumption-dependent and should not be treated as robust.
- Reporting only the largest classical EVPI over generators systematically understates the true upper envelope when the functional is concave.
- Rule-specific lower-expectation VOI is nonnegative whenever the policy class still contains constant (ignore-the-signal) policies; without them free information can have negative value.
- Partial- and sample-information envelopes generally require interior search; vertex enumeration is exact for their upper endpoints only when one action is pre-information optimal for every measure in the set.
Reading between the lines
- Health-technology assessment and research-prioritization bodies that already run probabilistic sensitivity analysis could treat the envelope as a routine robustness report without abandoning their existing single-measure estimators.
- The same split between rule-specific value and fixed-measure envelope should apply to other imprecise-probability decision criteria (minimax regret, weighted lower-upper rules) once the aggregation functional is swapped.
- When the admissible set is defined by p-boxes or moment constraints rather than an explicit finite generator list, the linear-program upper endpoint becomes an infinite-dimensional optimization that still inherits the same concavity structure.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper distinguishes two VOI estimands under a credal set of probability measures: a rule-specific value (e.g., Gamma-maximin / lower-expectation VOI) that fixes a decision criterion for acting under imprecision, and a fixed-measure envelope that records the range of classical single-measure VOI over the set. For EVPI it proves concavity on a compact convex credal set, exact attainment of the lower envelope endpoint at generators of a finitely generated set, a finite LP for the (possibly interior) upper endpoint, and an explicit two-state example in which Gamma-maximin EVPI strictly exceeds the entire classical envelope (Theorem 1). Continuity bounds, conditions under which EVPPI/EVSI endpoints remain generator-exact, nonnegativity of rule-specific VOI when constant policies are available, a unified outer-search estimation procedure, and a chemotherapy decision-model application complete the contribution.
Significance. If the structural claims hold, the paper supplies a clean and usable separation between robustness diagnostics and criterion-dependent decision values for VOI under imprecise probability. The concavity representation EVPI(P)=min_a E_P[D_a], the generator-exact lower endpoint, the LP for the upper endpoint, and the explicit overshoot example are elementary, checkable, and practically consequential: they show that Gamma-maximin VOI is not recovered from envelope endpoints and that vertex enumeration alone understates the upper EVPI range. The continuity bound, the EVPPI/EVSI counterexample, and the worked application that separates robust from measure-dependent research conclusions further strengthen the contribution for medical decision analysis and imprecise-probability decision theory. Reproducible code and explicit finite instances are additional strengths.
major comments (2)
- [Section 5.2–5.3, Tables 2–3] Section 5.2–5.3 and Table 2: the application treats C0 = conv{G1,...,G12} as the domain for exact EVPI endpoints, yet the continuous efficacy–harm correlation is handled only by a separate grid (an inner approximation). The paper correctly flags this, but the headline envelope [£84, £514] and the threshold classifications in Table 3 are therefore exact only for the zero-correlation slice. A short sensitivity statement quantifying how much the upper endpoint and the straddling of τ move when correlation is folded into the outer search (or a refined grid near the reported interior mixture) would make the applied claims load-bearing rather than illustrative.
- [Section 4, Eq. (21), Algorithm 1] Section 4 and Algorithm 1: for continuous or finely discretized Z the rule-specific EVPPI/EVSI estimator (21) requires a controlled policy class, yet the manuscript only sketches the options (structured policies, DP, regression) and notes that Bayes-optimal policies under individual measures are a heuristic. Because the paper’s central distinction is that rule-specific value is not recovered from the envelope, a concrete, reproducible policy class (or an explicit statement that the chemotherapy EVSI numbers are fixed-measure only) is needed so that the estimation procedure actually delivers the rule-specific estimand it defines.
minor comments (4)
- [Figure 1] Figure 1 is conceptually helpful but the threshold-classification bullets are dense; a one-line legend distinguishing robust vs. measure-dependent would improve readability.
- [Section 2.2] Notation for the lower/upper expectations switches between E_P, E□, and V^LE; a short notation table or consistent subscripting would reduce cognitive load.
- [Abstract / Remark 1] The deterministic named-strategy convention that produces the overshoot is well explained in Remark 1, but a single forward pointer from the abstract or introduction would help readers who otherwise misread the claim as contradicting minimax theorems.
- [Proposition 2] Supplement Section 5 is cited for the full piecewise-linear EVPPI counterexample; ensuring that calculation is self-contained in the main text or clearly archived would aid verification of Proposition 2(a).
Circularity Check
No significant circularity: EVPI concavity, generator-exact lower endpoint, LP upper endpoint, and envelope–rule gap are elementary consequences of affine structure under stated assumptions, not fits or self-definitional reductions.
full rationale
The paper's central claims (Theorem 1) follow directly from writing EVPI(P) = min_a EP[Da] with Da = M - NBa ≥ 0 (Eq. 16), each term affine in P, hence the pointwise minimum is concave; Jensen then places the infimum at a generator of a finitely generated convex hull; the upper endpoint is the standard epigraph LP max t s.t. t ≤ ∑ wk Dk,a; and the two-state mirror-image example produces EVPI_LE = 2 > 1 = upper envelope under the deterministic named-strategy convention. These are self-contained mathematical derivations from classical Bayes-value convexity, not parameters fitted to data and re-labeled as predictions, nor uniqueness theorems imported from the author's prior work. Self-citations (Iskandar on p-boxes and uncertainty representations) supply background tools for constructing credal sets and are not load-bearing for the envelope–rule gap. The worked chemotherapy example reports Monte Carlo estimates of the defined functionals; the numerical values are not forced by construction or normalization. Standing assumptions (finite A, bounded net benefits, compact convex P with continuous expectation maps) are stated explicitly and used only for Bauer's principle and continuity; when they fail the paper already notes that generator exactness need not hold (Proposition 2). Score 0 is therefore the correct outcome.
Assumptions & free parameters
free parameters (4)
- Willingness-to-pay λ =
20000 GBP/QALY
- Efficacy–harm correlation interval =
[-0.4, 0.4]
- Discrete evidence-source generators (3×2×2) =
12 generators at zero correlation
- Decision threshold τ for research worth =
example values 50/250/800 GBP
assumptions (7)
- domain assumption Finite action set A with |A|≥2 and deterministic named strategies (randomized mixtures only if added to A).
- domain assumption Net benefits bounded: |NBa(θ)| ≤ B for all a, θ.
- domain assumption Credal set P nonempty compact convex in a locally convex space of signed measures with continuous expectation maps for NBa and M.
- standard math EVPI(P) = E_P[M] - max_a E_P[NBa] with M = max_a NBa (classical perfect-information value).
- domain assumption Lower-expectation / Γ-maximin criterion: rank by inf_P E_P[NB].
- standard math For finitely generated P = conv{G_k}, affine functionals attain extrema at generators; concave continuous functionals attain minima at extreme points (Bauer).
- domain assumption Joint extensions for EVPPI/EVSI: Z = φ(θ) by push-forward; EVSI joints only via design kernel Kd(dy|θ).
invented entities (2)
-
Fixed-measure VOI envelope T_env(P) = [inf T(P), sup T(P)]
independent evidence
-
Rule-specific VOI VOI^ρ_Z under criterion ρ (e.g. lower expectation)
independent evidence
Cite this review
Pith. "Pith review of Value of Information under Imprecise Probabilities: Decision-Rule-Specific Values and Fixed-Measure Envelopes on a Credal Set." pith.science (2026). https://pith.science/paper/TSRWPKOD
@misc{pith2026260706570,
author = {Pith},
title = {Pith review of: Value of Information under Imprecise Probabilities: Decision-Rule-Specific Values and Fixed-Measure Envelopes on a Credal Set},
year = {2026},
howpublished = {\url{https://pith.science/paper/TSRWPKOD}},
note = {Machine review of arXiv:2607.06570}
}
read the original abstract
Value-of-information (VOI) analysis is usually conducted under a single probability measure. However, in practice, the available evidence often pins the measure down only to a set. Consequently, under a set of probability measures, VOI requires different formulations. First, we explicate a rule-specific VOI that fixes a decision rule for acting under imprecision (such as Gamma-maximin) and measures what the information is worth to a decision maker who uses that rule. Second, we derive a fixed-measure envelope that evaluates the classical VOI functional over all admissible precise measures. We formalize this distinction and explicate its consequences for the expected perfect, partial, and sample information. The expected value of perfect information is concave over the credal set. Hence, when the set is generated by finitely many measures, its lower envelope endpoint is obtained exactly from the generators, while its upper endpoint may be interior and is computed by a finite linear program. The Gamma-maximin value, in contrast, can exceed the entire envelope, so a rule-specific value is not recovered from the envelope's endpoints. A continuity bound limits how much the VOI can change as the measure varies, and we identify when the partial- and sample-information endpoints can still be obtained from the generators. Because the single-measure VOI must itself be estimated, the procedure we give combines standard estimators for it with a search over the credal set. By using a worked decision problem, we show how the two quantities separate conclusions that hold across every admissible measure from conclusions that depend on one unidentified choice of measure.
Figures
Reference graph
Works this paper leans on
-
[1]
James C. Felli and Gordon B. Hazen. Sensitivity analysis and the ex- pected value of perfect information.Medical Decision Making, 18(1): 95–109, 1998.doi:10.1177/0272989X9801800117
-
[2]
A. E. Ades, G. Lu, and K. Claxton. Expected value of sample informa- tioncalculationsinmedicaldecisionmodeling.Medical Decision Making, 24(2):207–227, 2004.doi:10.1177/0272989X04263162
-
[3]
Christopher Jackson, Anne Presanis, Stefano Conti, and Daniela De An- gelis. Value of information: Sensitivity analysis and research design in bayesian evidence synthesis.Journal of the American Statistical Associ- ation, 114(528):1436–1449, 2019.doi:10.1080/01621459.2018.1562932
-
[4]
Chap- man and Hall, London, 1991
Peter Walley.Statistical Reasoning with Imprecise Probabilities. Chap- man and Hall, London, 1991
1991
-
[5]
Thomas Augustin, Frank P. A. Coolen, Gert de Cooman, and Matthias C. M. Troffaes, editors.Introduction to Imprecise Probabilities. Wiley, Chichester, 2014
2014
-
[6]
Christopher H. Jackson, Linda D. Sharples, and Simon G. Thomp- son. Structural and parameter uncertainty in bayesian cost-effectiveness models.Journal of the Royal Statistical Society: Series C (Applied Statistics), 59(2):233–253, 2010.doi:10.1111/j.1467-9876.2009.00684.x
-
[7]
Christopher H. Jackson, Laura Bojke, Simon G. Thompson, Karl Clax- ton, and Linda D. Sharples. A framework for addressing structural un- certainty in decision models.Medical Decision Making, 31(4):662–674, 2011.doi:10.1177/0272989X11406986
-
[8]
Rowan Iskandar and Thomas A. Trikalinos. On representations and quantifications of uncertainty.Medical Decision Making, 46(3):267–270, 2026.doi:10.1177/0272989X251413698
Show all 43 references
-
[9]
DeGroot.Optimal Statistical Decisions
Morris H. DeGroot.Optimal Statistical Decisions. McGraw–Hill, New York, 1970
1970
-
[10]
Berger.Statistical Decision Theory and Bayesian Analysis
James O. Berger.Statistical Decision Theory and Bayesian Analysis. Springer, New York, 2 edition, 1985. 35
1985
-
[11]
Using value of information analysis to prioritise health research.PharmacoEconomics, 24(11):1055–1068, 2006.doi:10.2165/00019053-200624110-00003
Karl Claxton and Mark Sculpher. Using value of information analysis to prioritise health research.PharmacoEconomics, 24(11):1055–1068, 2006.doi:10.2165/00019053-200624110-00003
2006 doi
-
[12]
Murray, Hendrik E
Elisabeth Fenwick, Lotte Steuten, Saskia Knies, Salah Ghabri, Anirban Basu, Joseph F. Murray, Hendrik E. Koffijberg, Mark Strong, Gillian D. Sanders Schmidler, and Claire Rothery. Value of information analysis for research decisions–an introduction: Report 1 of the ISPOR value...
2020 doi
-
[13]
Koffijberg, Anirban Basu, Salah Ghabri, Saskia Knies, Joseph F
Claire Rothery, Mark Strong, Hendrik E. Koffijberg, Anirban Basu, Salah Ghabri, Saskia Knies, Joseph F. Murray, Gillian D. Sanders Schmidler, Lotte Steuten, and Elisabeth Fenwick. Value of informa- tion analytical methods: Report 2 of the ISPOR value of information analysis em...
2020 doi
-
[14]
Jackson, Gianluca Baio, Anna Heath, Mark Strong, Nicky J
Christopher H. Jackson, Gianluca Baio, Anna Heath, Mark Strong, Nicky J. Welton, and Edward C. F. Wilson. Value of information anal- ysis in models to inform health policy.Annual Review of Statistics and Its Application, 9:95–118, 2022.doi:10.1146/annurev-statistics-040120- 010730
2022 doi
-
[15]
Maxmin expected utility with non-unique prior.Journal of Mathematical Economics, 18(2):141–153, 1989
Itzhak Gilboa and David Schmeidler. Maxmin expected utility with non-unique prior.Journal of Mathematical Economics, 18(2):141–153, 1989
1989
-
[16]
The value of perfect information.Synthese, 17(1): 408–424, 1967.doi:10.1007/BF00485042
Klemens Szaniawski. The value of perfect information.Synthese, 17(1): 408–424, 1967.doi:10.1007/BF00485042
1967 doi
-
[17]
Nonexpected utility as aversion of informa- tion.Journal of Behavioral Decision Making, 1(3):169–175, 1988
Peter Wakker. Nonexpected utility as aversion of informa- tion.Journal of Behavioral Decision Making, 1(3):169–175, 1988. doi:10.1002/bdm.3960010305
1988 doi
-
[18]
Edward E. Schlee. The value of perfect information in non- linear utility theory.Theory and Decision, 30(2):127–131, 1991. doi:10.1007/BF00134120
1991 doi
-
[19]
Ambiguity and the value of information.Journal of Risk and Uncertainty, 40(2):133–145, 2010.doi:10.1007/s11166-010-9088-7
Arthur Snow. Ambiguity and the value of information.Journal of Risk and Uncertainty, 40(2):133–145, 2010.doi:10.1007/s11166-010-9088-7. 36
2010 doi
-
[20]
Dilation for sets of probabilities.The Annals of Statistics, 21(3):1139–1154, 1993
Teddy Seidenfeld and Larry Wasserman. Dilation for sets of probabilities.The Annals of Statistics, 21(3):1139–1154, 1993. doi:10.1214/aos/1176349254
1993 doi
-
[21]
Myers, and Kari Sentz
Scott Ferson, Vladik Kreinovich, Lev Ginzburg, Davis S. Myers, and Kari Sentz. Constructing probability boxes and dempster–shafer struc- tures. Technical Report SAND2002-4015, Sandia National Laboratories, Albuquerque, NM, 2002
2002
-
[22]
Troy Tucker
Scott Ferson and W. Troy Tucker. Sensitivity in risk analyses with uncertain numbers. Technical Report SAND2006-2801, Sandia National Laboratories, Albuquerque, NM, 2006
2006
-
[23]
Rowan Iskandar. Probability bound analysis: A novel approach for quantifying parameter uncertainty in decision-analytic modeling and cost-effectiveness analysis.Statistics in Medicine, 40(29):6501–6522, 2021.doi:10.1002/sim.9195
2021 doi
-
[24]
Can free evidence be bad? value of information for the imprecise probabilist.Philosophy of Science, 83(1): 1–28, 2016.doi:10.1086/684184
Seamus Bradley and Katie Steele. Can free evidence be bad? value of information for the imprecise probabilist.Philosophy of Science, 83(1): 1–28, 2016.doi:10.1086/684184
2016 doi
-
[25]
A contrast between two decision rules for use with (convex) sets of probabilities:Γ-maximin versus E-admissibility.Synthese, 140(1–2):69–88, 2004
Teddy Seidenfeld. A contrast between two decision rules for use with (convex) sets of probabilities:Γ-maximin versus E-admissibility.Synthese, 140(1–2):69–88, 2004. doi:10.1023/B:SYNT.0000029942.11359.8d
2004 doi
-
[26]
Matthias C. M. Troffaes. Decision making under uncertainty using im- precise probabilities.International Journal of Approximate Reasoning, 45(1):17–29, 2007.doi:10.1016/j.ijar.2006.06.001
2007 doi
-
[27]
On decision making under ambiguous prior and sam- pling information
Thomas Augustin. On decision making under ambiguous prior and sam- pling information. In Gert de Cooman, Terrence L. Fine, Seraf’in Moral, and Teddy Seidenfeld, editors,Proceedings of the Second International Symposium on Imprecise Probabilities and Their Applications, pages 9–
-
[28]
Shaker, Ithaca, NY, 2001
2001
-
[29]
Nathan Huntley and Matthias C. M. Troffaes. Normal form backward induction for decision trees with coherent lower previsions.Annals of Operations Research, 195(1):111–134, 2012.doi:10.1007/s10479-011- 0968-2. 37
2012 doi
-
[30]
Oakley, and Alan Brennan
Mark Strong, Jeremy E. Oakley, and Alan Brennan. Estimating multiparameter partial expected value of perfect information from a probabilistic sensitivity analysis sample: A nonparametric re- gression approach.Medical Decision Making, 34(3):311–326, 2014. doi:10.1177/0272989X13505910
2014 doi
-
[31]
Goldhaber-Fiebert, Gianluca Baio, Nicolas A
Anna Heath, Natalia Kunst, Christopher Jackson, Mark Strong, Fer- nando Alarid-Escudero, Jeremy D. Goldhaber-Fiebert, Gianluca Baio, Nicolas A. Menzies, Hawre Jalal, Alan Brennan, Nicky J. Welton, Ja- son Madan, A. E. Ades, Karl Claxton, Mark Sculpher, and John W. Stevens. Cal...
2020 doi
-
[32]
Natalia Kunst, Edward C. F. Wilson, David Glynn, Fernando Alarid- Escudero, Gianluca Baio, Alan Brennan, Michael Fairley, Jeremy D. Goldhaber-Fiebert, Christopher Jackson, Hawre Jalal, Nicolas A. Men- zies, Mark Strong, Howard Thom, and Anna Heath. Computing the expected value...
2020 doi
-
[33]
R package version 1.0.3
Christopher Jackson.voi: Expected Value of Information. R package version 1.0.3. Collaborative Network for Value of Information (ConVOI),
-
[34]
URLhttps://chjackson.github.io/voi/
-
[35]
The theory of interval-probability as a unifying concept for uncertainty.International Journal of Approximate Reason- ing, 24(2–3):149–170, 2000.doi:10.1016/S0888-613X(00)00032-3
Kurt Weichselberger. The theory of interval-probability as a unifying concept for uncertainty.International Journal of Approximate Reason- ing, 24(2–3):149–170, 2000.doi:10.1016/S0888-613X(00)00032-3
-
[36]
Aughenbaugh and Christiaan J
Jason M. Aughenbaugh and Christiaan J. J. Paredis. The value of using imprecise probabilities in engineering design.Journal of Mechanical Design, 128(4):969–979, 2006.doi:10.1115/1.2204976
2006 doi
-
[37]
Oakley, Alan Brennan, and Penny Breeze
Mark Strong, Jeremy E. Oakley, Alan Brennan, and Penny Breeze. Es- timating the expected value of sample information using the proba- bilistic sensitivity analysis sample: A fast, nonparametric regression- based method.Medical Decision Making, 35(5):570–583, 2015. doi:10.1177/...
2015 doi
-
[38]
Welton and Howard H
Nicky J. Welton and Howard H. Z. Thom. Value of information: We’ve got speed, what more do we need?Medical Decision Making, 35(5): 564–566, 2015.doi:10.1177/0272989X15579164
2015 doi
-
[39]
Claxton, Nicky J
Mathyn Vervaart, Mark Strong, Karl P. Claxton, Nicky J. Welton, Torbjørn Wisløff, and Eline Aas. An efficient method for com- puting expected value of sample information for survival data from an ongoing trial.Medical Decision Making, 42(5):612–625, 2022. doi:10.1177/0272989X211068019
2022 doi
-
[40]
Price, Nicky J
Malcolm J. Price, Nicky J. Welton, Andrew H. Briggs, and A. E. Ades. Model averaging in the presence of structural uncertainty about treatment effects: Influence on treatment decision and ex- pected value of information.Value in Health, 14(2):205–218, 2011. doi:10.1016/j.jval....
2011 doi
-
[41]
Oakley, and Jim Chilcott
Mark Strong, Jeremy E. Oakley, and Jim Chilcott. Managing struc- tural uncertainty in health economic decision models: A discrepancy approach.Journal of the Royal Statistical Society: Series C (Applied Statistics), 61(1):25–45, 2012.doi:10.1111/j.1467-9876.2011.01014.x
2012 doi
-
[42]
Mark Strong and Jeremy E. Oakley. When is a model good enough? de- riving the expected value of model improvement via specifying internal model discrepancies.SIAM/ASA Journal on Uncertainty Quantifica- tion, 2(1):106–125, 2014.doi:10.1137/120889563
2014 doi
-
[43]
Value of information choices that influence estimates: A sys- tematic review of prevailing considerations.Medical Decision Making, 38(7):888–900, 2018.doi:10.1177/0272989X18797948
Hendrik Koffijberg, Claire Rothery, Kalipso Chalkidou, and Janneke Grutters. Value of information choices that influence estimates: A sys- tematic review of prevailing considerations.Medical Decision Making, 38(7):888–900, 2018.doi:10.1177/0272989X18797948. 39
2018 doi
Reviewed July 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.