REVIEW 2 major objections 7 minor 38 references
Decision-analytical models as causal models
T0 review · 2 major / 7 minor · reviewed 2026-07-13 · grok-4.5
Pith's one-line read Decision-analytical models answer causal questions and inherit causal bias, which can be decomposed into model structure bias and parameter target bias.
desk verdict Clean formalization of multi-source decision models as causal models, with a usable bias decomposition; definitional but solid and worth engaging. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The decomposition of decision-analytical model bias into model bias plus target bias (the latter further split into internal- and external-validity components for each cost and effectiveness term under each intervention), obtained by contrasting the true functional of potential outcomes with the same functional evaluated under the empirical model that plugs in multi-source estimates.
What would settle it
A controlled simulation in which the true joint distribution of potential outcomes is known, a decision tree is correctly specified, yet a single unconventional conditional probability is deliberately misspecified by omitting one confounder; if the resulting ICER still lies on the correct side of a pre-set willingness-to-pay threshold, the claim that bias routinely propagates to reverse decisions is weakened.
Extended reading notes
Core claim
A decision-analytical model used for cost-effectiveness or related evaluations is a causal model whose total bias relative to the true counterfactual estimand equals the sum of model bias (structural misspecification of the data-generating process) and target bias (internal- or external-validity failure of any input causal parameter). Target bias can appear even when every source is a simple observational or trial data set, because the model demands parameters such as potential outcomes conditional on other potential outcomes that have no straightforward observed counterpart.
Load-bearing premise
The functions that convert health events into total costs and into quality-adjusted life years are known, correctly specified, and evaluate the same set of events under every intervention.
Editorial extensions
If this is right
- Analysts must write down the target decision-analytical model and its accompanying causal graph before estimating any input, so that every required potential-outcome parameter is explicit.
- Each input parameter needs its own identification argument and its own check for transportability to the decision context; shared assumptions across parameters cannot be assumed.
- Routine sensitivity analysis of random error is incomplete; systematic target bias should be quantified with formal causal bias analysis and allowed to revise the decision.
- When a model’s ICER sits near a willingness-to-pay threshold, even modest target bias in one branch can flip the recommended intervention.
Reading between the lines
- The same bias decomposition applies immediately to Markov and microsimulation models once the time index is added, so the paper’s framework already covers the models most used in practice.
- Journals and HTA bodies could require authors to report the causal graph and the list of identifying assumptions for every chance node, turning the paper’s recommendation into a reporting standard.
- Because bias propagation is non-linear for ratio estimands such as the ICER, small absolute errors in late branches can dominate; this suggests prioritising bias analysis on parameters that appear deep in the tree.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper formalizes decision-analytical models used in health economic evaluation as causal models in the potential-outcomes framework. It defines the target estimand T (e.g., the counterfactual ICER) as a functional of expected potential costs and effectiveness, introduces the idealized target model T_M and the empirical model estimate ˆT_M, and defines total decision-analytical model bias as T − E[ˆT_M]. This bias is decomposed into model bias (structural misspecification of the data-generating process) and target bias (internal and external validity failures for the causal input parameters), with the latter further split by cost/effectiveness component and intervention level (Eqs. 8–9; Appendix B). Using a rollback decision tree, SWIGs, and three data-availability scenarios, the authors show how unconventional conditional potential-outcome parameters arise, how identifying assumptions depend on source structure (including a strong complete-mediation case), and how bias in a single parameter can propagate to the decision criterion. Numerical simulations under a known SCM (Appendix A) and released code illustrate confounding, selection, and propagation effects.
Significance. The contribution is primarily conceptual and definitional, but it is load-bearing for practice: health economic decision models routinely synthesize multi-source causal parameters without a shared language for what must be identified and where bias enters. Making model bias versus target bias (internal/external) explicit, tying the target model to a causal factorization, and showing propagation even in a simple tree are useful and overdue. Strengths include algebraic consistency of the bias decomposition, careful scenario-by-scenario identification arguments, reproducible simulations with known ground truth, and open code. The weakest maintained assumption—that the cost and effectiveness maps f and g are known, correctly specified, and evaluate the same outcome set under every intervention—is stated rather than hidden, so it does not undermine the decomposition itself. If adopted, the framework would improve reporting of assumptions and motivate routine causal bias analysis alongside conventional sensitivity analysis.
major comments (2)
- The abstract and introduction present model bias and target bias as co-equal components of decision-analytical model bias, yet §4.1 explicitly sets a full treatment of model bias outside the paper’s scope and the numerical work (Appendix A) is constructed so that model bias is zero by design. The central definitional claim remains intact, but the manuscript should either (i) rebalance the abstract/intro to state that the primary development is target bias under a fixed structure, or (ii) add a short, concrete quantification of model bias (e.g., a misspecified factorization or omitted dependence between V2 and V3 | A, V1 as already mentioned in §3) so that both components are illustrated at comparable depth.
- §4.2.2 Scenario (iii), assumptions (A.4)–(A.8) and (A.13)–(A.16): identification of P(Ca | Ba) when A is unobserved in the (B,C) source rests on complete mediation of A’s effect on C through B plus no unmeasured A–C or B–C confounding after conditioning. This is correctly derived but is much stronger than the backdoor cases in (i)–(ii). Because the paper’s central practical message is that target bias can arise even in simple settings and can flip decisions, a brief sensitivity or partial-identification discussion for this scenario (or an explicit statement that Scenario (iii) is mainly cautionary) would better support the claim that analysts should scrutinize such parameters before plugging them into the tree.
minor comments (7)
- §2.1.2: typographical duplication “causal decision-analytical analytical model”.
- Figure 1 is referenced as “Flow of reasoning, from research question to estimate” but is not described in the text beyond the caption; a one-sentence walkthrough of the nodes would help readers who encounter the figure before §3’s definitions of T, TM, and ˆTM.
- Notation density (PM, PM|S, ˆPM|S, S, Sk, etc.) is high. A small notation table early in §3 would reduce cognitive load without changing content.
- §4.1.1 and Figure 5: the point that a bare decision tree is compatible with more than one causal structure is important; consider stating explicitly in the figure caption which edges differ from Figure 3 so the contrast is immediate.
- Appendix A tables report E[ICER] as the mean of iteration-specific ICERs alongside a “true ICER” from true marginals. A short note that the mean of ratios is not the ratio of means (and why both are shown) would avoid misreading by applied readers.
- Discussion gestures at Markov/microsimulation extensions and multi-state/causal survival settings. One or two sentences on which pieces of the bias decomposition carry over unchanged versus which require time-indexed potential outcomes would strengthen the “more generally” claim without expanding scope.
- References and cross-links are generally good; ensure the GitHub URL in the data-availability statement remains stable and that the simulation README maps runs to Appendix A.1–A.3 scenarios.
Circularity Check
No significant circularity: bias decomposition is definitional, simulations recover known SCM truth under correct identification, and no load-bearing self-citation or fitted-as-prediction steps exist.
full rationale
The paper's central objects (decision-analytical model bias, model bias, target bias, internal/external validity bias) are introduced by explicit definition as differences between population functionals of potential outcomes under the true data-generating process P versus under a model M with true or estimated parameters (Eqs. 5–9 and Appendix B). These are not fitted quantities later re-labeled as predictions, nor do any equations reduce a claimed first-principles result to an earlier free parameter. The algebraic rearrangement of the ICER bias into additive components is tautological once the components are defined that way; it does not smuggle an empirical claim. Appendix A simulations generate data from a fully known structural causal model whose parameters are never estimated from the ICER itself; they simply demonstrate recovery (or failure) of the known ground-truth costs/QALYs/ICER under correct versus incomplete adjustment. Citations are to standard external causal-inference literature (Hernán & Robins, Pearl, etc.) and do not form a self-citation chain that forces uniqueness or forbids alternatives. No ansatz is imported via prior author work, and no known empirical pattern is merely renamed. The framework is therefore self-contained against its own definitions and simulations; circularity score is zero.
Assumptions & free parameters
assumptions (5)
- domain assumption Stable Unit Treatment Value Assumption (consistency, no interference, no multiple versions) for both treatment assignment and selection
- domain assumption Conditional exchangeability of treatment assignment given measured covariates L (or partial exchangeability for single-arm sources)
- domain assumption Positivity of treatment assignment and of selection within every relevant covariate stratum
- ad hoc to paper Complete mediation of the effect of A on C through B when A is unobserved in the source that supplies (B,C)
- domain assumption Cost and effectiveness functions f and g are known, correctly specified, and evaluate the same outcome set under every intervention
invented entities (2)
-
model bias
-
target bias
Cite this review
Pith. "Pith review of Decision-analytical models as causal models." pith.science (2026). https://pith.science/paper/FYG5IPHB
@misc{pith2026260709397,
author = {Pith},
title = {Pith review of: Decision-analytical models as causal models},
year = {2026},
howpublished = {\url{https://pith.science/paper/FYG5IPHB}},
note = {Machine review of arXiv:2607.09397}
}
read the original abstract
Health economic evaluations are fundamentally concerned with answering causal questions by targeting estimands that contrast the costs and health consequences that would be observed under at least two different interventions. This requires the joint distribution of potential outcomes under each level of intervention, which, with appropriate causal assumptions, can in principle be identified from the joint distribution of observed health outcomes. Such data, however, are rarely available from a single source. This limitation has motivated the use of decision-analytical models to approximate the joint distribution of outcomes under each intervention directly, informed by causal parameters drawn and synthesized from multiple sources, so that the potential outcomes of interest can be approximated as an expectation over the model-implied outcome trajectories. The validity of this approach, however, depends on the credibility of the underlying assumptions. In this work, we formalize this procedure explicitly as a task of causal inference, thereby defining and decomposing decision-analytical model bias into components arising from model structure (model bias) and input parameters (target bias). Because decision-analytical models often rely on unconventional target parameters lacking straightforward observable analogues, and because bias in these parameters can propagate through the model, target bias may arise even in simple settings, a point of central focus in this work. More broadly, this work provides a unifying foundation for medical decision-analytical modelling and causal inference, making explicit the potential for decision-analytical model bias and the role of causal assumptions contributing to it. Ultimately, the resulting clinical decision is only as credible as the assumptions underlying it.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
When should decision-analytic modeling be used in the economic evaluation of health care? The European journal of health economics, formerly: HEPAC 2003; 4:143–50
Siebert U. When should decision-analytic modeling be used in the economic evaluation of health care? The European journal of health economics, formerly: HEPAC 2003; 4:143–50
2003
-
[2]
Causal Inference: What If
Hern´ an MA and Robins JM. Causal Inference: What If. Boca Raton: Chapman & Hall/CRC, 2020
2020
-
[3]
Causal evidence in health decision making: methodological approaches of causal inference and health decision science
K¨ uhne F, Schomaker M, Stojkov I, Jahn B, Conrads-Frank A, Siebert S, Sroczynski G, Puntscher S, Schmid D, Schnell-Inderst P, et al. Causal evidence in health decision making: methodological approaches of causal inference and health decision science. GMS German Medical Science 2022; 20:Doc12
2022
-
[4]
Decision making in health and medicine: integrating evidence and values
Hunink MM, Weinstein MC, Wittenberg E, Drummond MF, Pliskin JS, Wong JB, and Glasziou PP. Decision making in health and medicine: integrating evidence and values. Cambridge univer- sity press, 2014
2014
-
[5]
Individualization at the heart of comparative effectiveness research: the time for i-CER has come
Basu A. Individualization at the heart of comparative effectiveness research: the time for i-CER has come. Medical Decision Making 2009; 29:NP9–NP11
2009
-
[6]
The role of the expected value of individualized care in cost-effectiveness analyses and decision making
Gestel A van, Grutters J, Schouten J, Webers C, Beckers H, Joore M, and Severens J. The role of the expected value of individualized care in cost-effectiveness analyses and decision making. Value in Health 2012; 15:13–21
2012
-
[7]
Causal inference in statistics, social, and biomedical sciences
Imbens GW and Rubin DB. Causal inference in statistics, social, and biomedical sciences. Cam- bridge university press, 2015
2015
-
[8]
Statistics and causal inference
Holland PW. Statistics and causal inference. Journal of the American statistical Association 1986; 81:945–60
1986
Show all 38 references
-
[9]
Beyond conditional averages: Estimating the individual causal effect distribution
Post RA and Van Den Heuvel ER. Beyond conditional averages: Estimating the individual causal effect distribution. Journal of Causal Inference 2025; 13:20240007
2025
-
[10]
Using big data to emulate a target trial when a randomized trial is not available
Hern´ an MA and Robins JM. Using big data to emulate a target trial when a randomized trial is not available. American journal of epidemiology 2016; 183:758–64
2016
-
[11]
Target validity and the hierarchy of study designs
Westreich D, Edwards JK, Lesko CR, Cole SR, and Stuart EA. Target validity and the hierarchy of study designs. American journal of epidemiology 2019; 188:438–43
2019
-
[12]
External validity
Findley MG, Kikuta K, and Denly M. External validity. Annual review of political science 2021; 24:365–93
2021
-
[13]
A review of generalizability and transportability
Degtiar I and Rose S. A review of generalizability and transportability. Annual Review of Statis- tics and Its Application 2023; 10:501–24
2023
-
[14]
A taxonomy of model structures for economic evaluation of health technologies
Brennan A, Chick SE, and Davies R. A taxonomy of model structures for economic evaluation of health technologies. Health economics 2006; 15:1295–310
2006
-
[15]
Causal diagrams for epidemiologic research
Greenland S, Pearl J, and Robins JM. Causal diagrams for epidemiologic research. Epidemiology 1999; 10:37–48
1999
-
[16]
Single world intervention graphs (SWIGs): A unification of the counterfactual and graphical approaches to causality
Richardson TS and Robins JM. Single world intervention graphs (SWIGs): A unification of the counterfactual and graphical approaches to causality. Center for the Statistics and the Social Sciences, University of Washington Series. Working Paper 2013; 128:2013
2013
-
[17]
Directed Acyclic Graphs in Decision-Analytic Modeling: Bridging Causal Infer- ence and Effective Model Design in Medical Decision Making
Dijk SW, Korf M, Labrecque JA, Pandya A, Ferket BS, Hallsson LR, Wong JB, Siebert U, and Hunink MM. Directed Acyclic Graphs in Decision-Analytic Modeling: Bridging Causal Infer- ence and Effective Model Design in Medical Decision Making. Medical Decision Making 2025 :0272989X241310898
2025
-
[18]
Causal inference in statistics: An overview
Pearl J. Causal inference in statistics: An overview. 2009
2009
-
[19]
Toward causally interpretable meta-analysis: transporting inferences from multiple randomized trials to a new target population
Dahabreh IJ, Petito LC, Robertson SE, Hern´ an MA, and Steingrimsson JA. Toward causally interpretable meta-analysis: transporting inferences from multiple randomized trials to a new target population. Epidemiology 2020; 31:334–44
2020
-
[20]
Efficient and robust methods for causally interpretable meta-analysis: Transporting inferences from multiple random- ized trials to a target population
Dahabreh IJ, Robertson SE, Petito LC, Hern´ an MA, and Steingrimsson JA. Efficient and robust methods for causally interpretable meta-analysis: Transporting inferences from multiple random- ized trials to a target population. Biometrics 2023; 79:1057–72 25
2023
-
[21]
Target trial emulation: a framework for causal inference from observational data
Hern´ an MA, Wang W, and Leaf DE. Target trial emulation: a framework for causal inference from observational data. Jama 2022; 328:2446–7
2022
-
[22]
Experimental and Quasi-Experimental Designs for Gen- eralized Causal Inference
Shadish W, Cook T, and Campbell D. Experimental and Quasi-Experimental Designs for Gen- eralized Causal Inference. English. 2nd ed. Cengage Learning, 2002
2002
-
[23]
Causal transportability with limited experiments.Proceedings of the AAAI Conference on Artificial Intelligence
Bareinboim E and Pearl J. Causal transportability with limited experiments.Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 27. 1. 2013 :95–101
2013
-
[24]
External validity in fuzzy regression discontinuity designs
Bertanha M and Imbens GW. External validity in fuzzy regression discontinuity designs. Journal of Business & Economic Statistics 2020; 38:593–612
2020
-
[25]
External validity: From do-calculus to transportability across popu- lations.Probabilistic and causal inference: The works of Judea Pearl
Pearl J and Bareinboim E. External validity: From do-calculus to transportability across popu- lations.Probabilistic and causal inference: The works of Judea Pearl. 2022 :451–82
2022
-
[26]
Adjustment criteria for generalizing experimental findings
Correa J, Tian J, and Bareinboim E. Adjustment criteria for generalizing experimental findings. International Conference on Machine Learning. PMLR. 2019 :1361–9
2019
-
[27]
Causal inference and the data-fusion problem
Bareinboim E and Pearl J. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences 2016; 113:7345–52
2016
-
[28]
Positivity: Identifiability and estimability
Zivich PN, Cole SR, and Westreich D. Positivity: Identifiability and estimability. arXiv preprint arXiv:2207.05010 2022
2022 arXiv
-
[29]
Diagnosing and responding to violations in the positivity assumption
Petersen ML, Porter KE, Gruber S, Wang Y, and Van Der Laan MJ. Diagnosing and responding to violations in the positivity assumption. Statistical methods in medical research 2012; 21:31–54
2012
-
[30]
Concerning the consistency assumption in causal inference
VanderWeele TJ. Concerning the consistency assumption in causal inference. Epidemiology 2009; 20:880–3
2009
-
[31]
Generalizing study results: a potential outcomes perspective
Lesko CR, Buchanan AL, Westreich D, Edwards JK, Hudgens MG, and Cole SR. Generalizing study results: a potential outcomes perspective. Epidemiology 2017; 28:553–61
2017
-
[32]
Generalizing causal inferences from individuals in randomized trials to all trial-eligible individuals
Dahabreh IJ, Robertson SE, Tchetgen EJ, Stuart EA, and Hern´ an MA. Generalizing causal inferences from individuals in randomized trials to all trial-eligible individuals. Biometrics 2019; 75:685–94
2019
-
[33]
A graphical description of partial ex- changeability
Sarvet AL, Wanis KN, Stensrud MJ, and Hern´ an MA. A graphical description of partial ex- changeability. Epidemiology 2020; 31:365–8
2020
-
[34]
Analysis of a decision tree: roll back
LLC TS. Analysis of a decision tree: roll back. Available from:https://www.treeage.com/ help/Content/31-Analyzing-Decision-Trees/3-Roll-back.htm[Accessed on: 2024 Oct 4]
2024
-
[35]
Applying quantitative bias analysis to epidemiologic data
Fox MP, MacLehose RF, and Lash TL. Applying quantitative bias analysis to epidemiologic data. Vol. 10. Springer, 2021
2021
-
[36]
Causal clarity in statistical software
Korf MN, Van Geloven N, Krijthe JH, and Labrecque JA. Causal clarity in statistical software. International journal of epidemiology 2025; 54:dyaf136
2025
-
[37]
Ziektelast in de praktijk: de theorie en praktijk van het berekenen van ziektelast bij pakketbeoordelingen
Zorginstituut Nederland. Ziektelast in de praktijk: de theorie en praktijk van het berekenen van ziektelast bij pakketbeoordelingen. Accessed: 2025-10-03. 2018. Available from:https://www. zorginstituutnederland.nl/site/binaries/site-content/collections/documents/2018/ 05/07/z...
2025
-
[38]
Generating, presenting, and interpreting cost-effectiveness results in the context of uncertainty: a tutorial for deeper knowledge and better practice
Bilcke J and Beutels P. Generating, presenting, and interpreting cost-effectiveness results in the context of uncertainty: a tutorial for deeper knowledge and better practice. Medical Decision Making 2022; 42:421–35 26 A Numerical example Suppose we are interested in the incre...
2022
Reviewed July 13, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.