REVIEW 2 major objections 5 minor 36 references
A simulation and case study to evaluate the extrapolation performance of flexible Bayesian survival models when incorporating real-world data
T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims that adding long-term real-world survival data, even when moderately biased, yields less biased 40-year restricted mean survival estimates than trial data alone, provided the long-term treatment-effect assumption is…
desk verdict A careful simulation study that supports the value of even biased external data for survextrap extrapolation, but the abstract overstates the generality of the bias mechanism tested. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying mechanism is the M-spline hazard model implemented in the survextrap package: a hazard (or excess hazard) written as a weighted sum of positive cubic basis functions with a scale parameter, a smoothness prior, and knots placed at quantiles of trial event times. Extra knots placed beyond trial follow-up let the hazard vary in the long term, so the posterior from trial data alone expresses structural uncertainty rather than forcing a constant hazard. External registry data enter as aggregate counts of survivors over annual intervals, and population mortality enters as a fixed known background hazard in an excess-hazard (relative survival) framework. Treatment effects are modelled either as proportional hazards, as flexible non-proportional hazards with a hierarchical prior on time-varying coefficients, or by fitting arms separately, with optional treatment-effect waning that linearly shrinks the log hazard ratio to zero over a specified interval.
What would settle it
Simulate or reanalyse a dataset where the external registry population has a diverging disease-specific hazard over time, for example a different case mix or a time-varying rather than constant hazard ratio relative to the trial control arm, and check whether including the external data still lowers bias in 40-year restricted mean survival; if it increases bias, the paper's central claim fails in that setting.
Extended reading notes
Core claim
The central discovery is that incorporating long-term external data into a flexible Bayesian evidence-synthesis model removes most of the bias in extrapolated long-term survival, whereas trial data alone, even with extra spline knots, remains biased and highly uncertain. In the main simulation, unbiased external data reduce the bias in 40-year control-arm restricted mean survival from about -0.96 to -0.02 years, and even plus-or-minus 20 percent biased external data outperform models without external data. The model also recovers the 40-year restricted mean survival difference between arms under a constant treatment effect when a proportional-hazards assumption is used, while separate-arm models fail because they contain no long-term information about the active arm. When the true treatment effect wanes after trial end, estimates improve only under waning assumptions close to the truth, showing that external data on the control arm alone cannot resolve treatment-effect uncertainty.
Load-bearing premise
The main load-bearing premise is that the external real-world data are generated by the same all-cause hazard process as the trial control arm, differing only by a constant multiplicative bias factor; if real-world discrepancies are more complex, the improvement from including external data may not carry over.
Editorial extensions
If this is right
- Health technology assessments can anchor extrapolations to 40-year restricted mean survival using registry and life-table data rather than relying on short-term trial data alone.
- Moderately biased external data, with hazard rates up to 20 percent higher or lower than the truth, still reduce bias in long-term control-arm survival compared with trial-only models.
- Without long-term data, extra knots beyond trial follow-up honestly reflect structural uncertainty instead of giving falsely narrow constant-hazard extrapolations.
- Treatment-effect differences are trustworthy only under a correct long-term assumption, such as proportional hazards or an appropriate waning schedule; separate-arm modelling without active-arm long-term data is unreliable.
- Including unbiased external data can compensate for shorter trial follow-up: three-year trial data plus external data gave unbiased 40-year restricted mean survival estimates, while eight-year follow-up was needed without it.
Reading between the lines
- Implicit implication: if the constant-bias simulation mechanism is representative, then health technology assessments should treat the difference between trial-only and trial-plus-external-data extrapolations as a quantitative measure of structural uncertainty, not just a sensitivity check.
- Testable extension: run the same simulation design with external data generated from a shifted age distribution, a different case mix, or a time-varying hazard bias; the current design only varies a constant hazard multiplier, so it cannot tell how robust the benefit is to those discrepancies.
- Possible design: the paper mentions power priors and power likelihoods as ways to downweight suspicious external data; a natural comparison is fixed-weight inclusion versus power-likelihood weighting under the plus-or-minus 20 percent bias scenarios, with coverage of 40-year restricted mean survival as the outcome.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper evaluates the extrapolation performance of the survextrap flexible Bayesian survival model when incorporating external real-world data, using a head-and-neck cancer case study and a simulation study based on a Weibull mixture disease-specific hazard plus Gompertz other-cause mortality. The simulation compares models with and without external data under varying bias levels, knot placements, and treatment-effect assumptions, with estimands being 40-year restricted mean survival time (RMST) and its treatment difference. The main claims are that including long-term external data improves control-arm RMST extrapolation even under a constant multiplicative hazard bias of up to 20%, and that treatment-effect extrapolation requires explicit assumptions about waning. The paper reports bias, mean squared error, model posterior standard deviation, and credible-interval coverage, with Monte Carlo standard errors.
Significance. The paper is a timely and practically relevant evaluation of a widely used tool in health technology assessment. Its strengths include a data-generating mechanism that is structurally different from the M-spline model, true estimands computed from a very large simulated cohort, Monte Carlo errors reported for all performance measures, and public availability of code and data. If the central claims hold, the paper provides useful guidance on knot placement, external-data use, and treatment-effect sensitivity analysis. However, the external-data bias mechanism is limited to a constant multiplicative hazard shift, and one simulation result contradicts the general statement that external data always improves extrapolation accuracy. These issues need to be addressed before the conclusions can be fully accepted.
major comments (2)
- [Simulation Study, Data Generating Mechanism; Supplementary Methods] The external data are generated from the same underlying data-generating mechanism as the trial control arm, with only a constant multiplicative bias exp(v) applied to the all-cause hazard. This means the only discrepancy explored is a time-invariant proportional shift; real-world registries and EHR data can differ in case mix, age distribution, secular trends, and selection, which would induce non-proportional and non-multiplicative discrepancies. The abstract and Discussion state without this caveat that 'even using moderately biased external data gives improvements' and that incorporating external data 'still improved the quality of extrapolations'. The Discussion acknowledges the DGM dependence, but the headline claims are not qualified. Please either add simulations with structurally different external-data mechanisms (e.g., different age distributions or hazard-ratio bias that changes over time) or restrict the conclusions and abstract to the constant-bias setting.
- [Table 3, Scenario 2; Discussion] In Scenario 2 (waning treatment effect), the PH model with unbiased external data has bias 1.10, MSE 2.37, and coverage 0.83, whereas the same model without external data has bias 0.64, MSE 1.14, and coverage 0.89. This is a direct counterexample to the Discussion statement that incorporating external data 'still improved the quality of extrapolations in comparison to relying on trial data alone' when the treatment effect is of interest. The manuscript should discuss this exception and qualify the general claim about external data improving extrapolations.
minor comments (5)
- [Supplementary Table 2] The expressions for scenarios 2 and 3 are written as 'exp[β(t)] = ...' but the right-hand sides are negative or can be negative; presumably these are meant to be β(t) (the log hazard ratio) whose exponential is the hazard ratio. Also, the displayed 'tanh' formula is actually coth(x) = (e^x+e^{-x})/(e^x-e^{-x}); the standard tanh is (e^x-e^{-x})/(e^x+e^{-x}). Please correct these formulas or clarify the notation.
- [Simulation Study; Supplementary Methods] The main text states that survextrap version 0.8.12 was used, while the Supplementary Methods mention a feature available from version 0.8.16; please align the version numbers.
- [Figure 6 caption] The caption says 'The vertical line shows the true value', but the plots display bias; please specify whether the vertical line indicates zero bias or the true estimand value.
- [Abstract] The phrase 'In case studies and simulations' is grammatically awkward; consider 'In a case study and simulations' or 'In case-study and simulation analyses'.
- [Discussion] The sentence 'we demonstrate the robustness of survextrap when modelling external data that was imperfect' should be 'when modelling external data that are imperfect'.
Circularity Check
No significant circularity: simulation truth is generated by an independent data-generating mechanism, and self-citations are not load-bearing.
full rationale
The paper's central evaluation compares survextrap posterior estimates of 40-year RMST and RMSTD to true values obtained from a very large independent simulation (N=10^8) of a data-generating mechanism that is deliberately 'mathematically different from a spline, to not overly favour the spline-based implementation of survextrap' (Supplementary Methods). The external data are generated from the same DGM as the control arm with a constant multiplicative bias exp(v); this is a stated simulation assumption, not a parameter fitted by survextrap and not an input that forces the target RMST. The model is not given the disease-specific hazard or the bias; it estimates the M-spline excess hazard from trial and external aggregate data. Supplying the true Gompertz background mortality rates as an offset is standard relative-survival practice and does not determine the extrapolated disease-specific survival. Self-citations to the authors' prior work [13] set default degrees of freedom and priors, and [6] defines the package, but these are not the evaluation target; the paper includes scenarios where the model fails visibly (e.g., separate-arms models under non-PH truth, PH models under waning truth), showing the benchmark is not rigged. The abstract's broad claim about biased external data is limited by the constant-bias DGM, a generalizability caveat the paper itself states in the Limitations section, not a circularity.
Assumptions & free parameters
free parameters (5)
- Weibull mixture baseline hazard parameters (control arm DGM) =
p=0.41, gamma1=1.53, lambda1=0.52, gamma2=0.82, lambda2=0.13
- Gompertz other-cause mortality parameters =
lambda_gpm=4.3e-5, gamma_gpm=9.4e-2
- Treatment effect scenario parameters =
Scenario 1: HR=0.7; Scenario 2: tanh(-0.38, 0.38, 0.8, 1.2); Scenario 3: EMG(mu=0.8, sigma=0.4, lambda=0.35) scaled by…
- External data bias levels =
v = log(0.8), log(0.9), 0, log(1.1), log(1.2)
- Knot placements and priors =
Extra knots at 5, 10, 25 y (simulation) or 10, 15, 25 y (case study); priors log(eta)~N(0,20), sigma~Gamma(2,1)…
assumptions (5)
- domain assumption Additive hazards decomposition: all-cause hazard = disease-specific hazard + other-cause mortality hazard.
- domain assumption The M-spline hazard is constant after the last knot, and extra knots placed beyond trial follow-up allow the hazard to vary in the long term.
- domain assumption External data are modeled as if they describe the same survival process as the trial control arm, with no adjustment for the simulated bias.
- standard math Standard numerical integration (Gauss-Legendre) and cumulative hazard inversion for simulating survival times; Stan HMC for posterior sampling.
- ad hoc to paper Treatment-effect waning is modeled by linearly decreasing the log hazard ratio between time t_min and t_max.
Cite this review
Pith. "Pith review of A simulation and case study to evaluate the extrapolation performance of flexible Bayesian survival models when incorporating real-world data." pith.science (2026). https://pith.science/paper/6HJ4CRMU
@misc{pith2026250516835,
author = {Pith},
title = {Pith review of: A simulation and case study to evaluate the extrapolation performance of flexible Bayesian survival models when incorporating real-world data},
year = {2026},
howpublished = {\url{https://pith.science/paper/6HJ4CRMU}},
note = {Machine review of arXiv:2505.16835}
}
read the original abstract
Background: Assessment of long-term survival for health technology assessment often necessitates extrapolation beyond the duration of a clinical trial. Without robust methods and external data, extrapolations are unreliable. Flexible Bayesian survival models that incorporate longer-term data sources, including registry data and population mortality, have been proposed as an alternative to using standard parametric models with trial data alone. Methods: The accuracy and uncertainty of extrapolations from the survextrap Bayesian survival model and R package were evaluated. In case studies and simulations, we assessed the accuracy of estimates with and without long-term data, under different assumptions about the long-term hazard rate and how it differs between datasets, and about treatment effects. Results: The survextrap model gives accurate extrapolations of long-term survival when long-term data on the patients of interest are included. Even using moderately biased external data gives improvements over using the short-term trial data alone. Furthermore, the model gives accurate extrapolations of differences in survival between treatment groups, provided that a reasonably accurate assumption is made about how the treatment effect will change over time. If no long-term data are available, then the model can quantify structural uncertainty about potential future changes in hazard rates. Conclusions: This analysis shows that Bayesian modelling can give accurate and reliable survival extrapolations by making the most of all available trial and real-world data. This work improves confidence in the use of a powerful tool for evidence-based healthcare decision-making.
Figures
Reference graph
Works this paper leans on
-
[1]
Bayesian evidence synthesis to extrapolate survival estimates in cost -effectiveness studies
Demiris N, Sharples LD. Bayesian evidence synthesis to extrapolate survival estimates in cost -effectiveness studies. Stat Med . 2006;25(11):1960 -1975. doi:10.1002/sim.2366
-
[2]
Chen EYT, Leontyeva Y, Lin CN, Wang JD, Clements MS, Dickman PW. Comparing Survival Extrapolation within All -Cause and Relative Survival Frameworks by Standard Parametric Models and Flexible Parametric Spline Models Using the Swedish Cancer Registry. Med Decis Making . Published online February 5, 2024:0272989X241227230. doi:10.1177/0272989X241227230
-
[3]
Sweeting MJ, Rutherford MJ, Jackson D, et al. Survival Extrapolation Incorporating General Population Mortality Using Excess Hazard and Cure Models: A Tutorial. Med Decis Making. 2023;43(6):737-748. doi:10.1177/0272989X231184247
-
[4]
van Oostrum I, Ouwens M, Remiro -Azócar A, et al. Comparison of Parametric Survival Extrapolation Approaches Incorporating General Population Mortality for Adequate Health Technology Assessment of New Oncology Drugs. Value in Health. 2021;24(9):1294-1301. doi:10.1016/j.jval.2021.03.008
-
[5]
Kearns B, Stevenson MD, Triantafyllopoulos K, Manca A. Dynamic and Flexible Survival Models for Extrapolation of Relative Survival: A Case Study and Simulation Study. Med Decis Making. 2022;42(7):945-955. doi:10.1177/0272989X221107649
-
[6]
survextrap: a package for flexible and transparent survival extrapolation
Jackson CH. survextrap: a package for flexible and transparent survival extrapolation. BMC Medical Research Methodology . 2023;23(1):282. doi:10.1186/s12874-023-02094-1
-
[7]
Willigers BJA, Ouwens M, Briggs A, et al. The Role of Expert Opinion in Projecting Long-Term Survival Outcomes Beyond the Horizon of a Clinical Trial. Adv Ther . 2023;40(6):2741-2751. doi:10.1007/s12325-023-02503-3
-
[8]
An Evaluation of Survival Curve Extrapolation Techniques Using Long - Term Observational Cancer Data
Vickers A. An Evaluation of Survival Curve Extrapolation Techniques Using Long - Term Observational Cancer Data. Medical Decision Making . 2019;39(8):926. doi:10.1177/0272989X19875950
Show all 36 references
-
[9]
Extrapolation of Survival Curves from Cancer Trials Using External Information
Guyot P, Ades AE, Beasley M, Lueza B, Pignon JP, Welton NJ. Extrapolation of Survival Curves from Cancer Trials Using External Information. Med Decis Making. 2017;37(4):353-366. doi:10.1177/0272989X16670604
2017 doi
-
[10]
Blended Survival Curves: A New Approach to Extrapolation for Time-to-Event Outcomes from Clinical Trials in Health Technology Assessment
Che Z, Green N, Baio G. Blended Survival Curves: A New Approach to Extrapolation for Time-to-Event Outcomes from Clinical Trials in Health Technology Assessment. Med Decis Making . 2023;43(3):299 -310. doi:10.1177/0272989X221134545
2023 doi
-
[11]
Cope S, Ayers D, Zhang J, Batt K, Jansen JP . Integrating expert opinion with clinical trial data to extrapolate long -term survival: a case study of CAR -T therapy for children and young adults with relapsed or refractory acute lymphoblastic 33 leukemia. BMC Med Res Methodol ...
2019 doi
-
[12]
Extrapolating Survival Data Using Historical Trial -Based a Priori Distributions
Soikkeli F, Hashim M, Ouwens M, Postma M, Heeg B. Extrapolating Survival Data Using Historical Trial -Based a Priori Distributions. Value Health . 2019;22(9):1012-1017. doi:10.1016/j.jval.2019.03.017
2019 doi
-
[14]
Biases in Electronic Health Records Data for Generating Real-World Evidence: An Overview
Al-Sahab B, Leviton A, Loddenkemper T, Paneth N, Zhang B. Biases in Electronic Health Records Data for Generating Real-World Evidence: An Overview. J Healthc Inform Res. 2024;8(1):121-139. doi:10.1007/s41666-023-00153-2
2024 doi
-
[15]
Extrapolating Parametric Survival Models in Health Technology Assessment Using Model Averaging: A Simulation Study
Gallacher D, Kimani P, Stallard N. Extrapolating Parametric Survival Models in Health Technology Assessment Using Model Averaging: A Simulation Study. Med Decis Making. 2021;41(4):476-484. doi:10.1177/0272989X21992297
2021 doi
-
[17]
Enhanced secondary analysis of survival data: reconstructing the data from published Kaplan-Meier survival curves
Guyot P, Ades A, Ouwens MJ, Welton NJ. Enhanced secondary analysis of survival data: reconstructing the data from published Kaplan-Meier survival curves. BMC Medical Research Methodology. 2012;12(1):9. doi:10.1186/1471-2288-12-9
2012 doi
-
[19]
Population adjusted -indirect comparisons in health technology assessment: A methodological systematic review
Truong B, Tran LAT, Le TA, Pham TT, Vo TT. Population adjusted -indirect comparisons in health technology assessment: A methodological systematic review. Res Synth Methods. 2023;14(5):660-670. doi:10.1002/jrsm.1653
2023 doi
-
[20]
Target Trial Emulation: A Framework for Causal Inference From Observational Data
Hernán MA, Wang W, Leaf DE. Target Trial Emulation: A Framework for Causal Inference From Observational Data. JAMA. 2022;328(24):2446 -2447. doi:10.1001/jama.2022.21383
2022
-
[22]
Extrapolating Parametric Survival Models in Health Technology Assessment: A Simulation Study
Gallacher D, Kimani P, Stallard N. Extrapolating Parametric Survival Models in Health Technology Assessment: A Simulation Study. Med Decis Making . 2021;41(1):37-50. doi:10.1177/0272989X20973201
2021 doi
-
[23]
Survival extrapolation in the presence of cause specific hazards
Benaglia T, Jackson CH, Sharples LD. Survival extrapolation in the presence of cause specific hazards. Stat Med. 2015;34(5):796-811. doi:10.1002/sim.6375
2015 doi
-
[24]
JAGS: A Program for Analysis of Bayesian Graphical Models using Gibbs Sampling
Plummer M. JAGS: A Program for Analysis of Bayesian Graphical Models using Gibbs Sampling. 3rd International Workshop on Distributed Statistical Computing (DSC 2003); Vienna, Austria. 2003;124. 34
2003
-
[25]
Characterizing structural uncertainty in decision analytic models: a review and application of methods
Bojke L, Claxton K, Sculpher M, Palmer S. Characterizing structural uncertainty in decision analytic models: a review and application of methods. Value Health. 2009;12(5):739-749. doi:10.1111/j.1524-4733.2008.00502.x
2009
-
[26]
Impact of Extrapolation Model Choices on the Structural Uncertainty in Economic Evaluations for Cancer Immunotherapy: A Case Study of Checkmate 067
Shao T, Zhao M, Liang L, Shi L, Tang W. Impact of Extrapolation Model Choices on the Structural Uncertainty in Economic Evaluations for Cancer Immunotherapy: A Case Study of Checkmate 067. Pharmacoecon Open . 2023;7(3):383 -392. doi:10.1007/s41669-023-00391-5
2023 doi
-
[27]
An Efficient Method for Computing Expected Value of Sample Information for Survival Data from an Ongoing Trial
Vervaart M, Strong M, Claxton KP, Welton NJ, Wisløff T, Aas E. An Efficient Method for Computing Expected Value of Sample Information for Survival Data from an Ongoing Trial. Med Decis Making . 2022;42(5):612 -625. doi:10.1177/0272989X211068019
2022 doi
-
[28]
General-Purpose Methods for Simulating Survival Data for Expected Value of Sample Information Calculations
Vervaart M, Aas E, Claxton KP, et al. General-Purpose Methods for Simulating Survival Data for Expected Value of Sample Information Calculations. Med Decis Making. 2023;43(5):595-609. doi:10.1177/0272989X231162069
2023 doi
-
[29]
Calculating the Expected Net Benefit of Sampling for Survival Data: A Tutorial and Case Study
Vervaart M. Calculating the Expected Net Benefit of Sampling for Survival Data: A Tutorial and Case Study. Med Decis Making . 2024;44(7):719 -741. doi:10.1177/0272989X241279459
2024 doi
-
[30]
Perils of Randomized Controlled Trial Survival Extrapolation Assuming Treatment Effect Waning: Why the Distinction Between Marginal and Conditional Estimates Matters
Jennings AC, Rutherford MJ, Latimer NR, Sweeting MJ, Lambert PC. Perils of Randomized Controlled Trial Survival Extrapolation Assuming Treatment Effect Waning: Why the Distinction Between Marginal and Conditional Estimates Matters. Value Health. 2024;27(3):347-355. doi:10.1016...
2024 doi
-
[31]
The Extrapolation Performance of Survival Models for Data With a Cure Fraction: A Simulation Study
Kearns B, Stevenson MD, Triantafyllopoulos K, Manca A. The Extrapolation Performance of Survival Models for Data With a Cure Fraction: A Simulation Study. Value Health. 2021;24(11):1634-1642. doi:10.1016/j.jval.2021.05.009
2021 doi
-
[32]
Mixture and Non -mixture Cure Models for Health Technology Assessment: What You Need to Know
Latimer NR, Rutherford MJ. Mixture and Non -mixture Cure Models for Health Technology Assessment: What You Need to Know. Pharmacoeconomics. 2024;42(10):1073-1090. doi:10.1007/s40273-024-01406-7
2024 doi
-
[33]
The power prior: theory and applications
Ibrahim JG, Chen MH, Gwon Y, Chen F. The power prior: theory and applications. Stat Med. 2015;34(28):3724-3749. doi:10.1002/sim.6728
2015 doi
-
[34]
Combining experimental and observational data through a power likelihood
Lin X, Tarp JM, Evans RJ. Combining experimental and observational data through a power likelihood. Biometrics. 2025;81(1):ujaf008. doi:10.1093/biomtc/ujaf008 35 Supplementary Methods Details of the survextrap model An M-spline for the hazard function The model uses M-splines ...
2025 doi
-
[35]
Simulation -based assessment of a Bayesian survival model with flexible baseline hazard and time - dependent effects
Timmins IR, Torabi F, Jackson CH, Lambert PC, Sweeting MJ. Simulation -based assessment of a Bayesian survival model with flexible baseline hazard and time - dependent effects. Published online 2025. https://arxiv.org/abs/2503.21388
2025 arXiv
-
[36]
Multilevel network meta -regression for general likelihoods: synthesis of individual and aggregate data with applications to survival analysis
Phillippo DM, Dias S, Ades AE, Welton NJ. Multilevel network meta -regression for general likelihoods: synthesis of individual and aggregate data with applications to survival analysis. Published online 2024. https://arxiv.org/abs/2401.12640
2024 arXiv
-
[37]
survextrap: a package for flexible and transparent survival extrapolation
Jackson CH. survextrap: a package for flexible and transparent survival extrapolation. BMC Med Res Methodol . 2023;23(1):282. doi:10.1186/s12874-023- 02094-1
2023 doi
-
[38]
Radiotherapy plus Cetuximab for Squamous- Cell Carcinoma of the Head and Neck
Bonner JA, Harari PM, Giralt J, et al. Radiotherapy plus Cetuximab for Squamous- Cell Carcinoma of the Head and Neck. N Engl J Med . 2006;354(6):567 -578. doi:10.1056/NEJMoa053422
2006 doi
-
[39]
NICE DSU Technical Support Document 21: Flexible methods for survival analysis
Rutherford MJ, Lambert PC, Sweeting MJ, et al. NICE DSU Technical Support Document 21: Flexible methods for survival analysis. Published online 2020
2020
-
[40]
Simulating biologically plausible complex survival data
Crowther MJ, Lambert PC. Simulating biologically plausible complex survival data. Stat Med. 2013;32(23):4118-4134. doi:10.1002/sim.5823
2013 doi
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.