pith. machine review for the scientific record. sign in

arxiv: 2604.10706 · v1 · submitted 2026-04-12 · 📊 stat.ME

Recognition: unknown

Multiple Imputation Diagnostics when using Electronic Health Record Data in Observational Studies: A Case Study

Authors on Pith no claims yet

Pith reviewed 2026-05-10 15:49 UTC · model grok-4.3

classification 📊 stat.ME
keywords multiple imputationelectronic health recordsmissing data diagnosticsCARTchronic kidney diseaseneighborhood socioeconomic statuscardiovascular disease
0
0 comments X

The pith

In EHR data for kidney disease, the specific choice of multiple imputation method had little effect on study conclusions.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The authors examine how to handle missing values in electronic health records using multiple imputation in a real-world study linking neighborhood socioeconomic status to cardiovascular hospitalizations among chronic kidney disease patients. They implement a machine learning approach called CART for creating the imputations and apply multivariate graphical tools to check their validity. The work shows that imputed values for blood pressure and blood sugar markers vary depending on the exact imputation strategy, but these variations do not substantially change the main findings or predictive performance. This matters because electronic health records often have incomplete data, and clear validation steps can help researchers trust their results without needing perfect data.

Core claim

In this case study using patient data from two health systems, CART-based multiple imputation validated with graphical diagnostics revealed that the choice between different imputation specifications had minimal impact on inference about the association between lower neighborhood socioeconomic status and higher risk of cardiovascular disease hospitalization, as well as on prediction accuracy.

What carries the argument

CART imputation model validated using multivariate graphical diagnostics for missing data in EHR.

If this is right

  • Imputed distributions of key variables like systolic blood pressure and HbA1c differ based on marginal versus conditional imputation.
  • Inference on the link between neighborhood socioeconomic status and cardiovascular outcomes remains stable across MI variants.
  • Prediction models for the outcome show little sensitivity to the imputation approach.
  • Graphical diagnostics provide a practical way to assess imputation quality in observational EHR studies.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Researchers working with multi-center EHR data might apply the same diagnostics to test if results hold when combining systems.
  • The finding of robustness could encourage broader adoption of MI over simpler methods like mean imputation in health outcomes research.
  • Future work could compare these results against complete-case analysis to quantify the bias avoided.

Load-bearing premise

That the CART model accurately reflects the relationships among variables and that the graphical diagnostics catch any major imputation problems.

What would settle it

If alternative imputation techniques produced meaningfully different estimates of the effect of neighborhood socioeconomic status on cardiovascular hospitalizations.

Figures

Figures reproduced from arXiv: 2604.10706 by Jerome P. Reiter, Lingyu Zhou, Matthew L. Maciejewski, Nrupen A. Bhavsar, Samuel I. Berchuck.

Figure 1
Figure 1. Figure 1: Percent Missing Data Overall and by Tertile of Neighborhood Socioeconomic Status [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Marginal distribution of observed and imputed values of systolic blood pressure (SBP) and hemoglobin A1c (A1C) usin [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Joint distribution of observed and imputed values of systolic blood pressure (SBP) and Hemoglobin A1c (HbA1c) usin [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
read the original abstract

Missing values in electronic health record (EHR) data pose a significant challenge for epidemiologic research. Traditional methods for handling missing data, like mean imputation, may introduce bias. Multiple imputation (MI) offers a principled solution by generating multiple plausible values based on statistical models. However, MI requires careful model specification and validation of imputations, ideally using multivariate graphical tools. We demonstrate the application of such tools to validate MI in a study of chronic kidney disease, assessing cardiovascular outcomes linked to neighborhood socioeconomic status (nSES). This study used data from Duke University Health System (DUHS) and Lincoln Community Health Center (LCHC). Eligible patients had at least one encounter within DUHS or LCHC and had two estimated glomerular filtration rate (eGFR) values <60 mL/min per 1.73 m2 more than 90 days apart between January 1, 2007 and July 1, 2008. Socioeconomic status was assessed using the Agency for Healthcare Research and Quality (AHRQ) index based on census data. The main outcome was a cardiovascular disease-related hospitalization. Participants were mostly older (mean age 73 years), female (64%), and Black (43%). Participants living in lower nSES neighborhoods had higher mean systolic blood pressure (SBP: 140 mmHg) and hemoglobin A1c (HbA1c) levels (7.1%) as compared to participants living in higher nSES neighborhoods. A machine learning based approach, Classification and Regression Trees (CART), was the preferred approach to impute missing data. The distributions of imputed values of SBP and HbA1c were impacted by whether marginal or conditional values of SBP and HbA1c were imputed. The choice of MI had minimal impact on inference and prediction. Future research may want to extend our results and consider how results may differ when using EHR data from multiple health systems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper is a case study applying multiple imputation (MI) via Classification and Regression Trees (CART) to handle missing data in electronic health records from a cohort of chronic kidney disease patients at Duke University Health System and Lincoln Community Health Center. It examines associations between neighborhood socioeconomic status (nSES) and cardiovascular disease-related hospitalizations, using multivariate graphical diagnostics to validate imputations of variables such as systolic blood pressure and HbA1c. The authors report that imputed value distributions differ by marginal versus conditional imputation approaches, yet the choice of MI specification has minimal impact on the resulting inference for nSES-CVD associations and on predictive performance.

Significance. If the central empirical finding holds after addressing validation gaps, the work offers a concrete illustration of MI diagnostics in complex EHR settings, where missingness often involves time-varying labs and multi-center patterns. It provides practical reassurance that downstream epidemiologic conclusions may be robust to reasonable imputation model choices when CART and graphical checks are employed, which could inform applied researchers facing similar data challenges. As a single-site case study without new methodological contributions or quantitative robustness metrics, its broader impact is incremental rather than transformative.

major comments (2)
  1. [Results] Results (imputation impact on inference): The claim that 'the choice of MI had minimal impact on inference and prediction' is load-bearing for the paper's main conclusion but is supported only by qualitative statements without reported effect-size differences, confidence intervals, or formal statistical comparisons (e.g., overlap of coefficient estimates or prediction metrics) across the MI variants examined; this leaves open whether concordance arises from robustness or from shared bias under the CART model.
  2. [Methods] Methods (validation of imputations): The multivariate graphical diagnostics and CART approach are presented as sufficient to confirm valid imputations under an implicit MAR assumption, yet no quantitative diagnostics (such as posterior predictive checks, comparison of observed vs. imputed conditional moments, or sensitivity analyses under MNAR mechanisms) are reported; given the EHR context of encounter-level and center-specific missingness, this weakens the evidential basis for concluding that results are insensitive to model choice rather than jointly biased.
minor comments (2)
  1. [Abstract] Abstract: The eligibility criteria and cohort description would benefit from a supplementary table or flow diagram reporting the number of patients at each inclusion step to improve reproducibility.
  2. [Discussion] The paper could add a brief limitations paragraph explicitly addressing generalizability beyond the two health systems and the absence of MNAR sensitivity checks.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed comments on our case study manuscript. We address each major point below, providing the strongest honest defense of our approach while agreeing to strengthen the presentation where feasible.

read point-by-point responses
  1. Referee: [Results] Results (imputation impact on inference): The claim that 'the choice of MI had minimal impact on inference and prediction' is load-bearing for the paper's main conclusion but is supported only by qualitative statements without reported effect-size differences, confidence intervals, or formal statistical comparisons (e.g., overlap of coefficient estimates or prediction metrics) across the MI variants examined; this leaves open whether concordance arises from robustness or from shared bias under the CART model.

    Authors: We acknowledge that the original manuscript described the similarity in inferences primarily in qualitative terms. To strengthen this, the revised manuscript will include a table reporting the nSES coefficient estimates and 95% confidence intervals for the CVD hospitalization outcome under both marginal and conditional CART imputation specifications. We will also add any available predictive performance metrics (such as AUC or Brier score) for direct comparison. This will allow explicit assessment of effect-size overlap and address the concern about potential shared bias. revision: yes

  2. Referee: [Methods] Methods (validation of imputations): The multivariate graphical diagnostics and CART approach are presented as sufficient to confirm valid imputations under an implicit MAR assumption, yet no quantitative diagnostics (such as posterior predictive checks, comparison of observed vs. imputed conditional moments, or sensitivity analyses under MNAR mechanisms) are reported; given the EHR context of encounter-level and center-specific missingness, this weakens the evidential basis for concluding that results are insensitive to model choice rather than jointly biased.

    Authors: Multivariate graphical diagnostics are a standard and recommended tool for MI validation in the literature, particularly for complex EHR data where CART provides flexible, non-parametric modeling. We will revise the manuscript to include quantitative comparisons of observed versus imputed conditional means and variances for SBP and HbA1c, stratified by nSES and health system. However, full posterior predictive checks and MNAR sensitivity analyses would require substantial additional assumptions and analyses outside the scope of this applied case study; we will note this limitation explicitly and suggest it for future research. revision: partial

Circularity Check

0 steps flagged

No significant circularity in empirical case study

full rationale

This is an applied case study comparing multiple imputation methods (primarily CART-based) on EHR data for nSES-CVD associations, with the central finding that MI choice had minimal impact on inference and prediction. The claim rests on direct empirical comparisons of downstream results across imputation variants and qualitative multivariate graphical diagnostics, without any derivation, equation, or prediction that reduces by construction to fitted inputs or self-referential assumptions. No load-bearing self-citations, uniqueness theorems, or ansatzes are invoked; the work applies existing MI tools to a specific dataset and reports observed concordance, which is falsifiable against the data rather than tautological.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The work rests on standard multiple imputation assumptions and the suitability of CART for the EHR variables; no new entities or heavily fitted parameters are introduced beyond the case-specific application.

axioms (1)
  • domain assumption Missing at random assumption required for valid multiple imputation
    Implicit in the use of MI; stated as standard for the method in the abstract context.

pith-pipeline@v0.9.0 · 5676 in / 1272 out tokens · 62729 ms · 2026-05-10T15:49:12.682730+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 33 canonical work pages

  1. [1]

    Missing Data in Clinical Research: A Tutorial on Multiple Imputation

    Austin PC, White IR, Lee DS, van Buuren S. Missing Data in Clinical Research: A Tutorial on Multiple Imputation. Can J Cardiol. Sep 2021;37(9):1322-1331. doi:10.1016/j.cjca.2020.11.010

  2. [2]

    Theoretical epidemiology : principles of occurrence research in medicine

    Miettinen OS. Theoretical epidemiology : principles of occurrence research in medicine. A Wiley medical publication. John Wiley & Sons; 1985

  3. [3]

    A critical look at methods for handling missing covariates in epidemiologic regression analyses

    Greenland S, Finkle WD. A critical look at methods for handling missing covariates in epidemiologic regression analyses. Am J Epidemiol. Dec 15 1995;142(12):1255-64. doi:10.1093/oxfordjournals.aje.a117592

  4. [4]

    Multiple imputation for nonresponse in surveys

    Toutenburg HR, D.B. Multiple imputation for nonresponse in surveys. Statistical Papers. 1990/12/01 1990;31(1):180-180. doi:10.1007/BF02924688

  5. [5]

    Review: A gentle introduction to imputation of missing values

    Donders R, van der Heijden G, Stijnen T, Moons K. Review: A gentle introduction to imputation of missing values. Journal of clinical epidemiology. 11/01 2006;59:1087-91. doi:10.1016/j.jclinepi.2006.01.014

  6. [6]

    Regression with Missing X's: A Review

    Little RJA. Regression with Missing X's: A Review. Journal of the American Statistical Association. 1992/12/01 1992;87(420):1227-1237. doi:10.1080/01621459.1992.10476282

  7. [7]

    Analysis of Incomplete Multivariate Data

    Schafer J. Analysis of Incomplete Multivariate Data. Chapman and Hall; 1997

  8. [8]

    Unpredictable bias when using the missing indicator method or complete case analysis for missing confounder values: an empirical example

    Knol MJ, Janssen KJ, Donders AR, et al. Unpredictable bias when using the missing indicator method or complete case analysis for missing confounder values: an empirical example. J Clin Epidemiol. Jul 2010;63(7):728-36. doi:10.1016/j.jclinepi.2009.08.028

  9. [9]

    Bias and efficiency of multiple imputation compared with complete-case analysis for missing covariate values

    White IR, Carlin JB. Bias and efficiency of multiple imputation compared with complete-case analysis for missing covariate values. Stat Med. Dec 10 2010;29(28):2920-31. doi:10.1002/sim.3944

  10. [10]

    Model checking in multiple imputation: an overview and case study

    Nguyen CD, Carlin JB, Lee KJ. Model checking in multiple imputation: an overview and case study. Emerg Themes Epidemiol. 2017;14:8. doi:10.1186/s12982-017-0062-6

  11. [11]

    Multiple imputation for missing data: fully conditional specification versus multivariate normal imputation

    Lee KJ, Carlin JB. Multiple imputation for missing data: fully conditional specification versus multivariate normal imputation. Am J Epidemiol. Mar 1 2010;171(5):624-32. doi:10.1093/aje/kwp425

  12. [12]

    A comparison of inclusive and restrictive strategies in modern missing data procedures

    Collins LM, Schafer JL, Kam CM. A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychol Methods. Dec 2001;6(4):330-51

  13. [13]

    Multiple imputation of missing covariates with non-linear effects and interactions: an evaluation of statistical methods

    Seaman SR, Bartlett JW, White IR. Multiple imputation of missing covariates with non-linear effects and interactions: an evaluation of statistical methods. BMC Med Res Methodol. Apr 10 2012;12:46. doi:10.1186/1471-2288-12-46

  14. [14]

    Gaussian-based routines to impute categorical variables in health surveys

    Yucel RM, He Y, Zaslavsky AM. Gaussian-based routines to impute categorical variables in health surveys. Stat Med. Dec 20 2011;30(29):3447-60. doi:10.1002/sim.4355 21

  15. [15]

    Multiple imputation in the presence of non-normal data

    Lee KJ, Carlin JB. Multiple imputation in the presence of non-normal data. Stat Med. Feb 20 2017;36(4):606-617. doi:10.1002/sim.7173

  16. [16]

    Impact of socioeconomic status measures on hospital profiling in New York City

    Blum AB, Egorova NN, Sosunov EA, et al. Impact of socioeconomic status measures on hospital profiling in New York City. Circ Cardiovasc Qual Outcomes. May 2014;7(3):391-7. doi:10.1161/CIRCOUTCOMES.113.000520

  17. [17]

    Development and Validation of a County-Level Social Determinants of Health Risk Assessment Tool for Cardiovascular Disease

    Hong YR, Mainous AG, 3rd. Development and Validation of a County-Level Social Determinants of Health Risk Assessment Tool for Cardiovascular Disease. Ann Fam Med. Jul 2020;18(4):318-325. doi:10.1370/afm.2534

  18. [18]

    Value of Neighborhood Socioeconomic Status in Predicting Risk of Outcomes in Studies That Use Electronic Health Record Data

    Bhavsar NA, Gao A, Phelan M, Pagidipati NJ, Goldstein BA. Value of Neighborhood Socioeconomic Status in Predicting Risk of Outcomes in Studies That Use Electronic Health Record Data. JAMA Netw Open. Sep 7 2018;1(5):e182716. doi:10.1001/jamanetworkopen.2018.2716

  19. [19]

    Flexible Imputation of Missing Data

    van Buuren S. Flexible Imputation of Missing Data. Chapman and Hall; 2018

  20. [20]

    Multiple imputation using chained equations: Issues and guidance for practice

    White IR, Royston P, Wood AM. Multiple imputation using chained equations: Issues and guidance for practice. Stat Med. Feb 20 2011;30(4):377-99. doi:10.1002/sim.4067

  21. [21]

    Inference and missing data

    RUBIN DB. Inference and missing data. Biometrika. 1976;63(3):581-592. doi:10.1093/biomet/63.3.581

  22. [22]

    An introduction to statistical learning : with applications in R

    Gareth James DWTHRT. An introduction to statistical learning : with applications in R. New York : Springer, [2013] ©2013; 2013

  23. [23]

    Multiple Imputation for Missing Data via Sequential Regression Trees

    Burgette LF, Reiter JP. Multiple Imputation for Missing Data via Sequential Regression Trees. American Journal of Epidemiology. 2010;172(9):1070-1076. doi:10.1093/aje/kwq260

  24. [24]

    Stability analysis of fluid flows using Lagrangian Perturbation Theory (LPT): application to the plane Couette flow

    Akande O, Li F, Reiter J. An Empirical Comparison of Multiple Imputation Methods for Categorical Data. The American Statistician. 2017/04/03 2017;71(2):162-170. doi:10.1080/00031305.2016.1277158

  25. [25]

    Multiple imputation with large data sets: a case study of the Children's Mental Health Initiative

    Stuart EA, Azur M, Frangakis C, Leaf P. Multiple imputation with large data sets: a case study of the Children's Mental Health Initiative. Am J Epidemiol. May 1 2009;169(9):1133-9. doi:10.1093/aje/kwp026

  26. [26]

    Diagnostics for multivariate imputations

    Abayomi K, Gelman A, Levy M. Diagnostics for multivariate imputations. Applied Statistics. 01/01 2005;57:273-291

  27. [27]

    Multiple Imputation for Nonresponse in Surveys

    Campion WM, Rubin DB. Multiple Imputation for Nonresponse in Surveys. Journal of Marketing Research. 1989;26:485

  28. [28]

    The Multiple Adaptations of Multiple Imputation

    Reiter JP, Raghunathan TE. The Multiple Adaptations of Multiple Imputation. Journal of the American Statistical Association. 2007/12/01 2007;102(480):1462-1471. doi:10.1198/016214507000000932

  29. [29]

    Missing Data Analysis

    Little RJ. Missing Data Analysis. Annu Rev Clin Psychol. Jul 2024;20(1):149-173. doi:10.1146/annurev-clinpsy-080822-051727

  30. [30]

    Gaps in the usage and reporting of multiple imputation for incomplete data: findings from a scoping review of observational studies addressing causal questions

    Mainzer RM, Moreno-Betancur M, Nguyen CD, Simpson JA, Carlin JB, Lee KJ. Gaps in the usage and reporting of multiple imputation for incomplete data: findings from a scoping review of observational studies addressing causal questions. BMC Med Res Methodol. Sep 4 2024;24(1):193. doi:10.1186/s12874-024-02302-6 22

  31. [31]

    A practical guide to multiple imputation of missing data in nephrology

    Blazek K, van Zwieten A, Saglimbene V, Teixeira-Pinto A. A practical guide to multiple imputation of missing data in nephrology. Kidney Int. Jan 2021;99(1):68-74. doi:10.1016/j.kint.2020.07.035

  32. [32]

    Review and evaluation of imputation methods for multivariate longitudinal data with mixed-type incomplete variables

    Cao Y, Allore H, Vander Wyk B, Gutman R. Review and evaluation of imputation methods for multivariate longitudinal data with mixed-type incomplete variables. Stat Med. Dec 30 2022;41(30):5844-5876. doi:10.1002/sim.9592

  33. [33]

    A systematic review of how missing data are handled and reported in multi-database pharmacoepidemiologic studies

    Hunt NB, Gardarsdottir H, Bazelier MT, Klungel OH, Pajouheshnia R. A systematic review of how missing data are handled and reported in multi-database pharmacoepidemiologic studies. Pharmacoepidemiol Drug Saf. Jul 2021;30(7):819-826. doi:10.1002/pds.5245

  34. [34]

    Multiple Imputation with Massive Data: An Application to the Panel Study of Income Dynamics

    Si Y, Heeringa S, Johnson D, et al. Multiple Imputation with Massive Data: An Application to the Panel Study of Income Dynamics. J Surv Stat Methodol. Feb 2023;11(1):260-283. doi:10.1093/jssam/smab038

  35. [35]

    Missing Data Imputation in the Electronic Health Record Using Deeply Learned Autoencoders

    Beaulieu-Jones BK, Moore JH. Missing Data Imputation in the Electronic Health Record Using Deeply Learned Autoencoders. Pac Symp Biocomput. 2017;22:207-218. doi:10.1142/9789813207813_0021

  36. [36]

    A multi-step approach to managing missing data in time and patient variant electronic health records

    Cesare N, Were LPO. A multi-step approach to managing missing data in time and patient variant electronic health records. BMC Res Notes. Feb 17 2022;15(1):64. doi:10.1186/s13104-022-05911-w

  37. [37]

    Imputation of Missing Data in Electronic Health Records Based on Patients' Similarities

    Jazayeri A, Liang OS, Yang CC. Imputation of Missing Data in Electronic Health Records Based on Patients' Similarities. J Healthc Inform Res. Sep 2020;4(3):295-307. doi:10.1007/s41666-020-00073-5

  38. [38]

    Missing Data in Prediction Research: A Five-Step Approach for Multiple Imputation, Illustrated in the CENTER-TBI Study

    Gravesteijn BY, Sewalt CA, Venema E, Nieboer D, Steyerberg EW, Collaborators C-T. Missing Data in Prediction Research: A Five-Step Approach for Multiple Imputation, Illustrated in the CENTER-TBI Study. J Neurotrauma. Jun 1 2021;38(13):1842-1857. doi:10.1089/neu.2020.7218

  39. [39]

    Graphical and numerical diagnostic tools to assess suitability of multiple imputations and imputation models

    Bondarenko I, Raghunathan T. Graphical and numerical diagnostic tools to assess suitability of multiple imputations and imputation models. Stat Med. Jul 30 2016;35(17):3007-20. doi:10.1002/sim.6926

  40. [40]

    Biometrics , author =

    Rizopoulos D, Verbeke G, Molenberghs G. Multiple-imputation-based residuals and diagnostic plots for joint models of longitudinal and survival outcomes. Biometrics. Mar 2010;66(1):20-9. doi:10.1111/j.1541-0420.2009.01273.x