REVIEW 4 major objections 6 minor 1 cited by
Access to care improves EHR reliability and clinical risk prediction model performance
T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read Low access to care weakens EHR data and diabetes risk scores.
desk verdict Strong descriptive evidence on EHR missingness by access, but the headline sensitivity gap is likely inflated by an outcome-label artifact and the confidence intervals don't survive scrutiny. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central measurement device is the missing EHR diagnosis rate: the share of participants with a self-reported condition who have no record of that condition in the EHR, computed under the stated assumption that the patient is the source of truth. The prediction mechanism is a 2-year Type 2 diabetes incidence task built from a 2-year EHR lookback window of conditions, labs, vitals, procedures, medications, and demographics, trained with a LASSO logistic regression and evaluated at the Youden-J threshold stratified by access group. The load-bearing comparison is sensitivity at that threshold, which drops sharply for low-access patients, and the mitigation experiment adds self-reported conditions as features to test whether missing EHR data causes the gap.
What would settle it
Conduct a study where a sample of All of Us participants receives an independent clinical assessment (physical exam, lab panel, or adjudicated chart review) that does not rely on either the survey or the routine EHR, then compare missing-EHR rates for low-access versus standard-care groups against that gold standard. If the gap shrinks or disappears once self-report error is controlled, the reliability and prediction findings are largely an artifact of differential self-reporting. A complementary check: re-run the diabetes model using chart-reviewed or self-reported diabetes as the outcome instead of EHR codes; if sensitivity still differs by access group, the gap reflects true model failure, not just label noise.
Extended reading notes
Core claim
The paper's central discovery is a two-part empirical regularity in the All of Us cohort. First, EHR reliability — measured as the rate at which a patient's self-reported conditions are missing from their record — is systematically worse for patients with cost-constrained or delayed care: for 29 of 37 conditions the missing-diagnosis rate is significantly higher, and low-access participants average 2.0 missing EHR diagnoses versus 1.7 for standard-care participants. Second, this data degradation carries into clinical prediction: a logistic-regression model with L1 regularization for 2-year Type 2 diabetes incidence achieves comparable AUC across access groups but significantly lower balanced accuracy and sensitivity for low-access patients (sensitivity 57–58% versus 66.4% for standard care), meaning the model misses more true diabetes cases among people who already face access barriers. Including self-reported conditions as extra features lifts sensitivity for the cost-constrained group by 11.2 percentage points and largely erases the performance gap, indicating that missing data — not just algorithm design — drives the disparity.
Load-bearing premise
The load-bearing premise is that a participant's self-report on the Personal and Family Health History survey is the ground truth for whether they truly have a condition, so a condition missing from the EHR counts as an error; if self-reports are inaccurate in ways that differ by access group, the measured reliability gap — and the prediction gap built on it — is overstated.
Editorial extensions
If this is right
- EHR-based clinical risk scores will systematically under-flag Type 2 diabetes risk in patients who delay or cannot afford care, directing fewer preventive interventions to the group with the highest self-reported burden.
- Quality measures and epidemiological estimates that treat the EHR as ground truth will overstate the health of low-access populations; access-adjusted comparisons are needed.
- Collecting patient-reported conditions together with EHR data closes most of the sensitivity gap in diabetes incidence models, a concrete data-collection priority for health systems.
- Adding a simple access-group label as a model feature does not fix the gap; the improvement comes from supplying the missing clinical information, not from the label itself.
- Because the missingness pattern spans most examined conditions, similar access-driven degradation should be expected in other chronic-disease risk models built from EHR data.
Reading between the lines
- If the paper is right, the reliability gap may be overstated to the extent that untreated symptoms make low-access patients more likely to report a condition; a validation study using biomarkers or adjudicated diagnoses would sharpen the estimate.
- The same missing-data mechanism likely affects other EHR-based tasks such as readmission prediction or comorbidity adjustment, a generalization the authors leave implicit.
- Because the outcome label is an EHR record of diabetes, the sensitivity gap could partly reflect under-ascertainment of the outcome rather than model failure; chart-review or self-reported outcomes would separate label noise from model error.
- The paper notes the access coefficient doubles when only the prior year of EHR is used; this suggests shorter lookback windows may amplify the equity gap, a testable extension.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper uses All of Us survey and EHR data (N=134,513) to examine whether healthcare access barriers affect EHR data completeness and the performance of a machine learning model for 2-year Type 2 diabetes incidence. The authors report that low-access groups (cost-constrained or delayed care) have higher rates of self-reported conditions missing from the EHR (for 29 of 37 conditions), and in the diabetes prediction task (N=52,046) show lower balanced accuracy (3.6 percentage points) and sensitivity (9.4 percentage points) than standard-care patients. They propose two mitigations: adding an access indicator and adding self-reported condition features; the latter is reported to increase sensitivity for cost-constrained patients by 11.2 percentage points. The paper concludes that access barriers propagate through the machine learning pipeline and that patient-reported data can help close the gap.
Significance. If the main claims hold, this would be a valuable large-scale empirical demonstration that healthcare access affects both EHR data quality and clinical prediction performance, with direct implications for algorithmic fairness and for the design of data collection in learning health systems. The study leverages a large, diverse national cohort with linked survey and EHR data, and the authors are transparent about several limitations. However, the prediction-performance claim rests on an outcome label that is itself an EHR record, and the mitigation analysis likely includes the self-report of the outcome condition. These issues make the central quantitative claims uncertain and require additional analyses or reframing. The descriptive EHR-reliability comparisons are simpler and more robust, though they depend on the assumption that self-report is ground truth.
major comments (4)
- [Algorithmic predictive performance is lower for patients without access to care; Methods: Implications for Clinical…] The outcome is an indicator of an EHR record of Type 2 diabetes within two years after the index date, and the baseline exclusion removes only participants with an EHR-recorded Type 2 diabetes diagnosis. For low-access patients, diabetes that is already present but undocumented at baseline is not excluded, and incident diabetes may go unrecorded if the patient does not seek care within the follow-up window. The 9.4-percentage-point sensitivity gap could therefore reflect differential outcome ascertainment by access group rather than a true difference in model discrimination. The authors acknowledge in the Limitations that the outcome labels 'may be imprecise especially for individuals with lower access to care,' but they continue to interpret the gap as a performance disparity. I recommend reframing the task as predicting EHR-recorded T2D incidence and adding robustness checks, such as restricting the sample to participants with at least one EHR encounter in the follow-up period, adjusting for visit frequency, or using self-reported T2D as a secondary outcome.
- [Potential Solutions; Methods: Implications for Clinical Prediction Models] The second mitigation adds self-reported condition features to the prediction model. Because the cohort excludes only EHR-recorded T2D at baseline, participants with self-reported T2D at baseline are included. If the self-reported Type 2 diabetes variable is among the added features, the model can trivially identify patients who already know they have diabetes and who are likely to receive an EHR diagnosis within two years. This would inflate the reported 11.2-percentage-point sensitivity increase and does not represent a genuine improvement in incident risk prediction. Please report the results excluding the self-report of the outcome condition, and specify exactly which self-reported conditions were included in the mitigation model.
- [Algorithmic predictive performance is lower for patients without access to care; Material and Methods, final paragraph] The reported 95% confidence intervals for sensitivity are numerically inconsistent with the event counts. For example, the standard-care group has roughly 142 positive events (0.6% of 23,705), but the reported sensitivity of 66.4% with 95% CI 65.9–67.0 implies a standard error of about 0.0028, which would require an effective sample size in the tens of thousands rather than the number of true positives. The intervals appear to have been computed with the full cohort size in the denominator rather than the number of positive events. This error affects the statistical significance claims for the performance gaps and should be corrected by recomputing the intervals with the appropriate denominator.
- [Patients with lower access to care have lower EHR reliability; Comparing Self-Reported Conditions to EHR Conditions] The EHR-reliability analysis treats the participant's self-report as ground truth, stated in the Methods as 'assuming that the patient is the source of truth, and that any missing EHR record of a condition is an error.' This assumption is acknowledged but not examined. If low-access participants are more likely to self-report conditions that are not yet diagnosed (e.g., because they are experiencing untreated symptoms), the observed missingness gap would be inflated relative to true documentation errors. I recommend reframing the result as 'discordance between self-report and EHR' rather than 'EHR reliability,' and adding a sensitivity analysis that validates self-report against an objective source such as medication records or lab values for a subset of conditions.
minor comments (6)
- [Sample; Table 1] The cost-constrained and delayed-care groups are not mutually exclusive (69.1% of the delayed-care group also report affordability concerns), but the paper does not explain how participants in both groups are assigned in the analyses or in Figure 3. Please clarify whether the groups are treated as overlapping or whether participants in both are assigned to a single 'low access' category.
- [Algorithmic predictive performance is lower for patients without access to care] The text states that balanced accuracy drops significantly for the cost-constrained and delayed-care groups but does not report the numerical gap (the abstract reports 3.6 percentage points) or its confidence interval. Please provide these values alongside the sensitivity results.
- [Figure 2 caption] The caption says the figure shows 'all conditions with a statistically significant difference,' while the text reports 29 of 37 conditions were significant. Please clarify whether non-significant conditions are omitted and how the FDR correction was applied in the figure.
- [Table S2 caption] The caption states 'top ten most common self-reported conditions' but the table lists 15 conditions; please correct the caption.
- [Methods: Implications for Clinical Prediction Models] The list of clinical measures includes 'diabetes' alongside lab-based measurements, which is potentially confusing given that the outcome is Type 2 diabetes. Please clarify that this refers to a glucose-derived indicator (as in Table S3) and distinguish it from the outcome definition.
- [Sample section] There is a typo on page 2: 'patients with either cost-constrained care or delayed case' should read 'delayed care.'
Circularity Check
No significant circularity: the empirical comparisons are self-contained against external All of Us data, and the acknowledged self-report and EHR-label assumptions are validity limitations rather than constructional circularity.
full rationale
The paper's two headline results are empirical comparisons on external All of Us data, not formal derivations from fitted parameters. The EHR-reliability analysis defines missingness as a self-reported condition absent from the EHR, explicitly assuming that the patient is the source of truth; this is a measurement assumption, not a circular construction, and access groups are defined from a separate Health Care Access & Utilization survey. The diabetes prediction task uses an EHR-recorded two-year incidence outcome and evaluates held-out performance by access group; the sensitivity gap of 9.4 percentage points is a measured difference, not a quantity forced by the definition of access. The mitigation experiment adds self-reported conditions as features and reports improved sensitivity; this is an empirical model result, and while it may partly reflect detection of prevalent disease among low-access patients, the paper explicitly acknowledges this limitation when it states that outcome labels "may be imprecise especially for individuals with lower access to care." That limitation does not make the result circular. Self-citations (e.g., Chen et al. on AI and health disparities, Zink et al. on race adjustments) are contextual and are not used as the argument for the empirical findings. No equation in the paper reduces a predicted quantity to a fitted input or to an author-supplied uniqueness theorem. Therefore no circularity step meets the required evidentiary bar.
Assumptions & free parameters
free parameters (2)
- Youden's J classification cutoff =
threshold maximizing Youden's J (not numerically reported)
- LASSO regularization strength =
searched 1e-4 to 1e4 with 10 log-spaced intervals, selected by cross-validation
assumptions (4)
- domain assumption Self-reported conditions are ground truth for whether a patient has a condition; a missing EHR record of a self-reported condition is an error.
- domain assumption EHR-recorded Type 2 diabetes within 2 years of index date is a valid outcome for diabetes incidence, even for patients with lower access whose records are less complete.
- domain assumption The healthcare access groups defined by survey responses (cost-constrained, delayed, standard) accurately capture participants' access to care.
- domain assumption The All of Us sample, and the subset with linked EHR and health access survey, is appropriate for estimating these relationships despite being a convenience sample.
Cite this review
Pith. "Pith review of Access to care improves EHR reliability and clinical risk prediction model performance." pith.science (2026). https://pith.science/paper/TVAM5EKE
@misc{pith2026241207712,
author = {Pith},
title = {Pith review of: Access to care improves EHR reliability and clinical risk prediction model performance},
year = {2026},
howpublished = {\url{https://pith.science/paper/TVAM5EKE}},
note = {Machine review of arXiv:2412.07712}
}
read the original abstract
Disparities in access to healthcare have been well-documented in the United States, but their effects on electronic health record (EHR) data reliability and resulting clinical models are poorly understood. Using an All of Us dataset of 134,513 participants, we investigate the effects of access to care on the medical machine learning pipeline, including medical condition rates, data quality, outcome label accuracy, and prediction performance. Our findings reveal that patients with cost constrained or delayed care have worse EHR reliability as measured by patient self-reported conditions for 78% of examined medical conditions. We demonstrate in a prediction task of Type II diabetes incidence that clinical risk predictive performance can be worse for patients without standard care, with balanced accuracy gaps of 3.6 and sensitivity gaps of 9.4 percentage points for those with cost-constrained or delayed care. We evaluate solutions to mitigate these disparities and find that including patient self-reported conditions improved performance for patients with lower access to care, with 11.2 percentage points higher sensitivity, effectively decreasing the performance gap between standard versus delayed or cost-constrained care. These findings provide the first large-scale evidence that healthcare access systematically affects both data reliability and clinical prediction performance. By revealing how access barriers propagate through the medical machine learning pipeline, our work suggests that improving model equity requires addressing both data collection biases and algorithmic limitations. More broadly, this analysis provides an empirical foundation for developing clinical prediction systems that work effectively for all patients, regardless of their access to care.
Figures
Forward citations
Cited by 1 Pith paper
-
Learning-To-Measure: In-Context Active Feature Acquisition
One transformer, pretrained with sequence modeling, greedily acquires features to reduce prediction uncertainty across multiple tasks with missing data, matching or beating task-specific AFA baselines without per-task...
Reference graph
Works this paper leans on
-
[11]
Chen, I. Y. et al. Ethical Machine Learning in Healthcare. Annual Review of Biomedical Data Science 4, null (2021). 12. Murray, J. The ‘All of Us’ Research Program. N Engl J Med 381, 1884 (2019). 13. Pratley, R. E. The Early Treatment of Type 2 Diabetes. The American Journal of Medicine 126, S2–S9 (2013). 14. Razavian, N. et al. Population-Level Predictio...
work page 2021
-
[22]
O’Brien, E. C. et al. Concordance Between Patient-Reported Health Data and Electronic Health Data in the ADAPTABLE Trial. JAMA Cardiol 7, 1235–1243 (2022). 23. Weiner, M. et al. Accuracy, thoroughness, and quality of outpatient primary care documentation in the U.S. Department of Veterans Affairs. BMC Primary Care 25, 262 (2024). 24. Magura, S. & Kang, S....
work page 2022
-
[31]
M., Leff, M., Calonge, N., Garrett, C
Martin, L. M., Leff, M., Calonge, N., Garrett, C. & Nelson, D. E. Validation of self-reported chronic conditions and health services in a managed care population. Am J Prev Med 18, 215–218 (2000). 32. Getzen, E., Ungar, L., Mowery, D., Jiang, X. & Long, Q. Mining for equitable health: Assessing the impact of missing data in electronic health records. Jour...
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.