Pith. sign in

REVIEW 3 major objections 5 minor 83 references

On Using Large Language Models to Enhance Clinically-Driven Missing Data Recovery Algorithms in Electronic Health Records

T0 review · 3 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An automated ICD-10 algorithm using a clinician-approved, LLM-expanded roadmap recovers missing electronic health record values as accurately as expert manual chart review, and can be applied to every patient in a study.

desk verdict A useful LLM-assisted roadmap for EHR missing-data recovery, but the 'similar accuracy' claim rests on aggregate counts, not per-data-point agreement — fixable with a confusion matrix. read the letter →

arxiv 2510.03844 v1 pith:23C7IXMT submitted 2025-10-04 cs.LG stat.APstat.ME

Sarah C. Lotspeich , Abbey Collins , Brian J. Wells , Ashish K. Khanna , Joseph Rigdon , Lucy D'Agostino McGowan This is my paper · ORCID
classification cs.LGstat.APstat.ME
keywords electronichealthrecordsmissingdatachartreviewICD-10largelanguagemodelsallostaticloadcomputablephenotypequality
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to replace expensive, slow manual chart review with an automated algorithm that recovers missing values in electronic health records. The algorithm checks whether a patient's ICD-10 diagnosis codes match 'search terms' in a clinical roadmap built for each missing biomarker, and a match flags the missing value as unhealthy. To build the roadmap, a large language model proposed many new terms, which clinicians then pruned to keep only those they judged clinically relevant. In 100 patients whose charts had been reviewed by experts, the final algorithm recovered the same number of missing values (45 of 413) as the human reviewers. Applied automatically to all 1,000 patients, it recovered 531 of 4,009 missing components — recovery that would otherwise have required roughly 100 hours of manual review per 100 patients.

What carries the argument

The central object is the 'roadmap': a table linking each of the ten allostatic load index (ALI) components — a composite 'wear and tear' score from cardiovascular, metabolic, and inflammatory biomarkers — to clinical search terms (e.g., 'infection' for C-reactive protein, 'diabetes' for hemoglobin A1c). The algorithm merges that roadmap against each patient's ICD-10 diagnosis codes and treats any matched code as evidence that a missing biomarker value would be unhealthy. The LLM's role is to generate a broad superset of candidate search terms, which clinicians then review for relevance; the final roadmap is the clinician-approved subset. The comparison benchmark is expert manual chart revie

What would settle it

Among patients who have both a non-missing ALI biomarker measurement and an ICD-10 code that matches the roadmap's search term for that component, compute the proportion whose measurement actually crosses the unhealthy threshold. If that positive predictive value is not well above the base rate in unmatched patients, the algorithm's mechanism — diagnosis code implies unhealthy missing value — is unsupported. A second check: re-run the full comparison on a fresh sample of, say, 200 chart-reviewed patients not used to build the roadmap; the algorithm should recover the same fraction as chart rev

Watch

Extended reading notes

Core claim

The central claim is that a clinically-driven, LLM-enhanced missing data recovery algorithm based on ICD-10 codes can recover missing EHR data with accuracy similar to expert chart reviews, at a fraction of the cost, and can be applied to an entire study sample. On the 100-patient validation subset, the final roadmap — the LLM's context-aware suggestions filtered by clinician adjudication — recovered 45 of 413 missing ALI components, exactly matching the number recovered by manual chart review. In the full 1,000-patient sample, the same algorithm recovered 531 of 4,009 missing components, raising the median number of non-missing ALI components per patient from 6 to 7. The authors further sho

Load-bearing premise

The load-bearing assumption is that a patient who has an ICD-10 diagnosis code matching a roadmap search term (e.g., 'infection' for CRP) would, if tested, have had an unhealthy value of the missing biomarker; this mapping is never validated against patients who have both the code and a measured value.

Editorial extensions

If this is right

  • Missing-data recovery for EHR-based phenotypes no longer has to be limited to small chart-reviewed subsamples; the algorithm can be run on every patient in a cohort.
  • Whole-sample ALI estimates can be computed with less missingness, and the association between ALI and outcomes such as healthcare engagement closely resembles the chart-review-based estimates.
  • The roadmap logic is simple enough to encode directly in SQL, so it can be embedded in EHR data pipelines and reused for other computable phenotypes.
  • Because the algorithm only needs structured ICD-10 codes and no protected health information is sent to the LLM, the approach can be adopted in settings where privacy constraints would block text-based review.
  • The same pattern — LLM proposes, clinicians approve — could be extended to other data-quality dimensions like correctness and plausibility.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The key unvalidated proxy — that a matching ICD-10 code implies a missing lab would have been unhealthy — could be tested directly: among patients who have both the code and a measured value, the code's positive predictive value for an unhealthy measurement should be computed. If it is low, the recovered labels are noisy.
  • The final algorithm's exact match with chart review (45/413) hides the fact that unpruned roadmaps over-recovered CRP values (e.g., urinary tract infections flagged as infections); agreement therefore depends on how aggressively clinicians prune, and a different adjudication could change the recovered set.
  • Because the LLM's baseline prompt (no context) produced far worse coverage, the method's performance is sensitive to prompt design and model choice; a testable extension is to compare several LLMs and prompt strategies on the same validation set.
  • The binary 'unhealthy' recovery ignores magnitude; coupling this algorithm with existing EHR lab values could recover approximate continuous values rather than only thresholds, supporting different downstream analyses.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes an algorithmic approach to recover missing allostatic load index (ALI) components in EHR data by matching ICD-10 codes against clinically derived search-term 'roadmaps.' The authors compare the recovery counts of several roadmap variants — the clinicians' original roadmap, a baseline LLM-generated roadmap, an LLM-generated roadmap seeded with the clinicians' terms (LLM(context)), and a clinician-adjudicated version of that roadmap — against expert chart reviews on 100 patients, and then apply the final roadmap to 1,000 patients. The abstract claims the algorithm achieves 'similar accuracy to chart reviews' based on aggregate recovery counts (45 of 413 missing components recovered by both chart reviews and the LLM(context+clinicians) algorithm). The paper includes reproducible R code and clear descriptions of the prompts and adjudication process.

Significance. If the central accuracy claim were properly supported, this work would be a useful contribution: a scalable, low-cost alternative to manual chart review for recovering missing EHR data, with potential applications to computable phenotyping and learning health systems. The manuscript is transparently written, provides all R code and prompts, and reports recovery counts stratified by component and by patient. However, the headline conclusion rests on aggregate recovery counts rather than per-data-point agreement, which is a critical gap. The current evidence does not establish 'similar accuracy'; it establishes 'similar number of recovered components,' which is a weaker and potentially misleading claim.

major comments (3)
  1. [Results, 'Expert chart reviews versus missing data recovery algorithms' (Figure 1A–E)] The central claim of 'similar accuracy to chart reviews' is not supported by the reported analyses. The comparison is made only on aggregate recovery counts: chart reviews recovered 45 of 413 missing components, and the LLM(context+clinicians) algorithm also recovered 45. No confusion matrix, overlap count, sensitivity, positive predictive value, or agreement statistic is reported. The text itself acknowledges substantial disagreement: chart reviews recovered 24 homocysteine components the algorithm missed, and the algorithm recovered many CRP values that chart reviewers deemed clinically irrelevant (e.g., urinary tract infection). Equal aggregate counts are consistent with completely disjoint recovery sets. Please provide a per-data-point cross-tabulation of algorithm versus chart-review classifications for all 413 missing data points and report sensitivity/PPV (or kappa), stratified by
  2. [Methods, 'Enhancements with large language models' (Prompt 2) and Results, 'Clinicians’ adjudication'] The comparison is not independent of the baseline. The LLM(context) prompt explicitly instructs the model to include the examples from the clinicians' original roadmap, so the LLM's term list is seeded with the very terms used in the baseline algorithm. More importantly, the final LLM(context+clinicians) roadmap was selected by clinician adjudication after examining the LLM(context) matches in the same 100 patients whose chart reviews are used as the gold standard, and the reported 45/45 recovery count is therefore an in-sample fit. This selection process likely inflates agreement. To support the accuracy claim, the roadmap adjudication should be performed (or at least evaluated) on a held-out set of patients, or the authors should report cross-validated estimates of agreement.
  3. [Methods, 'Missing data recovery algorithms' and Results, 'Expert chart reviews versus missing data recovery algorithms'] The algorithm's core proxy — that a missing biomarker is 'unhealthy' whenever a matching ICD-10 diagnosis appears in the chart — is never validated against patients who have both a measured biomarker value and the corresponding diagnosis code(s). The CRP disagreement (e.g., urinary tract infection flagged by the algorithm but not by chart reviewers) demonstrates that the mapping is context-sensitive and can produce false positives. Without a validation study using patients with both measured values and diagnoses, the recovered 'unhealthy' labels, and any ALI estimates built on them, are at risk of systematic bias. Please validate the proxy in a subset of patients with non-missing measured values, reporting sensitivity and specificity of the diagnosis-code rule.
minor comments (5)
  1. [Abstract] Typo: 'plausability' should be 'plausibility'.
  2. [Reference [11]] Typo: 'rlectronic' should be 'electronic' in the title of the cited article.
  3. [Results, 'Expert chart reviews versus missing data recovery algorithms'] In the sentence 'LLM (baseline) only recovered three missing components (7%)', the percentage appears to be a numerical error. Three out of 413 missing components is approximately 0.7%, not 7%.
  4. [Table 2] The footnote for the LLM(context+clinicians) row ('*Clinicians were only asked to review ICD-10 codes from LLM(context) that matched patients in our sample; it was not possible for LLM(context+clinicians) to match more than 275 ICD-10 codes.') is useful but the asterisk placement in the table could be clearer; consider placing it directly on the '243' value or the row label.
  5. [Methods, 'Enhancements with large language models'] The description of the number of LLM generations (20) is present, but the choice of 20 is not justified. A short rationale (e.g., stability of the superset) would strengthen the reproducibility of the method.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the roadmap is an input and the chart reviews are an external benchmark, though the central 'similar accuracy' claim lacks per-point validation.

full rationale

The derivation chain is: (1) start with a clinician-defined roadmap from prior work (ref [26]); (2) optionally expand it with LLM-generated search terms, either without context or with the original roadmap as context; (3) have clinicians adjudicate the LLM(context) additions; (4) merge the resulting term list against ICD-10 diagnosis descriptions and count matches among missing ALI components. The comparison against expert chart reviews is a genuine external benchmark: those reviews were performed by human reviewers on the same 100 patients and were not generated by the algorithm. The LLM(context) prompt does include the clinicians' original roadmap, but the paper also evaluates LLM(baseline), which does not, and that arm performs much worse, so the context manipulation is an ablation, not a circularity. The final LLM(context+clinicians) roadmap was selected after observing its recovery counts on the 100-patient sample, which is an in-sample/selection-bias concern about the evaluation, but no numeric parameter was fit and the paper does not claim to predict those counts from a fitted value; the recovered count is an arithmetic consequence of the roadmap and ICD codes. The more serious weakness is non-circular: 'similar accuracy to chart reviews' is based only on aggregate recovery counts (45 vs 45) without a per-data-point confusion matrix or overlap measure, so the claim is unmeasured rather than definitionally forced. Self-citations to [26] supply the roadmap and the prior chart-review data, which are inputs/comparators, not conclusions derived from the present paper. Accordingly, no step reduces by construction to its own input.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The central claim depends on a small set of domain assumptions about the reliability of ICD-10 codes as proxies for unmeasured biomarkers and the representativeness of the chart-review subsample. There are no fitted numerical parameters beyond the hand-chosen LLM repetition count; the search-term roadmaps themselves are hand/LLM-selected, which is captured as a selection-bias red flag rather than a fitted parameter.

free parameters (1)
  • Number of LLM generations per roadmap = 20
    The authors take the superset of 20 repeated LLM calls; no sensitivity analysis is given for how the number of repeats affects the final term list or recovery counts.
assumptions (5)
  • domain assumption An ICD-10 code that matches a roadmap search term implies the corresponding ALI component, if measured, would be unhealthy.
    Core of the recovery algorithm; never validated against patients with both the code and the lab value; the CRP disagreements (UTIs excluded by chart reviewers) show the mapping is context-dependent.
  • domain assumption The 100 chart-reviewed patients are adequate to evaluate algorithm performance and the roadmap choice generalizes to the remaining 900.
    Chart-review subsample was intentionally enriched for healthcare engagement (50/50), so it is not a random sample; no external validation cohort is used.
  • domain assumption Manual chart review is an appropriate gold standard for missing-data recovery.
    Authors call chart-review data 'alloyed' gold standards—'presumed to be more accurate but still imperfect'—yet use the chart-review count as the benchmark for 'similar accuracy'.
  • ad hoc to paper A superset of 20 stochastic LLM outputs constitutes a stable term list.
    No repetition/seed analysis; LLM outputs are stochastic and the final list varies by run; the paper does not assess coverage stability.
  • domain assumption Requiring all search terms to appear literally in the ICD-10 description is a valid matching rule.
    String matching may miss synonyms or context; no evaluation of matching precision.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On Using Large Language Models to Enhance Clinically-Driven Missing Data Recovery Algorithms in Electronic Health Records." pith.science (2026). https://pith.science/paper/23C7IXMT

@misc{pith2026251003844,
  author       = {Pith},
  title        = {Pith review of: On Using Large Language Models to Enhance Clinically-Driven Missing Data Recovery Algorithms in Electronic Health Records},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/23C7IXMT}},
  note         = {Machine review of arXiv:2510.03844}
}
abstract

Objective: Electronic health records (EHR) data are prone to missingness and errors. Previously, we devised an "enriched" chart review protocol where a "roadmap" of auxiliary diagnoses (anchors) was used to recover missing values in EHR data (e.g., a diagnosis of impaired glycemic control might imply that a missing hemoglobin A1c value would be considered unhealthy). Still, chart reviews are expensive and time-intensive, which limits the number of patients whose data can be reviewed. Now, we investigate the accuracy and scalability of a roadmap-driven algorithm, based on ICD-10 codes (International Classification of Diseases, 10th revision), to mimic expert chart reviews and recover missing values. Materials and Methods: In addition to the clinicians' original roadmap from our previous work, we consider new versions that were iteratively refined using large language models (LLM) in conjunction with clinical expertise to expand the list of auxiliary diagnoses. Using chart reviews for 100 patients from the EHR at an extensive learning health system, we examine algorithm performance with different roadmaps. Using the larger study of $1000$ patients, we applied the final algorithm, which used a roadmap with clinician-approved additions from the LLM. Results: The algorithm recovered as much, if not more, missing data as the expert chart reviewers, depending on the roadmap. Discussion: Clinically-driven algorithms (enhanced by LLM) can recover missing EHR data with similar accuracy to chart reviews and can feasibly be applied to large samples. Extending them to monitor other dimensions of data quality (e.g., plausability) is a promising future direction.

Figures

Figures reproduced from arXiv: 2510.03844 by the authors.

Figure 1
Figure 1. Flow charts of the 1000 data points included in the expert chart reviews (10 components per patient across n = 100 patients). We compare the extracted electronic health records (EHR) data to the A) expert chart reviews and the missing data recovery algorithms based on the following roadmaps: B) clinicians’ original, C) LLM (baseline), D) LLM (context), and E) LLM (context + clinicians). Shaded boxes denote data poin… view at source ↗
Figure 2
Figure 2. Counts of missing allostatic load index components [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Counts of missing allostatic load index components [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Many patients’ expert chart reviewed and algorithmically augmented allostatic load indices (ALIs) differed [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Coefficient estimates (95% confidence intervals) using the extracted electronic health records (EHR) data [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

83 extracted references

  1. [1]

    Implementing the learning health system: from concept to action

    Greene SM, Reid RJ, Larson EB. Implementing the learning health system: from concept to action. Ann Intern Med. 2012;157(3):207-10. 12 On Using LLM to Enhance Clinically-Driven Missing Data Recovery Algorithms in EHRA PREPRINT Figure 5: Coefficient estimates (95% confidence intervals) using the extracted electronic health records (EHR) data (naive analysi...

    2012

  2. [2]

    Clarifying the concept of a learning health system for healthcare delivery organizations: Implications from a qualitative analysis of the scientific literature

    Easterling D, Perry AC, Woodside R, Patel T, Gesell SB. Clarifying the concept of a learning health system for healthcare delivery organizations: Implications from a qualitative analysis of the scientific literature. LHS. 2022;6(2):e10287

    2022

  3. [3]

    Toward a science of learning systems: A research agenda for the high-functioning learning health system

    Friedman C, Rubin J, Brown J, Buntin M, Corn M, Etheredge L, et al. Toward a science of learning systems: A research agenda for the high-functioning learning health system. J Am Med Inform Assoc. 2015;22(1):43-50

    2015

  4. [4]

    A framework for analysing learning health systems: Are we removing the most impactful barriers? LHS

    McLachlan S, Dube K, Johnson O, Buchanan D, Potts HWW, Gallagher T, et al. A framework for analysing learning health systems: Are we removing the most impactful barriers? LHS. 2019;3(4):e10189

    2019

  5. [5]

    Applying implementation science to advance electronic health record-driven learning health systems: Case studies, challenges, and recommendations

    Trinkley KE, Maw AM, Torres CH, Huebschmann AG, Glasgow RE. Applying implementation science to advance electronic health record-driven learning health systems: Case studies, challenges, and recommendations. J Med Internet Res. 2024;26:e55472

    2024

  6. [6]

    Desiderata for computable representa- tions of electronic health records-driven phenotype algorithms

    Mo H, Thompson WK, Rasmussen LV , Pacheco JA, Jiang G, Kiefer R, et al. Desiderata for computable representa- tions of electronic health records-driven phenotype algorithms. J Am Med Inform Assoc. 1988;22(6):1220–1230

    1988

  7. [7]

    Caveats for the use of operational electronic health record data in comparative effectiveness research

    Hersh WR, Weiner MG, Embi PJ, Logan JR, Payne PRO, Bernstam EV , et al. Caveats for the use of operational electronic health record data in comparative effectiveness research. Med Care. 2013;51(8):S30-7

    2013

  8. [9]

    The evolving use of electronic health records (EHR) for research

    Kim E, Rubinstein SM, Nead KT, Wojcieszynski AP, Gabriel PE, Warner JL. The evolving use of electronic health records (EHR) for research. Semin Radiat Oncol. 2019;29(4):354-61

    2019

Show all 83 references
  1. [10]

    Use of EHRs data for clinical research: Historical progress and current applications

    Nordo AH, Levaux HP, Becnel LB, Galvez J, Rao P, Stem K, et al. Use of EHRs data for clinical research: Historical progress and current applications. LHS. 2019;3(1):e10076

    2019

  2. [11]

    Challenges in and opportunities for rlectronic health record-based data analysis and interpretation

    Kim MK, Rouphael C, McMichael J, Welch N, Dasarathy S. Challenges in and opportunities for rlectronic health record-based data analysis and interpretation. Gut Liver. 2024;18(2):201–208

    2024

  3. [12]

    Assessing missing data assumptions in EHR-based studies: A complex and underappreciated task

    Haneuse S, Arterburn D, Daniels MJ. Assessing missing data assumptions in EHR-based studies: A complex and underappreciated task. JAMA Netw Open. 2021;4(2):e210184-4

    2021

  4. [13]

    Mining for equitable health: Assessing the impact of missing data in electronic health records

    Getzen E, Ungar L, Mowery D, Jiang X, Long Q. Mining for equitable health: Assessing the impact of missing data in electronic health records. J Biomed Inform. 2023;139:104269

    2023

  5. [14]

    The concept of allostasis in biology and biomedicine

    McEwen BS, Wingfield JC. The concept of allostasis in biology and biomedicine. Horm Behav. 2003;43(1):2-15

    2003

  6. [15]

    Allostasis: A new paradigm to explain arousal pathology

    Sterling P, Eyer J. Allostasis: A new paradigm to explain arousal pathology. In: Fisher S, Reason J, editors. Handbook of Life Stress, Cognition and Health. New York: John Wiley & Sons; 1988. p. 629-49. 13 On Using LLM to Enhance Clinically-Driven Missing Data Recovery Algorit...

    1988

  7. [16]

    Stress and the individual

    McEwen BS, Stellar E. Stress and the individual. Mechanisms leading to disease. Arch Intern Med. 1993;153(18):2093-101

    1993

  8. [17]

    What is in a name? Integrating homeostasis, allostasis and stress

    McEwen BS, Wingfield JC. What is in a name? Integrating homeostasis, allostasis and stress. Horm Behav. 2010;57(2):105-11

    2010

  9. [18]

    A systematic review of allostatic load, health, and health disparities

    Beckie TM. A systematic review of allostatic load, health, and health disparities. Biol Res Nurs. 2012;14(4):311- 46

    2012

  10. [19]

    Allostatic load burden and racial disparities in mortality

    Duru OK, Harawa NT, Kermah D, Norris KC. Allostatic load burden and racial disparities in mortality. J Natl Med Assoc. 2012;104(1):89-95

    2012

  11. [20]

    Combinations of biomarkers predictive of later life mortality

    Gruenewald TL, Seeman TE, Ryff CD, Karlamangla AS, Singer BH. Combinations of biomarkers predictive of later life mortality. Proc Natl Acad Sci. 2006;103(38):14158-63

    2006

  12. [21]

    Allostatic overload in patients with essential hypertension

    Guidi J, Lucente M, Piolanti A, Roncuzzi R, Rafanelli C, Sonino N. Allostatic overload in patients with essential hypertension. Psychoneuroendocrinology. 2020;113:104545

    2020

  13. [22]

    Allostatic Load and Its Impact on Health: A Systematic Review

    Guidi J, Lucente M, Sonino N, Fava G. Allostatic Load and Its Impact on Health: A Systematic Review. Psychother Psychosom. 2020;90(1):11-27

    2020

  14. [23]

    Stress, adaptation, and disease

    McEwen BS. Stress, adaptation, and disease. Allostasis and allostatic load. Ann N Y Acad Sci. 1998;840:33-44

    1998

  15. [24]

    Allostatic load and mortality: A systematic review and meta-analysis

    Parker HW, Abreu AM, Sullivan MC, Vadiveloo MK. Allostatic load and mortality: A systematic review and meta-analysis. Am J Prev Med. 2022;63(1):131-40

    2022

  16. [25]

    Allostatic load as a marker of cumulative biological risk: MacArthur studies of successful aging

    Seeman TE, McEwen BS, Rowe JW, Singer BH. Allostatic load as a marker of cumulative biological risk: MacArthur studies of successful aging. Proc Natl Acad Sci. 2001;98(8):4770-5

    2001

  17. [26]

    Overcoming data challenges through enriched validation and targeted sampling to measure whole-person health in electronic health records

    Lotspeich SC, Kedar S, Tahir R, Keleghan AD, Miranda A, Duda SN, et al. Overcoming data challenges through enriched validation and targeted sampling to measure whole-person health in electronic health records. J Biomed Inform. 2025;170:104904

    2025

  18. [27]

    Quality of data collection in a large HIV observational clinic database in sub-Saharan Africa: implications for clinical research and audit of care

    Kiragga AN, Castelnuovo B, Schaefer P, Muwonge T, Easterbrook PJ. Quality of data collection in a large HIV observational clinic database in sub-Saharan Africa: implications for clinical research and audit of care. J Int AIDS Soc. 2011;14(3):3

    2011

  19. [28]

    Improving public health information: A data quality intervention in KwaZulu-Natal, South Africa

    Mphatswe W, Mate KS, Bennett B, Ngidi H, Reddy J, Barker PM, et al. Improving public health information: A data quality intervention in KwaZulu-Natal, South Africa. Bull World Health Organ. 2012;90(3):176-82

    2012

  20. [29]

    Design and implementation of a health management information system in Malawi: Issues, innovations and results

    Chaulagai CN, Moyo CM, Koot J, Moyo HBM, Sambakunsi TC, Khunga FM, et al. Design and implementation of a health management information system in Malawi: Issues, innovations and results. Health Policy Plan. 2005;20:375-84

    2005

  21. [30]

    Analysing the hindrance to the use of information and technology for improving efficiency of health care delivery system in Tanzania

    Kimaro HC, Twaakyondo HM. Analysing the hindrance to the use of information and technology for improving efficiency of health care delivery system in Tanzania. Tanzan Health Res Bull. 2005;7:189-97

    2005

  22. [31]

    Multiwave validation sampling for error-prone electronic health records

    Shepherd BE, Han K, Chen T, Bian A, Pugh S, Duda SN, et al. Multiwave validation sampling for error-prone electronic health records. Biometrics. 2023;79(3):2649-63

    2023

  23. [32]

    Measuring the quality of observational study data in an international HIV research network

    Duda SN, Shepherd BE, Gadd CS, Masys DR, McGowan CC. Measuring the quality of observational study data in an international HIV research network. PloS One. 2012;7(4):e33908

    2012

  24. [33]

    Self-audits as alternatives to travel- audits for improving data quality in the Caribbean, Central and South America network for HIV epidemiology

    Lotspeich SC, Giganti MJ, Maia M, Vieira R, Machado DM, Succi RC, et al. Self-audits as alternatives to travel- audits for improving data quality in the Caribbean, Central and South America network for HIV epidemiology. Clin Transl Sci. 2020;4(2):125–132

    2020

  25. [34]

    Lessons learned from over a decade of data audits in international observational HIV cohorts in Latin America and East Africa

    Lotspeich SC, Shepherd BE, Kariuki MA, Kaloustian KW, McGowan CC, Musick B, et al. Lessons learned from over a decade of data audits in international observational HIV cohorts in Latin America and East Africa. Clin Transl Sci. 2023;7(1):e245

    2023

  26. [35]

    Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support

    Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–381

    2009

  27. [36]

    Using Anchors to Estimate Clinical State without Labeled Data

    Halpern Y , Choi Y , Horng S, Sontag D. Using Anchors to Estimate Clinical State without Labeled Data. AMIA Annual Symposium Proceedings. 2014:606–615

    2014

  28. [37]

    International Statistical Classification of Diseases and Related Health Problems: Tenth Revision

    World Health Organization. International Statistical Classification of Diseases and Related Health Problems: Tenth Revision. 2nd ed. Geneva, Switzerland: World Health Organization; 2004

    2004

  29. [38]

    Using electronic health records to generate phenotypes for research

    Pendergrass SA, Crawford DC. Using electronic health records to generate phenotypes for research. Curr Protoc Hum Genet. 2019;100(1):e80. 14 On Using LLM to Enhance Clinically-Driven Missing Data Recovery Algorithms in EHRA PREPRINT

    2019

  30. [39]

    High-throughput phenotyping with electronic medical record data using a common semi-supervised approach (PheCAP)

    Zhang Y , Cai T, Yu S, Cho K, Hong C, Sun J, et al. High-throughput phenotyping with electronic medical record data using a common semi-supervised approach (PheCAP). Nature Protocols. 2019;14(12):3426-44

    2019

  31. [40]

    Machine learning approaches for electronic health records phenotyping: A methodical review

    Yang S, Varghese P, Stephenson E, Tu K, Gronsbell J. Machine learning approaches for electronic health records phenotyping: A methodical review. J Am Med Inform Assoc. 2023;30(2):367–381

    2023

  32. [41]

    The validation of electronic health records in accurately identifying patients eligible for colorectal cancer screening in safety net clinics

    Petrik AF, Green BB, V ollmer WM, Le T, Bachman B, Keast E, et al. The validation of electronic health records in accurately identifying patients eligible for colorectal cancer screening in safety net clinics. Fam Pract. 2016;33(6):639-43

    2016

  33. [42]

    Toward a computable phenotype for determining eligibility of lung cancer screening using electronic health records

    Yang S, Huang Y , Lou X, Lyu T, Wei R, Mehta HJ, et al. Toward a computable phenotype for determining eligibility of lung cancer screening using electronic health records. JCO Clin Cancer Inform. 2025;(9):e2400139

    2025

  34. [43]

    Leder Macek AJ, Kirschenbaum JD, Ricklan SJ, Schreiber-Stainthorp W, Omene BC, Conderino S. Validation of rule-based algorithms to determine colorectal, breast, and cervical cancer screening status using electronic health record data from an urban healthcare system in New York...

    2021

  35. [44]

    Leveraging natural language process- ing to identify eligible lung cancer screening patients with the electronic health record

    Liu S, McCoy AB, Aldrich MC, Sandler KL, Reese TJ, Steitz B, et al. Leveraging natural language process- ing to identify eligible lung cancer screening patients with the electronic health record. Int J Med Inform. 2023;177:105136

    2023

  36. [45]

    Develop and validate a computable phenotype for the identification of Alzheimer’s disease patients using electronic health record data

    He X, Wei R, Huang Y , Chen Z, Lyu T, Bost S, et al. Develop and validate a computable phenotype for the identification of Alzheimer’s disease patients using electronic health record data. Alzheimers Dement (Amst). 2024;16(3):e12613

    2024

  37. [46]

    Development and validation of eRADAR: A tool using EHR data to detect unrecognized dementia

    Barnes DE, Zhou J, Walker RL, Larson EB, Lee SJ, Boscardin WJ, et al. Development and validation of eRADAR: A tool using EHR data to detect unrecognized dementia. J Am Geriatr Soc. 2020;68(1):103-11

    2020

  38. [47]

    Evaluation of an algorithm for identifying ocular conditions in electronic health record data

    Stein JD, Rahman M, Andrews C, Ehrlich JR, Kamat S, Shah M, et al. Evaluation of an algorithm for identifying ocular conditions in electronic health record data. JAMA Ophthalmol. 2019;137(5):491-7

    2019

  39. [48]

    Identifying lupus patients in electronic health records: Development and validation of machine learning algorithms and application of rule-based algorithms

    Jorge A, Castro VM, Barnado A, Gainer V , Hong C, Cai T, et al. Identifying lupus patients in electronic health records: Development and validation of machine learning algorithms and application of rule-based algorithms. Seminars in Arthritis and Rheumatism. 2019;49(1):84-90

    2019

  40. [49]

    Performance of a machine learning algorithm using electronic health record data to identify and estimate survival in a longitudinal cohort of patients with lung cancer

    Yuan Q, Cai T, Hong C, Du M, Johnson BE, Lanuti M, et al. Performance of a machine learning algorithm using electronic health record data to identify and estimate survival in a longitudinal cohort of patients with lung cancer. JAMA Netw Open. 2021;4(7):e2114723-3

    2021

  41. [50]

    Using a data quality framework to clean data extracted from the electronic health record: A case study

    Dziadkowiec O, Callahan T, Ozkaynak M, Reeder B, Welton J. Using a data quality framework to clean data extracted from the electronic health record: A case study. EGEMS. 2016;4(1):1201

    2016

  42. [51]

    A clustering approach for detecting implausible observation values in electronic health records data

    Estiri H, Klann JG, Murphy SN. A clustering approach for detecting implausible observation values in electronic health records data. BMC Med Inform Decis Mak. 2019;19(1):142

    2019

  43. [52]

    An automated data cleaning method for Electronic Health Records by incorporating clinical knowledge

    Shi X, Prins C, Van Pottelbergh G, Mamouris P, Vaes B, De Moor B. An automated data cleaning method for Electronic Health Records by incorporating clinical knowledge. BMC Med Inform Decis Mak. 2021;21(1):267

    2021

  44. [53]

    Labeling of medicines and patient safety: Evaluating methods of reducing drug name confusion

    Filik R, Purdy K, Gale A, Gerrett D. Labeling of medicines and patient safety: Evaluating methods of reducing drug name confusion. Hum Factors. 2006;48(1):39-47

    2006

  45. [54]

    Automated misspelling detection and correction in clinical free-text records

    Lai KH, Topaz M, Goss FR, Zhou L. Automated misspelling detection and correction in clinical free-text records. J Biomed Inform. 2015;55:188-95

    2015

  46. [55]

    MLM-based typographical error correction of unstructured medical texts for named entity recognition

    Lee EB, Heo GE, Choi CM, Song M. MLM-based typographical error correction of unstructured medical texts for named entity recognition. BMC Bioinformatics. 2022;23(1):486

    2022

  47. [56]

    Automated identifica- tion of implausible values in growth data from pediatric electronic health records

    Daymont C, Ross ME, Russell Localio A, Fiks AG, Wasserman RC, Grundmeier RW. Automated identifica- tion of implausible values in growth data from pediatric electronic health records. J Am Med Inform Assoc. 2017;24(6):1080–1087

    2017

  48. [57]

    Cleaning of anthropometric data from PCORnet electronic health records using automated algorithms

    Lin PD, Rifas-Shiman SL, Aris IM, Daley MF, Janicke DM, Heerman WJ, et al. Cleaning of anthropometric data from PCORnet electronic health records using automated algorithms. JAMIA Open. 2022;5(4):ooac089

    2022

  49. [58]

    Identifying erroneous height and weight values from adult electronic health records in the All of Us research program

    Guide A, Sulieman L, Garbett S, Cronin RM, Spotnitz M, Natarajan K, et al. Identifying erroneous height and weight values from adult electronic health records in the All of Us research program. J Biomed Inform. 2024;155:104660

    2024

  50. [59]

    Strategies for handling missing data in electronic health record derived data

    Wells BJ, Chagin KM, Nowacki AS, Kattan MW. Strategies for handling missing data in electronic health record derived data. EGEMS. 2013;1(3):1035

    2013

  51. [60]

    DensityTransfer: A data driven approach for imputing electronic health records

    Wang F, Zhou J, Hu J. DensityTransfer: A data driven approach for imputing electronic health records. In: 2014 22nd International Conference on Pattern Recognition; 2014. p. 2763-8. 15 On Using LLM to Enhance Clinically-Driven Missing Data Recovery Algorithms in EHRA PREPRINT

    2014

  52. [61]

    Opportunities and challenges in developing risk prediction models with electronic health records data: A systematic review

    Goldstein BA, Navar AM, Pencina MJ, Ioannidis JPA. Opportunities and challenges in developing risk prediction models with electronic health records data: A systematic review. J Am Med Inform Assoc. 2016;24(1):198-208

    2016

  53. [62]

    Characterizing and managing missing structured data in electronic health records: Data analysis

    Beaulieu-Jones BK, Lavage DR, Snyder JW, Moore JH, Pendergrass SA, Bauer CR. Characterizing and managing missing structured data in electronic health records: Data analysis. JMIR Med Inform. 2018 Feb;6(1):e11

    2018

  54. [63]

    A deep learning–based unsupervised method to impute missing values in patient records for improved management of cardiovascular patients

    Xu D, Sheng JQ, Hu PJH, Huang TS, Hsu CC. A deep learning–based unsupervised method to impute missing values in patient records for improved management of cardiovascular patients. IEEE J Biomed Health Inform. 2021;25(6):2260-72

    2021

  55. [64]

    A novel missing data imputation approach based on clinical conditional generative adversarial networks applied to EHR datasets

    Bernardini M, Doinychko A, Romeo L, Frontoni E, Amini MR. A novel missing data imputation approach based on clinical conditional generative adversarial networks applied to EHR datasets. Comput Biol Med. 2023;163:107188

    2023

  56. [65]

    Imputation of missing data in electronic health records based on patients’ similarities

    Jazayeri A, Liang OS, Yang CC. Imputation of missing data in electronic health records based on patients’ similarities. J Healthc Inform Res. 2020;4(3):295-307

    2020

  57. [66]

    Don’t do imputation: Dealing with informative missing values in EHR data analysis

    Li J, Wang M, Steinbach MS, Kumar V , Simon GJ. Don’t do imputation: Dealing with informative missing values in EHR data analysis. In: 2018 IEEE International Conference on Big Knowledge (ICBK); 2018. p. 415-22

    2018

  58. [67]

    Methods and dimensions of electronic health record data quality assessment: Enabling reuse for clinical research

    Weiskopf NG, Weng C. Methods and dimensions of electronic health record data quality assessment: Enabling reuse for clinical research. J Am Med Inform Assoc. 2013;20(1):144-51

    2013

  59. [68]

    ellmer: Chat with Large Language Models; 2025

    Wickham H, Cheng J, Jacobs A, Aden-Buie G, Schloerke B. ellmer: Chat with Large Language Models; 2025. R package version 0.3.0. Available from:https://ellmer.tidyverse.org

    2025

  60. [69]

    Gemini: A Family of Highly Capable Multimodal Models

    Gemini Team Google. Gemini: A Family of Highly Capable Multimodal Models. arXiv preprint arXiv:231211805. 2023

    2023

  61. [70]

    An evaluation of GPT models for phenotype concept recognition

    Groza T, Caufield H, Gration D, Baynam G, Haendel MA, Robinson PN, et al. An evaluation of GPT models for phenotype concept recognition. BMC Med Inform Decis Mak. 2024;24(1):30

    2024

  62. [71]

    Retrieving evidence from EHRs with LLMs: Possibilities and challenges

    Ahsan H, McInerney DJ, Kim J, Potter C, Young G, Amir S, et al. Retrieving evidence from EHRs with LLMs: Possibilities and challenges. Proc Mach Learn Res. 2024;248:489-505

    2024

  63. [72]

    Large language models for data extraction from unstructured and semi-structured electronic health records: a multiple model performance evaluation

    Ntinopoulos V , Rodriguez Cetina Biefer H, Tudorache I, Papadopoulos N, Odavic D, Risteski P, et al. Large language models for data extraction from unstructured and semi-structured electronic health records: a multiple model performance evaluation. BMJ Health Care Inform. 2025;32(1)

    2025

  64. [73]

    Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system

    Ancker JS, Edwards A, Nosal S, Hauser D, Mauer E, Kaushal R, et al. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Med Inform Decis Mak. 2017;17(1):36

    2017

  65. [74]

    Increasing patient portal usage: Preliminary outcomes from the MyChart Genius Project

    Ramsey A, Lanzo E, Huston-Paterson H, Tomaszewski K, Trent M. Increasing patient portal usage: Preliminary outcomes from the MyChart Genius Project. J Adolesc Health. 2018;62(1):29-35

    2018

  66. [75]

    From smartphone to EHR: A case report on integrating patient-generated health data

    Genes N, Violante S, Cetrangol C, Rogers L, Schadt EE, Chan YFY . From smartphone to EHR: A case report on integrating patient-generated health data. NPJ Digital Medicine. 2018;1(1):23

    2018

  67. [76]

    GPT-4 Technical Report

    OpenAI. GPT-4 Technical Report. arXiv preprint arXiv:230308774. 2023

    2023

  68. [77]

    Introducing the next generation of Claude; 2024

    Anthropic. Introducing the next generation of Claude; 2024. Available from: https://www.anthropic.com/ news/claude-3-family

    2024

  69. [78]

    Introducing Meta Llama 3: The most capable openly available LLM to date; 2024

    Meta AI. Introducing Meta Llama 3: The most capable openly available LLM to date; 2024. Available from: https://ai.meta.com/blog/meta-llama-3

    2024

  70. [79]

    Copilot; 2023

    Microsoft. Copilot; 2023. AI assistant developed by Microsoft.https://copilot.microsoft.com

    2023

  71. [80]

    Frequency and types of patient-reported errors in electronic health record ambulatory care notes

    Bell SK, Delbanco T, Elmore JG, Fitzgerald PS, Fossa A, Harcourt K, et al. Frequency and types of patient-reported errors in electronic health record ambulatory care notes. JAMA Netw Open. 2020;3(6):e205867-7

    2020

  72. [81]

    Adolescents identifying errors and omissions in their electronic health records: A national survey

    Hagström J, Blease C, Kharko A, Scandurra I, Hägglund M. Adolescents identifying errors and omissions in their electronic health records: A national survey. Stud Health Technol Inform. 2023;302(1):242–246

    2023

  73. [82]

    Learning about missing data mechanisms in electronic health records-based research: A survey-based approach

    Haneuse S, Bogart A, Jazic I, Westbrook EO, Boudreau D, Theis MK, et al. Learning about missing data mechanisms in electronic health records-based research: A survey-based approach. Epidemiology. 2016;27(1):82- 90

    2016

  74. [83]

    Automated ICD coding via unsupervised knowledge integration (UNITE)

    Sonabend W A, Cai W, Ahuja Y , Ananthakrishnan A, Xia Z, Yu S, et al. Automated ICD coding via unsupervised knowledge integration (UNITE). Int J Med Inform. 2020;139:104135

    2020

  75. [84]

    Leveraging error-prone algorithm-derived phenotypes: Enhancing association studies for risk factors in EHR data

    Lu Y , Tong J, Chubak J, Lumley T, Hubbard RA, Xu H, et al. Leveraging error-prone algorithm-derived phenotypes: Enhancing association studies for risk factors in EHR data. J Biomed Inform. 2024;157:104690. 16 On Using LLM to Enhance Clinically-Driven Missing Data Recovery Alg...

    2024

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.