Pith. sign in

REVIEW 4 major objections 5 minor 34 references

Signal Fidelity Index-Aware Calibration for Dementia Predictions Across Heterogeneous Real-World Data

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that diagnostic signal decay can be measured per patient with a Signal Fidelity Index and corrected by a multiplicative, label-free calibration, improving all dementia prediction metrics in simulated heterogeneous EHR data

desk verdict A real question and a simple, interpretable method, but the only evidence is a simulation that builds the effect in; the paper needs real-world validation before its practical claim can be taken seriously. read the letter →

arxiv 2509.08679 v1 pith:D72OB2N7 submitted 2025-09-10 cs.LG

Jingya Cheng , Jiazi Tian , Federica Spoto , Alaleh Azhir , Daniel Mork , Hossein Estiri This is my paper
classification cs.LG
keywords signalfidelityindexdiagnosticdecaylabel-freecalibrationclinicalpredictionmodelselectronichealthrecordsdomainshiftdementiasimulationstudy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a major reason clinical prediction models fail when moved across healthcare systems is diagnostic signal decay: the same diagnosis code can mean different things depending on how it was recorded, encoded, and transmitted. It introduces the Signal Fidelity Index (SFI), a per-patient composite of six data-quality components, and a post hoc calibration that multiplies raw model predictions by a fidelity-derived factor. In 2,500 synthetic datasets built from dementia-risk epidemiology, the calibration at its chosen strength improved every measured metric, with F1-score and recall landing within 1% of reference performance. The intended payoff is a practical, label-free way to adapt probability-based clinical models to new sites without target-domain outcome labels.

What carries the argument

The central object is the Signal Fidelity Index (SFI), the mean of six normalized components—specificity, temporal consistency, entropy, contextual concordance, medication alignment, and trajectory stability—computed from structured EHR data. It carries the argument as a per-patient proxy for diagnostic reliability. The calibration mechanism is a multiplicative adjustment: predicted probability is multiplied by 1 + alpha * (SFI_i - mean_SFI_ref) / mean_SFI_ref, so high-fidelity patients receive amplified predictions and low-fidelity patients attenuated ones. The paper's proof sketch shows this reduces calibration error when the residual between the true label and raw prediction is positively

What would settle it

Take a large claims dataset with adjudicated dementia outcomes, split it by site, train a model on one site, apply the SFI formula at alpha = 2.0, and compare calibrated versus raw predictions. If calibrated performance is not better, or if gains disappear when SFI components are computed from real codes, the claim that diagnostic signal decay is tractably correctable by this label-free adjustment is refuted.

Watch

Extended reading notes

Core claim

At the optimal calibration strength (alpha = 2.0), SFI-aware calibration significantly improves every evaluated metric of a random forest dementia predictor—AUC, balanced accuracy, detection rate, F1-score, precision, and recall—with gains ranging from 10.3% to 32.5% (p < 0.001 across 50 batches). The adjustment moves F1-score and recall to within 1% of reference standards and brings balanced accuracy and detection rate substantially closer to reference. The paper interprets this as evidence that diagnostic signal decay, not just covariate shift, is a measurable and correctable cause of model degradation in EHRs, and that measuring data fidelity directly is enough to recover much of the lost

Load-bearing premise

The core assumption is that the simulation's coding patterns—where high-fidelity diagnoses and medications are generated directly from the true dementia label—reflect how coding noise carries label information in real EHRs; if real diagnostic noise is not informative in that way, the calibration gains will not transfer.

Editorial extensions

If this is right

  • If the claim holds, a trained model can be recalibrated for a new healthcare site using only structured EHR fields and the site's average SFI, with no outcome labels and no retraining.
  • Cross-site performance drops previously attributed to generic distribution shift should be re-examined, because part of the drop may be diagnostic signal decay that this adjustment can recover.
  • The adjustment is model-agnostic and post hoc, so it can be attached to any probability-based classifier without changing the training procedure.
  • For dementia phenotyping, predictions built on stable, specific, contextually appropriate codes can be trusted more, while predictions from rule-out or provisional codes can be dampened.
  • The optimal alpha may need site-specific tuning, but the paper's plateau-detection procedure provides a way to select it from performance patterns rather than target labels.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct real-world test would apply SFI calibration to claims data with validated dementia outcomes; if the gains do not appear, the simulation's generative coding model—where diagnoses and medications are produced from the true label—would be the likely source of improvement rather than a general property of EHR signal decay.
  • The same calibration scheme could plausibly transfer to other chronic conditions with variable coding fidelity, such as diabetes or chronic kidney disease, but the six SFI components and alpha would need phenotype-specific redefinition.
  • Because SFI is computed from local structured data, it could be calculated at each site and used to calibrate models before aggregation in federated learning, giving a label-free way to handle coding differences between sites.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces a Signal Fidelity Index (SFI), a six-component composite measuring diagnostic data quality at the patient level, and proposes SFI-aware calibration, a multiplicative post-hoc adjustment to predicted probabilities. The method is evaluated in a large simulation study (50 batches, 2,500 datasets) using random forest models trained on age and race to predict dementia labels. The authors report that at an optimal alpha of 2.0, calibration significantly improves all six performance metrics (p<0.001) and moves several metrics closer to reference standards. The central claim is that this provides a label-free strategy for improving model performance across heterogeneous real-world healthcare data.

Significance. If the central claim were established, the paper would offer a practically valuable, interpretable tool for recalibrating clinical prediction models without target-domain labels, particularly for administrative datasets. The SFI is clearly motivated and its components are clinically interpretable; the authors also provide a reproducibility-oriented pipeline and an R package. However, the evidence presented is not sufficient to support the claim: the simulation constructs SFI from label-generated codes, the mathematical proof assumes the key correlation it purports to establish, and the reported numerical results contain internal inconsistencies. The manuscript does not provide real-data validation, and the Limitations section concedes that such validation is required. Thus the significance of the contribution is currently conditional on evidence that is not supplied.

major comments (4)
  1. [Methods, Diagnosis Code Assignment and Medication Patterns] The simulation generates diagnosis codes and medications directly from the true dementia label: high-fidelity codes receive 2x weighting for dementia patients and 1-5% rates for non-dementia patients, and dementia medications are assigned at 30%/30%/20%/10% vs. 1%/1%/0.5%/0.5%. Since the SFI is computed from these same codes and medications, SFI is mechanically correlated with the outcome. The calibration formula multiplies raw probabilities by a linear function of (SFI_i - mean SFI), so the adjustment injects label information through the back door. Consequently, the simulated performance gains (e.g., 10.3-32.5% in Table 5) are built into the generative model. No real-world dataset is used to test whether coding noise carries label information in this way, and the Limitations section acknowledges this. This is load-bearing for the central label-free claim.
  2. [Appendix B, Step 5] The proof of calibration-error improvement states: 'If (Y_i - yhat_raw) and (SFI_i - mean SFI) are positively correlated, then the expected error difference is positive.' This condition is exactly the claim that needs to be demonstrated. The proof does not derive this correlation from any property of diagnostic signal decay; it merely assumes it. In the simulation, the correlation is guaranteed by construction, but the paper provides no argument or evidence that it holds in real administrative data. This circularity leaves the theoretical foundation of the method unestablished.
  3. [Appendix C, Phase 1; Results, Phase 1] The alpha-selection procedure is not label-free. The optimal alpha is chosen by testing whether the paired difference between calibrated and raw performance is statistically significant on labeled simulated outcomes (p<0.05). In a true target domain without outcome labels, these p-values cannot be computed, so the method as described cannot be deployed in the claimed label-free setting. Moreover, the plateau-detection rule alpha_opt = min{alpha_i : p(delta(alpha_i))<0.05 and p(delta(alpha_{i+1}))>=0.05} is inapplicable in this experiment because Results report that all tested alpha values yield significant improvements and that all confidence intervals exclude zero; therefore no alpha satisfies the second condition. This is an internal inconsistency in the selection procedure and the reported results.
  4. [Tables 3-6 and Results text] The reported numerical results are internally inconsistent. Table 3 gives raw AUC=0.687, Detection Rate=0.098, Precision=0.693, Recall=0.308; Table 4 gives raw AUC=0.657, Detection Rate=0.080, Precision=0.470, Recall=0.380; Table 5 gives raw AUC=0.687, Precision=0.693, Recall=0.308, but calibrated Precision=0.914 while Table 4 gives calibrated Precision=0.620. Table 6 reports '92.2% and 92.6% closer' for F1 and Recall while the same table lists 99.0% and 99.2%; the text also reports 52.3% and 41.1% for Balanced Accuracy and Detection Rate while Table 6 lists 90.8% and 90.9%. These discrepancies mean the reader cannot determine the actual effect of the method, and they undermine the statistical claims built on these numbers.
minor comments (5)
  1. [References] Several references are duplicated: [24] and [28] are identical (Saerens et al.), [25] and [29] are identical (Zhao et al.), and [7] and [23] are identical (Yaffe et al.). These should be consolidated.
  2. [Methods, Statistical Analysis vs. Evaluation] The Statistical Analysis subsection describes paired t-tests on batch-level means, while the Evaluation subsection describes the Wilcoxon signed-rank test on median paired differences. The Results do not specify which test was used for the reported p<0.001 values. This should be clarified.
  3. [Results, Phase 1 vs. Evaluation] The Evaluation section states 'Optimal performance was achieved at alpha=1.5', while the Results and Abstract state that the optimal alpha is 2.0. These statements should be reconciled.
  4. [Tables 6 and text] The percentages 'closer to reference' in the prose do not match the values in Table 6 for several metrics, as detailed in major comment 4. The table and text should be made consistent.
  5. [Figures] Figure captions refer to color-coded points and error bars, but the figures are not available in the text; at minimum the captions should be self-explanatory without relying on unshown colors.

Circularity Check

3 steps flagged · score 8.0 of 10

Simulation generates SFI from the true dementia label, so the calibration gain is built into the data; Appendix B's proof merely assumes the correlation the method needs.

  1. self definitional [Methods, Simulation, Diagnosis Code Assignment (p.5)]
    "For patients with dementia labels, high-fidelity codes received 2× weighting compared to low-fidelity codes, reflecting the clinical reality that diagnosed dementia patients are more likely to receive definitive diagnostic codes."

    The SFI components (specificity, entropy, temporal consistency, contextual concordance) are computed from these diagnosis codes. Because the codes are generated from the true dementia label with a strong positive association and near-zero rates for non-dementia patients, SFI is a noisy monotone transform of the outcome in every simulated dataset. The calibration formula then multiplies the raw probability by a linear function of SFI, injecting label information through the back door. Thus the 'label-free' improvement is tested only in a world where SFI is constructed from the label; the paper provides no real-data validation, and its Limitations concede that the improvements 'require validation using large-scale administrative datasets.'

  2. other [Appendix B, Step 5, 'Interpretation' (p.23)]
    "If (Yi − ŷi,raw) and (SFIi − ¯SFIS) are positively correlated, then the expected error difference Δerror is positive, thus improving calibration."

    This is the load-bearing step of the mathematical proof, and it assumes exactly what the method needs to show: that the SFI deviation is correlated with the prediction residual. The proof does not derive this correlation from any property of real EHR data; it simply stipulates it. The simulation enforces the stipulation by generating SFI-relevant codes and medications from the true label. Consequently, the 'proof' is a tautology (if the adjustment is correlated with the error, it reduces error) rather than a first-principles justification of label-free calibration.

1 more flagged steps
  1. fitted input called prediction [Appendix C, Phase 1: Optimal Alpha Selection (p.24); Results, Phase 1 (p.10-11)]
    "αopt,m = min{αi : p(Δm(αi))<0.05 and p(Δm(αi+1))≥0.05}"

    The reported optimal α=2.0 is not actually selected by the plateau rule: when every alpha is statistically significant, as the Results state ('All tested alpha values resulted in statistically significant improvements for all metrics'), the condition p(Δm(αi+1))≥0.05 never triggers, so αopt,m is undefined and the reported value is the constraint cap α≤2.0. More importantly, alpha is tuned and then evaluated on the same 2,500 simulated datasets; the improvements at α=2.0 are in-sample performance of a fitted hyperparameter, not an out-of-sample prediction. This is a fitted input being presented as a predictive result.

full rationale

The central circularity is the simulation: SFI is derived from diagnosis codes and medication patterns that are themselves generated from the true dementia label. In the paper's own words, 'For patients with dementia labels, high-fidelity codes received 2× weighting...' and non-dementia patients have near-zero rates; medication assignments are similarly label-conditioned. The SFI therefore carries the outcome label by construction, and the calibration formula y_cal = y_raw * (1 + α(SFI−mean)/mean) injects that label information into the predictions. Appendix B's proof does not rescue the argument: its 'Interpretation' step assumes exactly the positive correlation between the residual and SFI deviation that the generative model manufactures. The alpha-tuning loop (Phase 1) adds another layer: α is selected and evaluated on the same simulated data, so the reported gains are in-sample. The self-citation to the authors' prior work [11] for the term 'diagnostic signal decay' is not load-bearing here and does not materially affect the score. The Limitations explicitly admit that real-world administrative validation is still required, confirming that the headline claim rests on the simulation alone. Because the method's benefit is forced by the label-conditioned generative model, the evidence does not support the stated 'practical, label-free strategy' for real EHR data. Score 8: the central result is forced by construction, though not quite a full identity between derivation and inputs (the raw model and reference benchmarks still contribute independent variation).

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central improvement comes from alpha tuned on labels and from a simulation where SFI is generated by the label. The only novel object, SFI, has no independent benchmark. The axioms listed are the load-bearing premises that, if false, would invalidate the reported gains.

free parameters (3)
  • alpha = 2.0 (chosen by significance-based plateau detection across 50 simulated batches)
    Multiplicative SFI adjustment strength; selected on labeled simulated outcome data, then reported as the optimal parameter.
  • diagnostic code generation multipliers = 2x high-fidelity weighting; low-fidelity 10-30%; high-fidelity 1-5%; medication rates 30/30/20/10% vs 1/1/0.5/0.5%
    Hand-set simulation parameters that determine how strongly SFI correlates with the true label; they directly control the size of the reported improvement.
  • dementia risk model coefficients = age effect 2^((age-65)/5); race multipliers 1.5/1.3/1.1/1.2
    Taken from literature, not fitted here, but they define the labels that in turn define the reference standards.
assumptions (4)
  • ad hoc to paper Prediction miscalibration correlates with (SFI_i - mean SFI); this is required for the error improvement to be positive.
    Appendix B, Step 5 states the error gain is positive only if the residual and SFI deviation are positively correlated; this is the conclusion being used as a premise.
  • domain assumption Diagnostic codes are generated from the true dementia label in the simulation.
    Methods, Diagnosis Code Assignment: high-fidelity codes 2x for dementia patients and near-zero for non-dementia patients, making SFI label-informative by construction.
  • domain assumption A model trained only on age and race is a fair deployed model for dementia prediction.
    Methods, Model Calibration Analysis; the reference model uses only demographic predictors, yet calibration is evaluated against it.
  • domain assumption Reference standards from the simulation are clinically meaningful benchmarks.
    Limitations paragraph 5 admits reference standards are optimal configurations within the simulation framework; real-world reference standards may differ.
invented entities (1)
  • Signal Fidelity Index (SFI)
    purpose: Patient-level composite quantifying diagnostic code quality as the mean of six components.
    No external validation against adjudicated diagnostic quality. In the simulation, SFI is computed from codes generated by the label, so it has no falsifiable handle outside the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Signal Fidelity Index-Aware Calibration for Dementia Predictions Across Heterogeneous Real-World Data." pith.science (2026). https://pith.science/paper/D72OB2N7

@misc{pith2026250908679,
  author       = {Pith},
  title        = {Pith review of: Signal Fidelity Index-Aware Calibration for Dementia Predictions Across Heterogeneous Real-World Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D72OB2N7}},
  note         = {Machine review of arXiv:2509.08679}
}
abstract

\textbf{Background:} Machine learning models trained on electronic health records (EHRs) often degrade across healthcare systems due to distributional shift. A fundamental but underexplored factor is diagnostic signal decay: variability in diagnostic quality and consistency across institutions, which affects the reliability of codes used for training and prediction. \textbf{Objective:} To develop a Signal Fidelity Index (SFI) quantifying diagnostic data quality at the patient level in dementia, and to test SFI-aware calibration for improving model performance across heterogeneous datasets without outcome labels. \textbf{Methods:} We built a simulation framework generating 2,500 synthetic datasets, each with 1,000 patients and realistic demographics, encounters, and coding patterns based on dementia risk factors. The SFI was derived from six interpretable components: diagnostic specificity, temporal consistency, entropy, contextual concordance, medication alignment, and trajectory stability. SFI-aware calibration applied a multiplicative adjustment, optimized across 50 simulation batches. \textbf{Results:} At the optimal parameter ($\alpha$ = 2.0), SFI-aware calibration significantly improved all metrics (p $<$ 0.001). Gains ranged from 10.3\% for Balanced Accuracy to 32.5\% for Recall, with notable increases in Precision (31.9\%) and F1-score (26.1\%). Performance approached reference standards, with F1-score and Recall within 1\% and Balanced Accuracy and Detection Rate improved by 52.3\% and 41.1\%, respectively. \textbf{Conclusions:} Diagnostic signal decay is a tractable barrier to model generalization. SFI-aware calibration provides a practical, label-free strategy to enhance prediction across healthcare contexts, particularly for large-scale administrative datasets lacking outcome labels.

Figures

Figures reproduced from arXiv: 2509.08679 by the authors.

Figure 1
Figure 1. Mean improvement from calibration across all alpha values (0.5 to 2.5) [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. 95% confidence intervals for calibration improvement across alpha [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. Performance comparison at optimal alpha ( [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

34 extracted references · 5 linked inside Pith

  1. [1]

    Early-onset Alzheimer disease.Neurol Clin

    Mendez MF. Early-onset Alzheimer disease.Neurol Clin. 2017;35(2):263- 281

    2017

  2. [2]

    The diagnosis of young-onset dementia.Lancet Neurol

    Rossor MN, Fox NC, Mummery CJ, Schott JM, Warren JD. The diagnosis of young-onset dementia.Lancet Neurol. 2010;9(8):793-806

    2010

  3. [3]

    2023 Alzheimer’s disease facts and figures

    Alzheimer’s Association. 2023 Alzheimer’s disease facts and figures. Alzheimers Dement. 2023;19(4):1598-1695

    2023

  4. [4]

    The global prevalence of dementia: a systematic review and metaanalysis.Alzheimers Dement

    Prince M, Bryce R, Albanese E, Wimo A, Ribeiro W, Ferri CP. The global prevalence of dementia: a systematic review and metaanalysis.Alzheimers Dement. 2013;9(1):63-75

    2013

  5. [5]

    Inequalities in dementia incidence between six racial and ethnic groups over 14 years

    Mayeda ER, Glymour MM, Quesenberry CP, Whitmer RA. Inequalities in dementia incidence between six racial and ethnic groups over 14 years. Alzheimers Dement. 2016;12(3):216-224

    2016

  6. [6]

    Analysis of dementia in the US population using Medicare claims: insights from linked survey and administrative claims data.Alzheimers Dement (N Y)

    Chen Y, Tysinger B, Crimmins E, Zissimopoulos JM. Analysis of dementia in the US population using Medicare claims: insights from linked survey and administrative claims data.Alzheimers Dement (N Y). 2019;5:197-207

    2019

  7. [7]

    Effect of socioeconomic disparities on incidence of dementia among biracial older adults: prospective study.BMJ

    Yaffe K, Falvey C, Harris TB, et al. Effect of socioeconomic disparities on incidence of dementia among biracial older adults: prospective study.BMJ. 2013;347:f7051

    2013

  8. [8]

    A meta-analysis of Alzheimer’s disease incidence and prevalence comparing African Americans and Caucasians.J Alzheimers Dis

    Steenland K, Goldstein FC, Levey A, Wharton W. A meta-analysis of Alzheimer’s disease incidence and prevalence comparing African Americans and Caucasians.J Alzheimers Dis. 2016;50(1):71-76

    2016

Show all 34 references
  1. [9]

    Association of incident dementia with hospitalizations.JAMA

    Phelan EA, Borson S, Grothaus L, Balch S, Larson EB. Association of incident dementia with hospitalizations.JAMA. 2012;307(2):165-172

    2012

  2. [10]

    Australian pop- ulation trends and disparities in diabetes-related hospitalizations.Diabetes Care

    Zilkens RR, Duke J, Horner B, Semmens JB, Bruce DG. Australian pop- ulation trends and disparities in diabetes-related hospitalizations.Diabetes Care. 2011;34(5):1130-1132

    2011

  3. [11]

    Quantifying Diagnostic Signal Decay in Dementia: A National Study of Medicare Hospitalization Data,

    F. Spoto, J. Tian, J. H¨ ugel, D. T. Ortega, C. S. Ritchie, D. Blacker, F. Dominici, C. J. Patel, D. Mork, and H. Estiri, “Quantifying Diagnostic Signal Decay in Dementia: A National Study of Medicare Hospitalization Data,”arXiv preprint arXiv:2506.14669, 2025

    2025 arXiv

  4. [12]

    Rethinking domain adaptation for ma- chine learning over clinical language.JAMIA Open

    Laparra E, Bethard S, Miller TA. Rethinking domain adaptation for ma- chine learning over clinical language.JAMIA Open. 2020;3: 146–150

    2020

  5. [13]

    Preventing failures due to dataset shift: Learning predictive models that transport.arXiv[stat.ML]

    Subbaswamy A, Schulam P, Saria S. Preventing failures due to dataset shift: Learning predictive models that transport.arXiv[stat.ML]. 2018. Available: http://arxiv.org/abs/1812.04597 19

    2018 arXiv

  6. [14]

    Feature robustness in non-stationary health records: Caveats to deployable model performance in common clinical machine learning tasks

    Nestor B, McDermott MBA, Boag W, Berner G, Naumann T, Hughes MC, et al. Feature robustness in non-stationary health records: Caveats to deployable model performance in common clinical machine learning tasks. arXiv[cs.LG]. 2019. Available: http://arxiv.org/abs/1908.00690

    2019 arXiv

  7. [15]

    EHR foundation models improve robustness in the presence of temporal distribution shift.Sci Rep

    Guo LL, Steinberg E, Fleming SL, Posada J, Lemmon J, Pfohl SR, et al. EHR foundation models improve robustness in the presence of temporal distribution shift.Sci Rep. 2023;13: 3767

    2023

  8. [16]

    Scal- able and accurate deep learning with electronic health records.npj Digital Medicine

    Rajkomar A, Oren E, Chen K, Dai AM, Hajaj N, Hardt M, et al. Scal- able and accurate deep learning with electronic health records.npj Digital Medicine. 2018;1: 18

    2018

  9. [17]

    Calibration drift in regression and machine learning models for acute kidney injury.J Am Med Inform Assoc

    Davis SE, Lasko TA, Chen G, Siew ED, Matheny ME. Calibration drift in regression and machine learning models for acute kidney injury.J Am Med Inform Assoc. 2017;24: 1052–1061

    2017

  10. [18]

    Machine learning approaches for electronic health records phenotyping: a methodical review

    Yang S, Varghese P, Stephenson E, Tu K, Gronsbell J. Machine learning approaches for electronic health records phenotyping: a methodical review. J Am Med Inform Assoc. 2023;30: 367–381

    2023

  11. [19]

    Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis.IEEE J Biomed Health Inform

    Shickel B, Tighe PJ, Bihorac A, Rashidi P. Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis.IEEE J Biomed Health Inform. 2018;22: 1589–1604

    2018

  12. [20]

    Electronic health records-driven pheno- typing: challenges, recent advances, and perspectives.J Am Med Inform Assoc

    Pathak J, Kho AN, Denny JC. Electronic health records-driven pheno- typing: challenges, recent advances, and perspectives.J Am Med Inform Assoc. 2013;20: e206–11

    2013

  13. [21]

    Federated Learning for Healthcare Informatics.Int J Healthc Inf Syst Inform

    Xu J, Glicksberg BS, Su C, Walker P, Bian J, Wang F. Federated Learning for Healthcare Informatics.Int J Healthc Inf Syst Inform. 2020; 1–19

    2020

  14. [22]

    The future of digital health with federated learning.NPJ Digit Med

    Rieke N, Hancox J, Li W, Milletar ` ı F, Roth HR, Albarqouni S, et al. The future of digital health with federated learning.NPJ Digit Med. 2020;3: 119

    2020

  15. [23]

    Effect of socioeconomic disparities on incidence of dementia among biracial older adults: prospective study

    Yaffe K, Falvey C, Harris TB, et al. Effect of socioeconomic disparities on incidence of dementia among biracial older adults: prospective study. BMJ. 2013;347:f7051

    2013

  16. [27]

    Conformal prediction under covariate shift

    Angelopoulos AN, Bates S. Conformal prediction under covariate shift. Journal of Machine Learning Research, 23:1–45, 2022

    2022

  17. [28]

    Adjusting the outputs of a classifier to new a priori probabilities: a simple procedure.Neural Computation, 14(1):21–41, 2002

    Saerens M, Latinne P, Decaestecker C. Adjusting the outputs of a classifier to new a priori probabilities: a simple procedure.Neural Computation, 14(1):21–41, 2002

    2002

  18. [29]

    Calibrating large language models with in-context learning.arXiv preprint arXiv:2305.12345, 2023

    Zhao Z, Wallace E, Feng S, Klein D, Singh S. Calibrating large language models with in-context learning.arXiv preprint arXiv:2305.12345, 2023. Available:http://arxiv.org/abs/2305.12345

    2023 arXiv

  19. [30]

    Evidential deep learning to quantify classification uncertainty.Advances in Neural Information Processing Sys- tems, 31:3179–3189, 2018

    Sensoy M, Kaplan L, Kandemir M. Evidential deep learning to quantify classification uncertainty.Advances in Neural Information Processing Sys- tems, 31:3179–3189, 2018. 21 Appendix A: Signal Fidelity Index Formulations For patienti, the SFI is defined as: SFIi = 1 6 (Specifici...

    2018

  20. [32]

    Define the Signal Fidelity Index (SFI) for each patient asSF Ii, and let ¯SF IS denote the average SFI in the training data

    Setup and Definitions Let ˆyi,raw =P(Y= 1|X i, DS) be the predicted probability for patientibased on a model trained on source datasetD S. Define the Signal Fidelity Index (SFI) for each patient asSF Ii, and let ¯SF IS denote the average SFI in the training data

  21. [33]

    Miscalibration and First-Order Approximation We hypothesize a functionfthat maps raw predictions and fidelity to calibrated outputs: ˆyi,ideal =f(ˆyi,raw, SF Ii) 22 Approximatingfvia first-order Taylor expansion around ¯SF IS: f(ˆyi,raw, SF Ii)≈f(ˆy i,raw, ¯SF IS) + (SF Ii − ¯...

  22. [34]

    SFI-Aware Calibration Formula We model the adjustment as: ˆyi,calibrated = ˆyi,raw · 1 +α· SF Ii − ¯SF IS ¯SF IS whereαis a tunable sensitivity parameter

  23. [35]

    Calibration Error Improvement Let the calibration error be defined as the squared difference from the true label: ∆error =E (Yi −ˆyi,raw)2 −(Y i −ˆyi,calibrated)2 Expanding this: ∆error =E 2(Yi −ˆyi,raw)(ˆyi,calibrated −ˆyi,raw)−(ˆyi,calibrated −ˆyi,raw)2 Substitute the calibr...

  24. [36]

    Interpretation If (Yi −ˆyi,raw) and (SF Ii − ¯SF IS) are positively correlated, then the expected error difference ∆ error is positive, thus improving calibration

  25. [37]

    The optimal alpha for each metric was defined as: αopt,m = min{αi :p(∆ m(αi))<0.05 andp(∆ m(αi+1))≥0.05}, subject to the constraintα opt,m ≤2.0 to prevent over-calibration

    Optimal Alpha To find optimalα, minimize calibration error: α∗ = E[(Yi −ˆyi,raw)·ˆyi,raw · SF Ii− ¯SF IS ¯SF IS ] E[ˆy2 i,raw ·( SF Ii− ¯SF IS ¯SF IS )2] 23 Appendix C: Calibration Algorithm Details Phase 1: Optimal Alpha Selection For each metricmand alpha valueα i, the mean ...

  26. [2022]

    Available:http://arxiv.org/abs/2203.09876. 20

    arXiv

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.