Towards Transparent and Accurate Diabetes Prediction Using Machine Learning and Explainable Artificial Intelligence

Carmine Gravino; Fabio Palomba; Pir Bakhsh Khokhar; Viviana Pentangelo

REVIEW 4 major objections 5 minor 2 references

Towards Transparent and Accurate Diabetes Prediction Using Machine Learning and Explainable Artificial Intelligence

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper reports that a soft-voting ensemble of Random Forest, XGBoost, and LightGBM predicts diabetes at 92.50% test accuracy with ROC-AUC 0.975, and that its explainability tools consistently rank BMI, age, general health, income, and…

desk verdict A broad but incremental XAI comparison on a public diabetes dataset, undercut by an internal accuracy mismatch and ambiguous SMOTE ordering, but worth a careful referee. read the letter →

arxiv 2501.18071 v2 pith:JYPQXXZY submitted 2025-01-30 cs.LG cs.AIcs.SE

Pir Bakhsh Khokhar , Viviana Pentangelo , Fabio Palomba , Carmine Gravino This is my paper · ORCID

classification cs.LGcs.AIcs.SE

keywords diabetespredictionensemblelearningsoftvotingexplainableAISHAPLIMESMOTEBRFSS

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that diabetes screening does not have to choose between accuracy and interpretability. Using the 253,680-row Diabetes Binary Health Indicators dataset, it builds seven machine-learning models plus a soft-voting ensemble of Random Forest, XGBoost, and LightGBM and reports a test accuracy of 92.50% with ROC-AUC 0.975. It then applies a suite of explainability tools and claims that they consistently identify BMI, age, general health, income, and physical activity as the main drivers of diabetes risk. If these results hold, clinicians could receive a prediction tool that tells them not only who is at risk but also why, and can even suggest what changes would reduce that risk.

What carries the argument

The load-bearing object is the soft-voting ensemble of Random Forest, XGBoost, and LightGBM, in which each model's predicted probabilities are averaged so that confident models contribute more than they would in a hard-vote majority. It is supported by SMOTE, which synthesizes minority-class rows to counter the dataset's 13.07% diabetes prevalence, and by StandardScaler normalization. The interpretability machinery is a suite rather than a single tool: SHAP supplies global and local additive attributions, LIME supplies instance-level explanations, Explainable Boosting Machines give an interpretable-by-design model, partial dependence plots expose pairwise feature interactions, anchors produce rule thresholds, and counterfactual explanations quantify required feature changes. The paper also defines five explanation-quality metrics, fidelity, faithfulness, sparsity, stability, and consistency, to measure whether the explanations are trustworthy.

What would settle it

Re-run the exact pipeline with a test split that is fixed before any SMOTE synthesis and contains only original BRFSS rows; if test accuracy and ROC-AUC fall materially below 92.50% and 0.975, the central claim is not supported. The re-run should also reconcile the abstract's 92.50% test accuracy with Table II's 92.80% test accuracy.

Watch

Extended reading notes

Core claim

In the paper's own terms, the central result is that combining three tree-based learners, Random Forest, XGBoost, and LightGBM, into a soft-voting ensemble, after SMOTE balancing and standard scaling, yields a diabetes classifier with 92.50% test accuracy and an ROC-AUC of 0.975 on the Diabetes Binary Health Indicators dataset. The accompanying XAI analysis is a second result: SHAP, EBM, and permutation importance consistently rank BMI, general health, age, income, and physical activity as the most influential predictors, while LIME, SHAP force and waterfall plots, anchors, and counterfactual explanations convert those rankings into patient-level statements, such as BMI above 30 combined with low physical activity marking high risk. A third result is quantitative: explanation quality is measurable, with reported fidelity 0.744, faithfulness 0.709, sparsity 10, stability 5.35e-05, and consistency 0.73. The paper frames these together as evidence that a transparent, clinically usable diabetes prediction framework is achievable.

Load-bearing premise

The headline accuracy rests on the assumptions that SMOTE-created synthetic rows never entered the validation or test sets and that the BRFSS self-reported diabetes label is an accurate ground truth; if either fails, the reported 92.50% would overstate performance on real, clinically confirmed patients.

Editorial extensions

If this is right

A soft-voting ensemble of Random Forest, XGBoost, and LightGBM can outperform each of its members, so ensemble construction is a plausible route to better diabetes screening.
If the reported explanations are representative, screening tools can focus data collection on cheap-to-measure factors such as BMI, age, general health, income, and physical activity.
Combining global and local explanation methods can produce actionable thresholds and what-if recommendations, such as targeting BMI reduction or increased physical activity for high-risk patients.
The reported explanation metrics give a concrete template for auditing whether an explanation method is faithful and stable before clinical deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's protocol description does not make explicit that SMOTE was applied only to the training split, so the safest reading is that 92.5% is an upper-bound estimate; an independent re-run with a strictly untouched test set would settle how much of the accuracy is real.
Because the outcome label is a self-reported diabetes diagnosis from BRFSS, the model may be predicting diagnosis-and-reporting behavior rather than biological diabetes; external validation with clinically confirmed cases would test this.
Income surfacing as a top predictor suggests that social determinants carry predictive signal in this dataset, but the paper does not disentangle income from healthcare access, a natural follow-up question.
The agreement across SHAP, EBM, and permutation importance could itself be used as a low-cost model-agnostic consistency check before applying black-box classifiers in other chronic-disease settings.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You asked what I think of arXiv:2501.18071. Short version: it is a competent, incremental application of standard ML and XAI tools to a public diabetes dataset. The genuine strength is breadth—seven models plus an ensemble, and five explanation techniques (SHAP, LIME, EBM, anchors, counterfactuals) with explainability metrics. Most prior work in this space uses one or two XAI methods; this paper gives a more complete picture, and that has practical value for someone wanting a survey-in-one-paper of what works on BRFSS-style data.

The central accuracy claim, however, is not clean. The abstract says the ensemble hits 92.50% test accuracy and 0.975 ROC-AUC; Table II says 92.80% and 0.97. That is a direct internal contradiction, not a rounding difference, and it needs to be fixed. The related SMOTE concern is real but I read it as a reporting flaw rather than evidence of cheating: Section III-B describes splitting first, and Section III-C says models were trained on the balanced training set created from the original training set, which implies SMOTE after the split. But the preprocessing section lists SMOTE earlier, no diagram or code is provided, and no leakage check is reported. That ambiguity is enough to make the headline numbers unverifiable as submitted, and it is the first thing a referee should demand be clarified.

The feature-importance claim in the abstract—BMI, Age, General Health, Income, Physical Activity—is also oversimplified. The SHAP, EBM, and permutation importance figures agree on BMI and general health, but they do not all rank the same five predictors in the same order. That is normal for different explanation methods, but the abstract states the list as if it were a single consensus result. The discussion overreaches in other ways too: claiming the model performs "similar to human practitioners" or "establishes a new standard" goes beyond the evidence.

What holds up? The methodology is standard, the dataset is public and large, and the reported accuracies are plausible for this dataset—others have gotten close to 90% with similar approaches. So the core result is believable once the inconsistencies are resolved. I would not call this a novel scientific contribution, but it is a useful comparative benchmark for the applied ML-for-health audience.

Should it be peer reviewed? Yes, with major revisions. The paper deserves referee time because the XAI comparison is broader than typical and the methodological ambiguity is fixable. I would not cite it in its current form, but after a careful revision addressing the accuracy mismatch, the SMOTE pipeline, and the overclaiming, it could become a reasonable reference for applied XAI work.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes a machine-learning pipeline for binary diabetes prediction on the public Diabetes Binary Health Indicators dataset (derived from BRFSS). Preprocessing consists of median imputation, StandardScaler, and SMOTE; several base models (Random Forest, XGBoost, LightGBM, SVM, Decision Tree, Logistic Regression, Naive Bayes) are trained and compared with a soft-voting ensemble of RF, XGBoost, and LightGBM. The central claim is that the ensemble achieves 92.50% test accuracy and 0.975 ROC-AUC. The paper also applies SHAP, EBM, permutation importance, LIME, PDPs, anchors, and counterfactual explanations, and reports explainability metrics (fidelity, faithfulness, sparsity, stability, consistency) to address four research questions on accuracy, explanation usefulness, combination of methods, and evaluation of explanations.

Significance. If the reported results are correct, the paper provides a useful large-scale empirical comparison of ML and XAI methods on a widely used public dataset, and it explicitly attempts to assess explanation quality rather than only reporting feature rankings. The use of a public dataset, multiple XAI techniques, and a comparative performance table are strengths. However, the contribution is incremental: the methodology is standard, no code or data artifacts are released, and the headline accuracy claim is not yet fully supported because of a mismatch between the abstract and Table II and an ambiguity about whether SMOTE was applied before or after the train/validation/test split. The explainability findings are plausible post-hoc summaries but should not be read as independent external validation of the predictors.

major comments (4)

[Abstract / Section IV-A / Table II] The abstract and Section IV-A state that the ensemble achieved 92.50% test accuracy and 0.975 ROC-AUC, but Table II reports 92.80% test accuracy and 0.97 test ROC-AUC for the same ensemble. This is a direct numerical inconsistency in the central claim. The authors must correct the numbers, state which value is the final test result, and explain any discrepancy (for example, whether validation and test metrics were conflated or whether the abstract predates a re-run).
[Section III-A, III-B, III-C] The SMOTE pipeline ordering is ambiguous and load-bearing. Section III-A lists SMOTE as part of preprocessing, Section III-B describes the 70/15/15 split afterward, and Section III-C says models were 'trained on the balanced training set created from the original training set.' If SMOTE was applied before the split, synthetic minority instances would appear in the validation and test sets, inflating all reported metrics and invalidating them as estimates of generalization. The authors must specify explicitly that SMOTE was fitted only on the training split, that validation and test sets contain only original instances, and preferably report a leakage check or the exact pipeline order.
[Section III-C] The hyperparameter tuning procedure is described only as '3-fold cross-validation and Randomized SearchCV' with no search spaces, no fold construction details, no selected hyperparameters, and no random seeds. Since the accuracy claim depends on the tuned ensemble, this missing information prevents reproduction and makes it impossible to assess overfitting risk. The authors should provide the hyperparameter grids, final hyperparameter values, and the exact evaluation protocol, or release the code.
[Section IV-B and Abstract] The feature-importance claim is not consistently supported by the reported explanations. The abstract and conclusion list BMI, Age, General Health, Income, and Physical Activity as the most influential predictors, but Section IV-B reports SHAP drivers as BMI, GenHlth, Age, and income; the EBM figure highlights BMI and GenHlth; and Fig. 4 (permutation importance) is described as highlighting BMI, general health, and physical health (PhysHlth), with no mention of Income. The authors should reconcile these lists and state which features are consistently top-ranked across methods, or soften the abstract claim accordingly.

minor comments (5)

[Table III] The counterfactual explanation table lists coefficient-like values (e.g., BMI 0.24, PhysActivity 0.57) but does not define the sign convention, the units, or how the 'cost' column should be interpreted; the cost values appear blank or truncated. Please add a legend or a worked example.
[Section IV-B.6 / Table IV] The explainability metrics are presented without definitions or formulas; for example, Sparsity 10.0 and Stability 5.35e-05 need units and direction of goodness so that a reader can interpret whether larger or smaller values are better.
[Section V, Limitations and Future Directions] The sentence 'However, the ligand model cannot be applied technically in the clinical setting due to its large computational requirements' appears to contain a typo (likely 'LightGBM model' or 'the model') and should be corrected.
[Section VI, Conclusion] The conclusion states that 'Interoperability with electronic health records' is essential; this appears to be a word-choice issue, since the intended concept is likely 'interpretability' or 'integration' rather than interoperability.
[Section II, Literature Review] The related-work table is dense and some entries lack dataset details or clear XAI-metric comparisons; adding a column for the evaluation measures used by prior studies would strengthen the positioning of the paper's explainability-metric analysis.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the accuracy and XAI feature rankings are empirical outputs from a public dataset and held-out evaluation, not derivations equivalent to their inputs.

full rationale

The paper's central claims are empirical: an ensemble of Random Forest, XGBoost, and LightGBM with soft voting is reported to reach 92.50% test accuracy and ROC-AUC 0.975 on the public Diabetes Binary Health Indicators dataset, and SHAP, EBM, LIME, PDP, and counterfactual tools are applied post hoc to describe the fitted model's behavior. No parameter is defined in terms of the target, no prediction is a renamed fitted value, and no uniqueness theorem or prior result by these authors is invoked to force a conclusion. The XAI rankings are summaries of the trained ensemble, not inputs used to define the outcome or to fit the model. The pipeline ambiguities flagged by the reader, such as whether SMOTE was applied before or after the train/validation/test split, and the mismatch between Table II (92.80%) and the abstract (92.50%), are experimental-validity and reporting-consistency concerns rather than circularity: even if synthetic instances contaminated the test set, that would be leakage bias, not a derivation that reduces to its own inputs. The paper contains no self-citations from the present authors, so there is no self-citation chain bearing on the result. The empirical accuracy and explainability findings are therefore self-contained and not circular.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central performance claim rests on standard machine learning tools and a public dataset. The main uncharged inputs are the undocumented hyperparameter choices, the SMOTE configuration and ordering, and survey-data assumptions about labels and self-reporting. The paper introduces no new theoretical entities or fitted constants beyond ordinary model parameters.

free parameters (2)

Model hyperparameters = Not reported
All seven models are tuned with RandomizedSearchCV and 3-fold cross-validation, but the search grids and selected values are omitted; the headline accuracy depends on these undocumented choices.
SMOTE sampling strategy = Not reported
The paper says SMOTE is used on the minority class but does not state whether it was configured to fully balance classes, to use a milder ratio, or to be applied only to the training split.

assumptions (4)

domain assumption The BRFSS 'Diabetes_binary' variable is an accurate label for diabetes status.
The model is trained and evaluated against this survey-derived label; misclassification in the label propagates into every reported metric and feature ranking (Section III-A).
domain assumption Self-reported health indicators such as physical activity, general health, and income are measured without systematic bias.
Feature importance and counterfactual recommendations are interpreted as clinically meaningful in the Discussion, but the inputs are self-reports from a telephone survey.
domain assumption SMOTE-generated synthetic instances preserve the true joint distribution of features and labels.
Oversampling changes the training distribution; the paper assumes this improves generalization rather than introducing artifacts (Section III-A).
domain assumption Missing values are missing at random, so median imputation is unbiased.
Median imputation is applied without reporting the proportion of missing values or the mechanism of missingness (Section III-A).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Towards Transparent and Accurate Diabetes Prediction Using Machine Learning and Explainable Artificial Intelligence." pith.science (2026). https://pith.science/paper/JYPQXXZY

@misc{pith2026250118071,
  author       = {Pith},
  title        = {Pith review of: Towards Transparent and Accurate Diabetes Prediction Using Machine Learning and Explainable Artificial Intelligence},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JYPQXXZY}},
  note         = {Machine review of arXiv:2501.18071}
}

read the original abstract

Diabetes mellitus (DM) is a global health issue of significance that must be diagnosed as early as possible and managed well. This study presents a framework for diabetes prediction using Machine Learning (ML) models, complemented with eXplainable Artificial Intelligence (XAI) tools, to investigate both the predictive accuracy and interpretability of the predictions from ML models. Data Preprocessing is based on the Synthetic Minority Oversampling Technique (SMOTE) and feature scaling used on the Diabetes Binary Health Indicators dataset to deal with class imbalance and variability of clinical features. The ensemble model provided high accuracy, with a test accuracy of 92.50% and an ROC-AUC of 0.975. BMI, Age, General Health, Income, and Physical Activity were the most influential predictors obtained from the model explanations. The results of this study suggest that ML combined with XAI is a promising means of developing accurate and computationally transparent tools for use in healthcare systems.

Figures

Figures reproduced from arXiv: 2501.18071 by the authors.

**Figure 1.** Proposed Methodology Diagram for Explainable Diabetes Prediction Using Machine Learning A. Data Source and Data Preprocessing The Diabetes Binary Health Indicators dataset, obtained from the BRFSS, included 253,680 patients with 22 numeric attributes of clinical risk factors, including BMI, cholesterol, blood pressure, activity level, and smoking status. The seWhich types of explanation methods are the most useful … view at source ↗

**Figure 6.** Positive and Negative Feature Effects for a Single Prediction using SHAP Force Plot [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗

**Figure 8.** Cumulative Feature Contributions Impacting Predictions - Decision Plot [PITH_FULL_IMAGE:figures/full_fig_p007_8.png] view at source ↗

Figures from the paper (2 more)

**Figure 9.** Figure 9: Partial Dependence Plot – Interaction between blood pressure and cholesterol levels. demonstrate that controlling factors such as HighBP (coefficient of 1.15) and HighChol (coefficient of 1.17), in addition to bettering dieting and curtailing excessive alcohol intake …

**Figure 10.** Figure 10: Anchor Threshold – Physical activity and BMI thresholds for high diabetes risk. 6) RQ4 - Quality, and Reliability of Explainability Methods: The evaluation metrics used to assess the quality of explainability methods in diabetes prediction models are provided in Tabl…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

2 extracted references · 2 linked inside Pith

[6]

The fidelity value ensures that the explanations accurately represent the model predictions, and faithfulness of 0.709 confirms that they also depict the model’s decision- making

RQ4 - Quality, and Reliability of Explainability Methods: The evaluation metrics used to assess the quality of explain- ability methods in diabetes prediction models are provided in Table IV. The fidelity value ensures that the explanations accurately represent the model predictions, and faithfulness of 0.709 confirms that they also depict the model’s dec...

arXiv
[13]

Jafarzadeh, M

H. Jafarzadeh, M. Mahdianpari, E. Gill, F. Mohammadimanesh, and S. Homayouni. Bagging and boosting ensemble classifiers for classi- fication of multispectral, hyperspectral and polsar data: a comparative evaluation. Remote Sensing, 13(21):4405, 2021. [14] Kaggle. Visceral adipose tissue measurements during pregnancy., 2023. [15] A. Khan, I. Petropoulos, G...

arXiv 2021

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [6]

The fidelity value ensures that the explanations accurately represent the model predictions, and faithfulness of 0.709 confirms that they also depict the model’s decision- making

RQ4 - Quality, and Reliability of Explainability Methods: The evaluation metrics used to assess the quality of explain- ability methods in diabetes prediction models are provided in Table IV. The fidelity value ensures that the explanations accurately represent the model predictions, and faithfulness of 0.709 confirms that they also depict the model’s dec...

arXiv

[2] [13]

Jafarzadeh, M

H. Jafarzadeh, M. Mahdianpari, E. Gill, F. Mohammadimanesh, and S. Homayouni. Bagging and boosting ensemble classifiers for classi- fication of multispectral, hyperspectral and polsar data: a comparative evaluation. Remote Sensing, 13(21):4405, 2021. [14] Kaggle. Visceral adipose tissue measurements during pregnancy., 2023. [15] A. Khan, I. Petropoulos, G...

arXiv 2021