Pith. sign in

REVIEW 4 major objections 7 minor 10 references

Comparative Analysis of Stroke Prediction Models Using Machine Learning

T0 review · 4 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Random Forest and SVM reach 99% stroke-prediction accuracy after resampling

desk verdict The paper's 99% accuracy is likely a balancing-before-split artifact; the age-specific feature analysis is the only fresh part. read the letter →

arxiv 2505.09812 v1 pith:5A24XJXQ submitted 2025-05-14 cs.LG

Anastasija Tashkova , Stefan Eftimov , Bojan Ristov , Slobodan Kalajdziski This is my paper · ORCID
classification cs.LG
keywords strokepredictionmachinelearningclassimbalanceSMOTEoversamplingRandomForestfeatureimportanceSHAP
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper compares five machine learning models for stroke prediction on a public 5,110-patient dataset and argues that the main obstacle is class imbalance, not model choice. After correcting the imbalance with oversampling or SMOTE, the paper reports that Random Forest and Support Vector Machines exceed 99% test accuracy, with Random Forest reaching precision, recall, and F1-scores of 0.997 under oversampling. It also claims that age, average glucose level, and BMI are the most influential predictors, and that these rankings shift for patients aged 65-80, where work type, glucose, hypertension, and heart disease gain weight. If the claims hold, the practical message for stroke-risk screening is that tuned ensemble models plus a balancing step can give very high discrimination on benchmark data, but the authors note sensitivity must improve before clinical deployment.

What carries the argument

The load-bearing mechanism is the combination of resampling and hyperparameter-tuned classifiers. Oversampling duplicates minority-class stroke records; SMOTE (creating synthetic minority examples by interpolating between neighboring samples) generates new ones instead, and randomized hyperparameter search tunes each model before evaluation. This balancing step is what allows Random Forest and SVM to reach the reported 99% accuracy, while tree-based feature importance and SHAP (Shapley-value-based feature attribution) plots carry the interpretability claim about age, glucose, BMI, and the elderly-specific shift in feature rankings.

What would settle it

Split the original 5,110 records into train and test before applying any balancing, then train with oversampling or SMOTE only on the training side and evaluate on the untouched test fold that keeps the natural 4.87% stroke rate; if Random Forest and SVM accuracy falls well below 99% and sensitivity is low, the reported metrics depend on evaluating already-balanced data.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that Random Forest and SVM benefit dramatically from resampling: with oversampling they reach 99.02% and 99.28% test accuracy respectively, and Random Forest achieves precision, recall, and F1 all at 0.997. SMOTE yields slightly lower but still strong results (Random Forest 93.98%, SVM 93.52%), while undersampling degrades accuracy to 74-83%, leading the authors to conclude oversampling and SMOTE are the appropriate fixes for this severely imbalanced dataset. On feature importance, age is consistently the top predictor, followed by average glucose level and BMI; restricting the cohort to patients aged 65-80 makes work type and glucose level more influential and raises the prominence of hypertension and heart disease. The authors accordingly claim that ensemble models handle imbalance better than linear models, and that age-specific predictive models would increase clinical usefulness, while acknowledging that sensitivity remains the limiting factor for real-world stroke screening.

Load-bearing premise

The paper assumes that test accuracy on a held-out slice of the same dataset, measured after synthetic or duplicated stroke cases were added, reflects how well the model would predict strokes for patients it has never seen.

Editorial extensions

If this is right

  • On this dataset, class imbalance rather than model choice appears to be the binding constraint: once balanced, even a kernel SVM reaches 99% test accuracy.
  • Because sensitivity remains the reported weak spot, clinical use would require threshold tuning or cost-sensitive learning, not simply maximizing accuracy.
  • XGBoost offers the most balanced precision-recall profile under SMOTE (F1 0.926), making it the candidate to prefer when missing a stroke is costly.
  • Screening instruments for patients aged 65-80 should weight glucose, work type, hypertension, and heart disease more heavily than general-population rankings suggest.
  • Age, average glucose level, and BMI are sufficient as the primary screening variables in a general adult population.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 99% figure is best read as an upper bound: because the test split comes from the same dataset after synthetic or duplicated samples were added, it cannot be read as expected performance on a fresh hospital population with the natural 4.87% stroke rate.
  • A stronger validation design would split before balancing, train on balanced data, and test on an untouched, naturally imbalanced cohort; I would expect reported sensitivity to drop under that design.
  • The age-stratified SHAP result suggests a testable extension: build separate risk scores for the 65-80 band and compare net reclassification against a single all-ages model on external data.
  • Because SMOTE interpolates between existing minority points, it can hide poor generalization when the minority class is small; a direct check is to see how performance changes as the number of synthetic examples is reduced.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. This paper presents a comparative evaluation of five machine learning classifiers (Logistic Regression, Random Forest, Decision Tree, SVM, and XGBoost) on the publicly available Stroke Prediction Dataset (5,110 records, 4.87% stroke cases). The authors describe a preprocessing pipeline that imputes missing BMI values with an iterative random forest imputer, encodes categorical variables, and applies oversampling, undersampling, and SMOTE to address class imbalance. They report that SVM and Random Forest achieve over 99% test accuracy under oversampling, with Random Forest reaching precision, recall, and F1 of 0.997. The paper also presents feature importance analyses, including a SHAP-based comparison for the full dataset and for elderly patients aged 65–80, and concludes that age, average glucose level, and BMI are the most influential predictors while sensitivity remains a limiting factor for clinical use.

Significance. If the reported accuracy figures were valid, the paper would provide a useful benchmark for stroke prediction on a widely used public dataset and a reasonable illustration of imbalance-handling techniques. The SHAP analysis for the elderly subgroup is a constructive direction. However, the central empirical claim is undermined by an unresolved methodological ambiguity: the manuscript does not state whether class balancing was applied before or after the train/test split. Because the reported metrics are the paper's main contribution, the current version cannot be considered reliable as a scientific record. The paper also lacks confidence intervals, a detailed data-split description, or code, which limits reproducibility. These issues must be resolved before the findings can be assessed.

major comments (4)
  1. [Section II-E and Section V-A] The manuscript never states whether balancing was applied to the full dataset before splitting or only to the training partition. If oversampling and SMOTE were applied before the split, then the test set contains duplicated real patients and synthetic patients generated from training data, so the reported test accuracy, precision, and recall do not measure generalization to new patients. This concern is load-bearing because the headline result of 99.28% accuracy for SVM under oversampling depends entirely on this choice. The authors must specify the exact pipeline order and, ideally, re-run the evaluation with balancing performed inside each training fold only.
  2. [Section V-C and Abstract] There is an internal inconsistency between the abstract's statement that 'sensitivity remains a limiting factor for real-world clinical applications' and the reported Random Forest recall of 0.997 under oversampling. On the original imbalanced distribution (4.87% positives), such a recall is implausible, whereas on a balanced test set it is trivially achievable. This tension supports the interpretation that the test set was balanced before splitting, and it must be resolved by reporting performance on an untouched, original-distribution test set.
  3. [Section IV-B] Hyperparameter tuning with RandomizedSearchCV appears to have been performed on the balanced dataset, since the tuning stage is described after the balancing techniques are introduced in Section II-E. If cross-validation during tuning also includes synthetic or duplicated minority instances, the selected hyperparameters may be optimized for artifacts of the balancing procedure rather than for genuine signal. The authors should clarify the timing of balancing relative to hyperparameter tuning and ensure that no information from synthetic or duplicated instances leaks into model selection.
  4. [Section V-A] The paper reports accuracy differences as small as 0.26% (e.g., SVM 99.28% versus Random Forest 99.02% under oversampling) without confidence intervals, statistical significance tests, or repeated-seed variation. Given that the original dataset contains only 249 positive cases, such differences are likely within sampling noise. The authors should report confidence intervals or use repeated stratified cross-validation with variance estimates to support claims of relative model superiority.
minor comments (7)
  1. [Section I-C] The 'previous studies' cited in the research gap are Kaggle forum posts and dataset discussion pages, not peer-reviewed comparative machine learning studies; the authors should cite actual published work on stroke prediction with class imbalance and feature importance.
  2. [Section II-E] The phrase 'By combining these techniques' is misleading because the experimental results in Section V treat oversampling, undersampling, and SMOTE as separate alternatives, not as a combination; the text should be reworded to reflect the actual procedure.
  3. [Table I] Hyperparameter names are inconsistently formatted (e.g., 'n estimators' should be 'n_estimators', 'max depth' should be 'max_depth') and the table would benefit from a consistent code-style notation.
  4. [Section V-B] The reported test accuracies are not accompanied by the specific train/test split ratio, random seed, or number of repeated runs, so the reader cannot assess the stability of the numbers.
  5. [Section VI] The text refers to 'Feature 7', 'Feature 8', and 'Feature 9' without providing a feature-to-index mapping anywhere in the paper; please include a table or list that defines these indices.
  6. [References] References [7] and [8] cite blog posts rather than the original methodological sources; for SMOTE, the authors should cite Chawla et al. (2002), and for class imbalance handling, a more standard textbook or survey reference would be appropriate.
  7. [Section V-C] The notation 'Precision, Recall, and F1-score all at 0.997' would be clearer if each metric were reported with a model-specific label and an explicit number of significant figures, since the classification report likely gives more decimal places.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an empirical benchmark study, and the reported accuracies, while possibly affected by methodological ambiguity around the train/test split, are not shown to reduce to the model's own inputs by construction.

full rationale

The paper does not present a derivation chain in which a predicted quantity is defined in terms of the fitted parameters or in which a result is imported from a self-citation. It is a comparative machine-learning study on a public Kaggle dataset: models are trained, hyperparameters are tuned with RandomizedSearchCV, and test accuracies are reported after applying oversampling, SMOTE, or undersampling. The reported test accuracy is the benchmark result itself, not a quantity derived from a fitted equation. The main concern present in the text is that the manuscript never explicitly states whether the balancing techniques were applied only to the training fold or to the entire dataset before splitting, which could inflate test metrics if the test set contained duplicated or synthetic rows. However, this is a methodological ambiguity and a potential leakage risk, not a demonstrated circular reduction: no quoted passage shows that the test set was constructed from the same synthetic/duplicated rows used for training, and no equation or definition equates the reported accuracy with the training objective by construction. The references cited are external datasets and general methodology sources, not load-bearing self-citations. Because the rules require quoting a specific reduction to claim circularity, and no such reduction is present, the appropriate finding is no significant circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new entities are invented. The paper's central results depend on several methodological choices: the balancing technique, the split, and the imputation method. These are free parameters in the analysis pipeline that are not fixed by external data or theory, and their choices determine the headline accuracy numbers.

free parameters (3)
  • Class balancing strategy (oversampling, undersampling, SMOTE)
    The choice among three balancing techniques is a methodological decision that directly determines the reported accuracy numbers, and the paper prefers the technique that produces the highest accuracy (oversampling) without external validation.
  • Train/test split ratio and random seed
    The paper does not state the split ratio or random seed, yet the headline numbers like 99.28% accuracy depend on this choice.
  • Imputation model for BMI (Iterative Imputer with Random Forest Regressor)
    The choice of imputation model influences the dataset and therefore the results; the paper does not report the impact of this choice on model performance.
assumptions (4)
  • domain assumption The Stroke Prediction Dataset accurately represents the target clinical population for stroke risk prediction.
    The paper treats the Kaggle dataset as a basis for generalizable stroke prediction, but this dataset is not a clinical cohort and has known limitations, which the paper does not address.
  • domain assumption SMOTE-generated synthetic instances and duplicated oversampled instances are valid substitutes for real patient data in evaluation.
    The paper evaluates models on data that includes synthetic minority examples, assuming this measures real-world sensitivity.
  • domain assumption Accuracy, precision, recall, and F1 computed on a single train/test split are sufficient evidence of model quality.
    The paper relies on point estimates from one split without confidence intervals or repeated cross-validation, treating these as reliable indicators.
  • standard math Standard machine learning algorithms as implemented by common libraries are correct and unbiased baseline tools.
    The paper uses off-the-shelf implementations without formal verification, which is a normal background assumption for such benchmark studies.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Comparative Analysis of Stroke Prediction Models Using Machine Learning." pith.science (2026). https://pith.science/paper/5A24XJXQ

@misc{pith2026250509812,
  author       = {Pith},
  title        = {Pith review of: Comparative Analysis of Stroke Prediction Models Using Machine Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5A24XJXQ}},
  note         = {Machine review of arXiv:2505.09812}
}
read the original abstract

Stroke remains one of the most critical global health challenges, ranking as the second leading cause of death and the third leading cause of disability worldwide. This study explores the effectiveness of machine learning algorithms in predicting stroke risk using demographic, clinical, and lifestyle data from the Stroke Prediction Dataset. By addressing key methodological challenges such as class imbalance and missing data, we evaluated the performance of multiple models, including Logistic Regression, Random Forest, and XGBoost. Our results demonstrate that while these models achieve high accuracy, sensitivity remains a limiting factor for real-world clinical applications. In addition, we identify the most influential predictive features and propose strategies to improve machine learning-based stroke prediction. These findings contribute to the development of more reliable and interpretable models for the early assessment of stroke risk.

Figures

Figures reproduced from arXiv: 2505.09812 by the authors.

Figure 1
Figure 1. Exploratory Data Analysis (EDA) of Stroke Prediction Dataset [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Summary results from models [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. SHAP summary plots for the XGBoost model [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (1 more)
Figure 3
Figure 3. Figure 3: XGBoost Model Feature Importance 2) and heart disease (Feature 3) as more significant than in the full dataset, aligning with expectations that cardiovascular conditions play a crucial role in this age group. SHAP analysis of the XGBoost model ( [PITH_FULL_IMAGE:figur…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

10 extracted references · 10 canonical work pages

  1. [1]

    https://www.ahajournals.org/doi/10.1161/JAHA.124.036142

    Global Burden of Disease Study, Lancet Neurol., 2024. https://www.ahajournals.org/doi/10.1161/JAHA.124.036142

    work page doi:10.1161/jaha.124.036142 2024

  2. [2]

    Insights, 2025

    Machine Learning in Healthcare Transformation, Northwest Educ. Insights, 2025. https://northwest.education/insights/machine-learning/how-ml-will-transform-healthcare/

    work page 2025

  3. [3]

    https://pmc.ncbi.nlm.nih.gov/articles/PMC11106277/

    Fedesoriano, Stroke Dataset Discussion, Kaggle Forum, 2021. https://pmc.ncbi.nlm.nih.gov/articles/PMC11106277/

    work page 2021

  4. [4]

    Stroke Prediction Dataset - Discussion on Data and Methodology

    Fedesoriano. "Stroke Prediction Dataset - Discussion on Data and Methodology." Kaggle, 2021. https://www.kaggle.com/datasets/fedesoriano/stroke-prediction-dataset/discussion/453963

    work page 2021

  5. [5]

    Soriano, Stroke Prediction Dataset, Kaggle, 2021

    F. Soriano, Stroke Prediction Dataset, Kaggle, 2021. https://www.kaggle.com/datasets/fedesoriano/stroke-prediction-dataset

    work page 2021

  6. [6]

    Leveraging Multivariate Analysis for Stroke Risk, Saudi J. Neurol. Sci. https://nsj.org.sa/content/29/3/190

    work page

  7. [7]

    Chawla et al., SMOTE for Classification, Domino AI Blog, 2024

    N. Chawla et al., SMOTE for Classification, Domino AI Blog, 2024. https://domino.ai/blog/smote-oversampling-technique

    work page 2024

  8. [8]

    https://blog.roboflow.com/handling-unbalanced-classes/

    Roboflow Team, Handling Class Imbalance, Roboflow Blog, 2025. https://blog.roboflow.com/handling-unbalanced-classes/

    work page 2025

Show all 10 references
  1. [9]

    http://ijlbpr.com/uploadfiles/45vol13issue7pp249-253.pdf

    XGBoost for Stroke Prediction, IJLBPR J., 2024. http://ijlbpr.com/uploadfiles/45vol13issue7pp249-253.pdf

    2024

  2. [10]

    0<>r= Ν39r

    11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEco...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.