Pith. sign in

REVIEW 4 major objections 6 minor 22 references

Classification of Deceased Patients from Non-Deceased Patients using Random Forest and Support Vector Machine Classifiers

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper reports that Random Forest and Support Vector Machine classifiers, using demographics, first lab values, and diagnosis codes, separate deceased from non-deceased COVID-19 patients with near-perfect accuracy (99.78% and 100%).

desk verdict This is a course project report whose near-perfect accuracy is best explained by target leakage from outcome-informative diagnosis codes, not by a real predictive signal. read the letter →

arxiv 2411.18759 v1 pith:N7SLS7AB submitted 2024-11-27 cs.LG

Dheeman Saha , Aaron Segura , Biraj Tiwari This is my paper · ORCID
classification cs.LG MSC 68T0562H30
keywords COVID-19mortalitypredictionrandomforestsupportvectormachinesepsisclinicaldataminingfeatureimportanceconfusionmatrix
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper attempts to show that machine-learning classifiers can tell, from data available early in a hospital stay, which COVID-19 patients will die and which will survive. Using a balanced cohort of 9,366 patients from a national COVID-19 database, the authors train a Support Vector Machine and a Random Forest on demographics, first laboratory values, and diagnosis codes. They report near-perfect accuracy: 99.78% for SVM and 100% for Random Forest in 10-fold cross-validation, with 100% sensitivity for both. If correct, this would imply that death risk in COVID-19 is almost completely separable in the features physicians already collect. The authors themselves note that the classes appear so easily separable that overfitting is a concern.

What carries the argument

The argument rests on two standard classifiers: a Random Forest of 2,000 trees with depth up to 50 using Gini-index splits, and a Support Vector Machine with an RBF kernel and regularization C=0.01. Feature selection is done with an ExtraTreeClassifier that ranks the original 227 attributes by Gini importance, reducing to the top 20. Missing lab values are imputed by sampling from a normal distribution fitted to the non-missing z-scores, and all numeric features are min-max scaled to [0,1]. Performance is assessed with 10-fold cross-validation, confusion matrices, and ROC curves.

What would settle it

Re-run the classifiers using only diagnosis codes present on or before the admission encounter (or within 24 hours of admission), leaving all other features unchanged; if accuracy drops substantially from the reported 99.78% and 100%, the original result was driven by outcome leakage.

Watch

Extended reading notes

Core claim

The central claim is that a Support Vector Machine and a Random Forest, trained on selected numeric and binary features, separate deceased from non-deceased COVID-19 patients with essentially perfect accuracy. The confusion matrices show 99.78% accuracy for SVM (on 25% of the data) and 100% for Random Forest (under 10-fold cross-validation), with sensitivity of 100% and specificity above 99.7% for both. The most important features, ranked by Gini gain, are oxygen saturation by pulse oximetry, erythrocyte count, acute kidney failure, INR, and severe sepsis with septic shock. The paper reports that the two classes appear almost linearly separable in the selected features, which the authors say explains the extreme accuracy.

Load-bearing premise

The diagnosis codes used as predictors are pulled from a table containing all records from 2015 onward, so they can be recorded during the same hospitalization and are partly determined by the outcome; if they are not available at admission, the model is predicting death using the death itself.

Editorial extensions

If this is right

  • If the near-perfect separation is real, hospitals could flag high-risk COVID-19 patients from the first lab panel and admission codes, enabling earlier escalation of care.
  • The top features (oxygen saturation, kidney failure, INR, sepsis codes) match the clinical picture of viral sepsis and end-organ damage, so the model is consistent with known pathophysiology.
  • The reported ease of separation suggests the two classes are almost linearly separable in the chosen feature space, meaning simpler scoring systems might match the classifiers.
  • The authors propose extending the work to time-series models that predict a surviving patient's transition to death, which would have direct ICU and ventilator planning value.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The Condition table contains all diagnosis records from 2015 onward, so codes like severe sepsis with septic shock and cardiac arrest can be recorded after the outcome; this leakage likely inflates the accuracy, and a study using only admission-day codes would probably show much lower performance.
  • The 100% Random Forest accuracy on a balanced test set is a red flag for label leakage or overfitting; external validation on a different hospital system's data would be a stronger test.
  • Imputing missing labs from a normal distribution of z-scores can erase clinical meaning (for example, not ordering a lab is informative), so the imputation method itself may contribute to the clean separation.
  • If the leakage were removed, the remaining predictors (oxygen saturation, erythrocytes, sodium, temperature) would still plausibly carry real prognostic signal, but likely at accuracy levels well below 100%.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper trains Random Forest and Support Vector Machine classifiers on a Cerner COVID-19 dataset of 9,366 balanced patients (4,683 deceased, 4,683 non-deceased) to predict in-hospital death. Features include demographics, first laboratory values, and binary indicators of conditions from the Condition table. After ExtraTreeClassifier-based feature selection and missing-value imputation, the authors report near-perfect separation: 99.78% accuracy for SVM on a 25% split and 100% accuracy for RF under 10-fold CV, with AUC=1.00. They also perform hierarchical clustering and DBSCAN. The central claim is that admission-time features (demographics, first labs, preexisting conditions) can almost perfectly classify deceased versus non-deceased COVID-19 patients.

Significance. If the central claim were credible, this would be a striking result. The paper has some strengths: it reports hyperparameters and pseudocode, describes the data extraction pipeline, and the authors openly acknowledge possible overfitting in the Discussion. However, the claim is not supported. The Condition table includes diagnosis records from any service date after 1/1/2015, so the binary predictors—severe sepsis with septic shock, cardiac arrest, acute kidney failure—can be recorded during the hospitalization and are partly determined by the outcome. Feature selection on the full dataset and full-data imputation compound the bias, and the SVM result is not from 10-fold CV. Because the leakage is in the data construction itself, the reported accuracies cannot be interpreted as evidence of generalizable predictive performance.

major comments (4)
  1. [Dataset Explanation and Analysis, Table 4] The Condition table is the paper's core predictor source for comorbidities, yet the paper's own description (Table 4) says it includes 'all available diagnosis records from encounters with service dates on or after 1/1/2015,' with no present-on-admission flag and no restriction to diagnoses recorded before admission. The top binary features in the feature-importance list (Table 4: 'Acute kidney failure, unspecified,' 'Severe sepsis with septic shock,' 'Cardiac arrest cause unspecified,' 'Acute respiratory distress syndrome,' 'Metabolic encephalopathy') are conditions that typically arise during a fatal hospital course. Because the outcome is death during the stay, these codes are partially determined by the outcome, so the near-perfect accuracy in Tables 8 and 9 is most plausibly a target-leakage artifact rather than evidence of genuine predictive signal. This issue is load-bearing: the paper's central claim is that admission-time features can separate survivors from decedents at ~100% accuracy, but the features are not constrained to be admission-time.
  2. [Algorithms and Hyperparameters (feature selection)] The ExtraTreeClassifier feature selection is performed on the full 9,366-patient dataset before any train/test split. Using the entire labeled dataset to choose the top 20 features means test-fold information enters the feature set, and the subsequent 10-fold CV accuracies in Table 9 and the single-split result in Table 8 are optimistically biased. Feature selection must be nested inside each CV fold (or at least performed on the training folds only) to yield unbiased estimates.
  3. [Experimental Results, Tables 8 and 9] The SVM confusion matrix in Table 8 is computed on a single 25% test split, while the abstract and Section 1 (step 8) claim 10-fold cross-validation for SVM. The text explicitly says only the RF classifier used 10-fold CV. No per-fold accuracies, standard deviations, or confidence intervals are reported for either classifier, and the ROC in Figure 7 is only the RF curve with AUC=1.00. Without repeated CV or a proper held-out set, the 99.78% and 100% accuracy figures are not statistically grounded.
  4. [Dataset Explanation and Analysis, step 5 (Algorithm 1)] Missing lab values are imputed by sampling from a normal distribution of the z-score of non-missing values, and this imputation is performed before splitting into training and test folds. The imputation uses the full-data distribution, so the test rows influence the imputed values used in training, another source of leakage that inflates the reported accuracy. Imputation should be fit on training folds only and applied to test folds.
minor comments (6)
  1. [Throughout] There are two figures labelled 'Figure 3' (dendrogram and feature-importance bar chart) and the text refers to 'Tables 99 & 99' in the appendix, which do not exist; renumbering and cross-references are needed.
  2. [Abstract and Table 8] The abstract claims a 10-fold validation procedure was used for both classifiers, but the SVM confusion matrix in Table 8 is explicitly for 25% of samples; the abstract and the methods section should agree.
  3. [Figure 2] Figure 2 is described as showing the 14 most important numeric features, while Table 4 lists 20 features and the text discusses 20; the mismatch should be reconciled.
  4. [Hyperparameters table] The hyperparameter table includes a 'Random Forest Regressor' block although the task is classification; this is confusing and should be removed or clearly separated from the classifier settings.
  5. [References] Most references are web tutorials and blog posts (e.g., Ref. 1 is an outlier-detection tutorial, Ref. 6 is a Medium post on cross-validation) rather than peer-reviewed sources; the paper would benefit from citing the primary machine learning and medical literature.
  6. [Language and typos] There are numerous typos and repeated words, e.g., 'that that that' in the ROC paragraph and 'Deseased' in the Figure 1 caption; a careful proofreading pass is needed.

Circularity Check

2 steps flagged · score 6.0 of 10

Near-perfect accuracy is driven by target leakage: Condition-table codes such as severe sepsis, cardiac arrest, and acute kidney failure are recorded in the same hospitalization whose outcome is the label, so the inputs encode the outcome.

  1. self definitional [Dataset Explanation and Analysis, Table 4 (Condition); Experimental Results, Table 4 (feature ranking)]
    "Table 4) Condition - For each qualifying patient, includes all available diagnosis records from encounters with service dates on or after 1/1/2015. ... From Table 4 and Figure 3, out of the top 20 most important features of 227, we see that the five most important features which classify whether a patient will die from a COVID-19 infection in the dataset are Oxygen saturation in Arterial blood by pulse oximetry, Erythrocyte count in Blood, Acute kidney failure, INR, and Severe sepsis with septic shock."

    The label is death during the hospital stay, but the Condition table used as the source of 'preexisting conditions' includes every diagnosis record from encounters with service dates on or after 1/1/2015, with no present-on-admission flag and no restriction to diagnoses recorded before admission. The top-ranked binary features include 'Acute kidney failure, unspecified', 'Severe sepsis with septic shock', 'Cardiac arrest cause, unspecified', 'Acute kidney failure with tubular necrosis', 'Acute respiratory distress syndrome', and 'Metabolic encephalopathy' — all diagnoses that are typically coded during a fatal hospital course.

  2. other [Algorithms and Hyperparameters (feature selection); Experimental Results, Tables 8-9]
    "For the feature selection purpose, we used the ExtraTreeClassifier... Finally, using the ExtraTreeClassifier, we ranked all the features in the descending order based on their feature importance value. From the outcome we have selected the top 20 features as the values of the discarded features are extremely small. ... In Tables 8 and 9, we see the Confusion Matrices for SVM and RF classifier and obtain very high accuracies for both classifiers, accuracy of 99.78% and 100%, respectively."

    Feature selection is performed on the full labeled dataset before the 10-fold cross-validation, so the top 20 features are chosen using the labels of all patients, including those that later appear in validation folds. The reported cross-validation accuracy is therefore not a clean out-of-sample estimate: the validation labels have already influenced which features were retained. This does not by itself force the perfect accuracy, but it compounds the Condition-table leakage by allowing the feature-ranking step to select exactly the outcome-correlated codes that make the classes nearly perfectly separable.

full rationale

The paper's central claim — that demographics, first lab values, and preexisting conditions separate deceased from non-deceased COVID-19 patients with 99.78-100% accuracy — is not supported as an independent prediction. The most serious issue is target leakage in the Condition table: the table is stated to include all diagnosis records from encounters with service dates on or after 1/1/2015, and the highest-ranked binary features are severe sepsis with septic shock, cardiac arrest, and acute kidney failure variants, which are typically recorded during the terminal admission being predicted. Since the outcome is death during that stay, these features are partly determined by the outcome itself. The paper also performs ExtraTreeClassifier feature selection on the full labeled data before cross-validation, further allowing test-fold information to influence model construction. I do not treat the self-contained formatting or the external references as circular; there is no load-bearing self-citation. However, because the near-perfect accuracy can be explained by the input features encoding the outcome, the derivation chain is partially circular: the 'prediction' reduces, to a substantial degree, to reading the death label back out of the Condition table. Score 6 reflects that the central performance claim is compromised by construction, while some features (e.g., oxygen saturation, demographics) may still carry legitimate signal if the condition codes were removed.

Assumptions & free parameters 10 free parameters · 3 assumptions · 0 invented entities

The analysis rests on three main assumptions: diagnosis codes are admission-time predictors, missing lab values can be treated as random and imputed from a normal distribution, and feature selection on the full dataset does not leak test information. The first is violated by the data description, and the third is violated by the procedure.

free parameters (10)
  • Random Forest n_estimators = 2000
    Chosen by hand; no validation-set tuning reported.
  • Random Forest max_depth = 50
    Chosen by hand; may contribute to overfitting.
  • Random Forest min_samples_split = 10
    Chosen by hand.
  • SVM C = 0.01
    User-specified regularization; no tuning procedure shown.
  • SVM gamma = 1
    Kernel coefficient for RBF; chosen by hand.
  • SVM kernel = rbf
    Kernel type chosen without a disciplined cross-validated comparison.
  • DBSCAN eps = 0.3
    Selected to produce two clusters; not an independent prediction.
  • DBSCAN min_samples = 5
    Selected to yield two clusters.
  • Top feature count = 20
    Number of features kept after ExtraTree ranking; threshold chosen by inspecting importance values.
  • Lab response-rate threshold = 50%
    Attributes with more than 50% non-missing values are kept; threshold is arbitrary.
assumptions (3)
  • domain assumption Diagnosis codes in the Condition table are available at admission and represent pre-existing conditions.
    The paper states it models preexisting conditions, but the Condition table includes all diagnosis records from service dates on or after 1/1/2015, so severe sepsis, cardiac arrest, and acute kidney failure can be recorded during the hospital stay and are partly downstream of the outcome.
  • domain assumption Missing lab values are missing at random and can be imputed from a normal distribution of observed values.
    Missingness in EHR data is often informative; random normal imputation can inject artificial signal and is not validated.
  • domain assumption ExtraTree feature importance computed on the full dataset selects features that generalize.
    The feature ranking is done before cross-validation, using test-fold information; this violates the assumption that feature selection is part of the training process only.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Classification of Deceased Patients from Non-Deceased Patients using Random Forest and Support Vector Machine Classifiers." pith.science (2026). https://pith.science/paper/N7SLS7AB

@misc{pith2026241118759,
  author       = {Pith},
  title        = {Pith review of: Classification of Deceased Patients from Non-Deceased Patients using Random Forest and Support Vector Machine Classifiers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N7SLS7AB}},
  note         = {Machine review of arXiv:2411.18759}
}
read the original abstract

Analyzing large datasets and summarizing it into useful information is the heart of the data mining process. In healthcare, information can be converted into knowledge about patient historical patterns and possible future trends. During the COVID-19 pandemic, data mining COVID-19 patient information poses an opportunity to discover patterns that may signal that the patient is at high risk for death. COVID-19 patients die from sepsis, a complex disease process involving multiple organ systems. We extracted the variables physicians are most concerned about regarding viral septic infections. With the aim of distinguishing COVID-19 patients who survive their hospital stay and those COVID-19 who do not, the authors of this study utilize the Support Vector Machine (SVM) and the Random Forest (RF) classification techniques to classify patients according to their demographics, laboratory test results, and preexisting health conditions. After conducting a 10-fold validation procedure, we assessed the performance of the classification through a Receiver Operating Characteristic (ROC) curve, and a Confusion Matrix was used to determine the accuracy of the classifiers. We also performed a cluster analysis on the binary factors, such as if the patient had a preexisting condition and if sepsis was identified, and the numeric values from patient demographics and laboratory test results as predictors.

Figures

Figures reproduced from arXiv: 2411.18759 by the authors.

Figure 1
Figure 1. Four Boxplots of Numeric Attributes showing the spread of the standardized data [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Boxplots of 14 Most Important Numeric Features in classifying deceased and non-deceased COVID-19 patients of the standardized data between 0 and 1 ranked from most important to least most important, Oxygen saturation in Arterial blood by Pulse oximetry to Platelets [#/volume] in Blood by Automated count, respectively, In [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 8
Figure 8. Count of patients separated by gender and Ethnicity grouped by Deceased status [PITH_FULL_IMAGE:figures/full_fig_p029_8.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 21 canonical work pages

  1. [1]

    In healthcare, information can be converted into knowledge about patient historical patterns and possible future trends

    CS-521 Project Report Classification of Deceased Patients from Non-Deceased Patients using Random Forest and Support Vector Machine Classifiers Biraj Tiwari (tbiraj@unm.edu) Dheeman Saha (dsaha@unm.edu) Aaron Segura (AaSegura@salud.unm.edu) Abstract Analyzing large datasets and summarizing it into useful information is the heart of the data mining process...

    work page 2020

  2. [2]

    Available from: https://www.researchgate.net/figure/Proposed-pseudo-code-of the-SVM-algorithm_fig1_329793718 [accessed 6 Dec, 2020]

    A system on chip for melanoma detection using FPGA-based SVM classifier - Scientific Figure on ResearchGate. Available from: https://www.researchgate.net/figure/Proposed-pseudo-code-of the-SVM-algorithm_fig1_329793718 [accessed 6 Dec, 2020]

    work page 2020

  3. [3]

    personid: The ID of the person associated with the result result: The display name of the test or measurement numericvalue: The nsumeric value of the result

    Result - For each qualifying patient, includes all result records from encounters with service dates on or after 1/1/2015. personid: The ID of the person associated with the result result: The display name of the test or measurement numericvalue: The nsumeric value of the result. Table

    work page 2015

  4. [4]

    Condition - For each qualifying patient, includes all available diagnosis records from encounters with service dates on or after 1/1/2015. personid: The ID of the person associated with the condition codetype: The type of coding system used for recording the condition conditioncode: The code value that identifies the condition, for example, an ICD-10-CM c...

    work page 2015

  5. [5]

    Since we have data for each personid with service dates on or after 1/1/2015, we restricted to the COVID-19 positive patients as indicated by their laboratory status and the year

    work page 2015

  6. [6]

    } A distance function /;45(O! ,O&) function 73;2-2Oℎ;O-PQPR4532(S,3T4,U;.V54): 1 for ; = 1 to . 2 Q = {%$} 3 end for 4 Q = {O!,…,O

    Here, a high Silhouette score suggests that the objects are well matched to their own cluster and poorly matched to their neighborhood clusters. Value Interpretation 0.71-1.0 A strong structure has been found 0.51-0.70 A reasonable structure has been found 0.26-0.50 The structure is weak and could be artificial. Try additional methods of data analysis. <0...

    work page 2000

  7. [7]

    Available from: https://machinelearningmastery.com/model-based-outlier-detection-and-removal-in-python/[accessed 6 Dec, 2020]

    4 Automatic Outlier Detection Algorithms in Python). Available from: https://machinelearningmastery.com/model-based-outlier-detection-and-removal-in-python/[accessed 6 Dec, 2020]

    work page 2020

  8. [8]

    Available from: https://www.researchgate.net/figure/Pseudocode-for-SVM-Classifier_fig1_331875695[accessed 4 Dec, 2020]

    AN INTELLIGENT CRYPTO-LOCKER RANSOMWARE DETECTION TECHNIQUE USING SUPPORT VECTOR MACHINE CLASSIFICATION AND GREY WOLF OPTIMIZATION ALGORITHMS. Available from: https://www.researchgate.net/figure/Pseudocode-for-SVM-Classifier_fig1_331875695[accessed 4 Dec, 2020]

    work page 2020

Show all 22 references
  1. [9]

    CDC Clinical Information published 2020, August

    2020

  2. [10]

    McKinney, W., & others. (2010). Data structures for statistical computing in python. In Proceedings of the 9th Python in Science Conference (Vol. 445, pp. 51–56)

    2010

  3. [11]

    Available from: https://cse.buffalo.edu/~jing/cse601/fa13/materials/clustering_density.pdf [accessed 5 Dec, 2020]

    Clustering Lecture 4: Density-based Methods. Available from: https://cse.buffalo.edu/~jing/cse601/fa13/materials/clustering_density.pdf [accessed 5 Dec, 2020]

    2020

  4. [12]

    Cross Validation — Why & How, Available from: https://towardsdatascience.com/cross-validation-430d9a5fee22[accessed 6 Dec, 2020]

    2020

  5. [13]

    Available from: https://towardsdatascience.com/feature-selection-techniques-in-machine-learning-with-python-f24e7da3f36e[accessed 5 Dec, 2020]

    Feature Selection Techniques in Machine Learning with Python. Available from: https://towardsdatascience.com/feature-selection-techniques-in-machine-learning-with-python-f24e7da3f36e[accessed 5 Dec, 2020]

    2020

  6. [14]

    Available from: https://www.saedsayad.com/clustering_hierarchical.htm [accessed 6 Dec, 2020]

    Hierarchical Clustering. Available from: https://www.saedsayad.com/clustering_hierarchical.htm [accessed 6 Dec, 2020]

    2020

  7. [15]

    Hunter, J. D. (2007). Matplotlib: A 2D graphics environment. Computing in Science &amp; Engineering, 9(3), 90–95

    2007

  8. [16]

    S., Wendell, P., Das, T., Armbrust, M., Dave, A., … others

    Zaharia, M., Xin, R. S., Wendell, P., Das, T., Armbrust, M., Dave, A., … others. (2016). Apache spark: a unified engine for big data processing. Communications of the ACM, 59(11), 56–65. Appendix Here we have included the additional things we have performed for the data analys...

    2016

  9. [17]

    Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE). Available from: http://www.eumetrain.org/data/4/451/english/msg/ver_cont_var/uos3/uos3_ko1.htm#:~:text=The%20MAE%20measures%20the%20average,measures%20accuracy%20for%20continuous%20variables.&text=The%20MAE%20is%20a...

    2020

  10. [18]

    Oliphant, T. E. (2006). A guide to NumPy (Vol. 1). Trelgol Publishing USA

    2006

  11. [19]

    Retrieved October 11, 2020, from https://www.cdc.gov/epsis/clinicaltools/index.html

    2020

  12. [20]

    SVM pseudocode algorithm, http://pages.cs.wisc.edu/~matthewb/pages/notes/pdf/ensembles/RandomForests.pdf [accessed 5 Dec, 2020]

    2020

  13. [21]

    2825-2830, 2011[accessed 1 Dec, 2020]

    Scikit-learn: Machine Learning in Python Scikit-learn: Machine Learning in Python, Pedregosa et al., JMLR 12, pp. 2825-2830, 2011[accessed 1 Dec, 2020]

    2011

  14. [22]

    Waskom, M., Botvinnik, Olga, O&#x27;Kane, Drew, Hobson, Paul, Lukauskas, Saulius, Gemperline, David C, … Qalieh, Adel. (2017). mwaskom/seaborn: v0.8.1 (September 2017). Zenodo. https://doi.org/10.5281/zenodo.883859

    2017 doi

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.