{"id":"19ca510f-93a0-419b-8a60-fb99477f8f06","arxiv_id":"2501.01183","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"An ANN using 12 clinical features predicts ICU readmission in intracerebral hemorrhage patients with AUROC 0.899 on MIMIC data, per the authors' report.","lead":"This paper trains an artificial neural network on two intensive care databases to predict whether patients who survived a bleeding stroke will be readmitted to the ICU. The model reports an AUROC of 0.899, which the authors claim beats simpler machine learning baselines and a prior study.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported sensitivity and specificity are numerically inconsistent with the stated accuracy and class imbalance: accuracy 0.881 with sensitivity 0.893 and specificity 0.796 implies 87.6% readmission prevalence, implausible; the values appear swapped (sensitivity 0.796, specificity 0.893) for a…","rationale":"The reader's strongest claim is the ANN's superiority with specific metrics. The most load-bearing assumption in that claim is that the reported metrics are computed correctly for a well-defined readmission outcome. I found a concrete violation: the three reported metrics for the ANN are arithmetically inconsistent unless the test set is 87.6% readmitted. The paper's own description of ADASYN and Table 3 indicate readmission is the minority class, so this prevalence is not credible. Swapping sensitivity and specificity yields a consistent set for a realistic ~12% readmission rate, meaning the authors likely mislabeled the metrics or the class encoding. This does not necessarily invalidate the AUROC, but it undercuts the specific sensitivity claim and raises doubt about the entire evaluation pipeline. The reader focused on the undefined time window and ADASYN split; my calculation supplies concrete evidence that the outcome and metric definitions need verification. I therefore keep the verdict CONDITIONAL rather than escalating, because the issue can be resolved by releasing the confusion matrix and label definitions.","tokens_in":10130,"tokens_out":11880,"duration_ms":118105,"concrete_test":"Compute the test-set confusion matrix with verified label encoding (readmission=1) and report the readmission prevalence. If prevalence is ~12%, sensitivity for readmission should be ~0.796 and specificity ~0.893, confirming a swap; if prevalence is ~88%, the outcome definition is anomalous. Additionally rerun the ANN with ADASYN applied only after the train-test split and recompute AUROC to rule out data leakage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 3.2 (Table 4), the authors report for the ANN: accuracy 0.881, sensitivity 0.893, specificity 0.796. These three numbers are mutually inconsistent under the standard definitions unless the test-set readmission prevalence is (0.881-0.796)/(0.893-0.796) = 0.876 (87.6%). The paper describes readmission as the minority class (hence ADASYN), and Table 3's Group 1/0 split cannot plausibly be 87.6% readmitted. If instead sensitivity and specificity are swapped (sensitivity for readmission = 0.796, specificity = 0.893), the same accuracy follows with a realistic readmission prevalence of about 12.4%. This indicates that the reported sensitivity of 0.893 for identifying patients likely to be readmitted is not the model's true sensitivity; either the labels are reversed in evaluation or the metrics are mislabeled. Because the central claim explicitly touts sensitivity 0.893 and compares it to the prior LightGBM's 0.226, the claim as stated is unreliable without an exact confusion matrix and label-definition check.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript develops machine-learning models to predict ICU readmission in patients with intracerebral hemorrhage (ICH) using merged MIMIC-III and MIMIC-IV data. After literature review, recursive feature elimination, and expert input, 12 features are selected. An ANN with four hidden layers and ADASYN is proposed and compared with XGBoost and Random Forest, with claims of AUROC 0.899 (95% CI 0.860–0.911), accuracy 0.881, sensitivity 0.893, and specificity 0.796, outperforming the baseline models and a prior LightGBM study. The paper also reports VIF diagnostics, SHAP analysis, and a comparison with Miao et al. The central claim is that the ANN provides superior predictive performance for ICU readmission in ICH patients.","tokens_in":10378,"tokens_out":3919,"duration_ms":40338,"significance":"If the reported performance is reliable, the model could be a clinically useful, low-dimensional risk-stratification tool for ICH patients, and the comparison with prior work would represent a clear improvement in sensitivity. Strengths of the manuscript include the use of a public, well-known critical-care database, explicit inclusion/exclusion criteria, two baseline models, VIF-based multicollinearity assessment, and SHAP-based interpretability. However, the central claim is currently undermined by internal contradictions in the statistical reporting and unresolved questions about data leakage and outcome definition. The contributions are potentially valuable but require substantial revision before the performance claims can be accepted.","major_comments":[{"comment":"The text states that in the training-versus-test comparison, 'all 12 characteristics had p-values less than 0.05,' but Table 2 reports p-values ranging from 0.088 to 0.992, all above 0.05. These two statements are directly contradictory. The authors must correct either the narrative or the table and reconcile the conclusion about whether the training and test sets are representative.","section":"Section 3.1, Table 2"},{"comment":"The reported ANN metrics are numerically inconsistent under standard definitions. With accuracy = 0.881, sensitivity = 0.893, and specificity = 0.796, the identity accuracy = prevalence × sensitivity + (1 − prevalence) × specificity implies a test-set readmission prevalence of (0.881 − 0.796)/(0.893 − 0.796) = 0.876, i.e., 87.6% readmitted. This contradicts the paper's description of readmission as the minority class motivating ADASYN. If sensitivity and specificity are swapped, the implied prevalence becomes 12.4%, which is more plausible. The authors must report the confusion matrix, class counts, and an unambiguous definition of the positive class, and correct the sensitivity/specificity labels if they are reversed.","section":"Section 3.2, Table 4"},{"comment":"The manuscript does not specify whether ADASYN, KNN imputation, and RFE-based feature selection were applied before or after the 80/20 train-test split. If ADASYN synthesizes samples from the full dataset, or if imputation or feature selection uses test-set information, the test AUROC can be optimistically biased. The authors must clarify that all preprocessing and feature selection steps are fitted on the training set only, or re-run the pipeline with a strict split-before-preprocessing protocol.","section":"Section 2.6 and Section 2.4"},{"comment":"The outcome 'ICU readmission' is never operationally defined. The paper does not state the readmission time window (e.g., 30-day, 90-day, or any subsequent ICU admission within the same hospitalization or after discharge), nor how the merged MIMIC-III and MIMIC-IV cohorts handle overlapping or repeated admissions. Without an explicit outcome definition, the label is ambiguous and the reported sensitivity and comparison with Miao et al. cannot be interpreted. The authors should define the outcome precisely and report the resulting event rate.","section":"Section 2.2 and Section 3.2"}],"minor_comments":[{"comment":"The first sentence states that the study predicted 'ICU mortality in patients undergoing invasive mechanical ventilation,' which appears to be a copy-paste error from another manuscript; this manuscript is about ICU readmission in ICH patients.","section":"Section 4.1"},{"comment":"The limitation section mentions both 'Random Oversampling and ADASYN,' but Section 2.6 describes only ADASYN. The authors should clarify which sampling methods were actually used.","section":"Section 4.2"},{"comment":"In Table 5, the values '0.8930.796' appear without a separator for the ANN sensitivity and specificity; this should be corrected to '0.893, 0.796'.","section":"Section 3.2, Table 5"},{"comment":"The text describes XGBoost as having 'interoperability'; the intended word is likely 'interpretability'.","section":"Section 1"},{"comment":"The paper should report the class distribution and prevalence of the outcome in both the training and test sets, as well as 95% confidence intervals for accuracy, sensitivity, and specificity, not only for AUROC.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant clinical prediction problem and uses public data, but the current version contains a direct statistical contradiction and a metric inconsistency that affect the headline claims. The manuscript could become acceptable after the authors resolve the sensitivity/specificity issue, specify the readmission definition, and demonstrate a leakage-free preprocessing pipeline. Given that these are load-bearing but potentially fixable, I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe main result—AUROC 0.899 for ANN on ICH ICU readmission—looks suspect before you even get to external validity. Table 4 reports accuracy, sensitivity, and specificity that are mutually inconsistent. Accuracy 0.881 with sensitivity 0.893 and specificity 0.796 forces a readmission prevalence of 87.6%, which no one believes; the Random Forest row is even worse because its accuracy (0.834) lies outside the [spec, sens] interval, which is arithmetically impossible. The likely explanation is mislabeled columns, but as written the headline claim is not reproducible from the numbers given.\n\nThat's a shame, because the paper isn't worthless. It's a straightforward but sensible extension of Miao et al.'s LightGBM work to an ANN on the same public MIMIC III/IV cohort. The feature set (RFE plus two expert-added features) is described, the VIF check is a good multicollinearity sanity check, and the SHAP analysis gives a useful look at what drives predictions. Those parts are fine.\n\nThe soft spots go beyond the metrics. The readmission time window is never defined. It's unclear whether ADASYN was applied before or after the 80/20 split; if before, the test set is contaminated. Section 3.1 says all training-test t-tests were significant, while Table 2 lists p-values all above 0.05—a direct contradiction. And Section 4.1 is a leftover passage about ICU mortality prediction in mechanically ventilated patients, which has nothing to do with ICH readmission. That's an editorial error that should not have survived a single read.\n\nThe comparison with Miao et al. is also not apples-to-apples unless the cohort definitions and outcome windows match exactly, which the paper doesn't demonstrate.\n\nSo: who gets value from this? People interested in a concrete example of how feature selection and SHAP can be applied to a small clinical cohort might read it, but the core predictive claim is not yet usable. It deserves a serious referee, because the question is legitimate and the data are public. With an exact confusion matrix, a defined outcome, and a corrected Section 4.1, the paper could be salvageable. As-is, I would not cite the AUROC.\n\nMy recommendation: send it to peer review, but with a strong request that the authors provide the confusion matrix and clarify the preprocessing order before any acceptance.","headline":"The AUROC 0.899 claim is undermined by internally inconsistent sensitivity/specificity numbers and a leftover discussion passage; fixable but not acceptable as-is.","tokens_in":10934,"tokens_out":4705,"would_cite":false,"duration_ms":42414,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 12-feature artificial neural network predicts ICU readmission in intracerebral hemorrhage patients with an AUROC of 0.899 (95% CI 0.860–0.911), outperforming random forest, XGBoost, and the prior LightGBM model.","keywords":["ICU readmission","intracerebral hemorrhage","artificial neural network","machine learning","ADASYN","SHAP","MIMIC-III","MIMIC-IV"],"falsifier":"Re-run the published pipeline with ADASYN applied strictly after the train-test split, a fixed readmission window of 30 days, and the same 12 features; if the test AUROC no longer beats XGBoost's 0.870 or falls outside the reported 95% confidence interval of 0.860–0.911, the claimed superiority of the ANN is not supported.","tokens_in":9908,"feed_emoji":"🧠","tokens_out":10342,"duration_ms":90378,"temperature":0.7,"pith_summary":"Strokes from bleeding inside the brain (intracerebral hemorrhage, ICH) carry high death and complication rates, and returning to the ICU after a first stay is a serious, costly event. This paper tries to show that a compact artificial neural network can predict that return from 12 routinely available clinical values—age, hospital stay, SpO2, five electrolytes and blood counts, and three clotting measures—extracted from the MIMIC-III and MIMIC-IV critical-care databases. The authors report an area under the receiver-operating-characteristic curve (AUROC) of 0.899 (95% CI 0.860–0.911), meaning the model ranks readmitted patients above non-readmitted ones almost 90% of the time, and they claim this beats random forest (0.818), XGBoost (0.870), and the best prior model on this task (LightGBM, 0.736). The practical point the paper argues for is sensitivity: the network identifies 89.3% of patients who will be readmitted, compared with 22.6% for the prior model, at a specificity of 79.6%. If the result holds, a discharge-time risk score built from this feature set could guide closer monitoring and smarter allocation of ICU beds.","feed_headline":"Neural net IDs brain-bleed ICU returns at 0.899 AUROC","feed_subtitle":"A 12-feature model catches 89% of true readmissions, beating XGBoost and a prior LightGBM study.","key_machinery":"The load-bearing mechanism is the pairing of a 12-feature clinical panel with a four-hidden-layer feedforward neural network trained under ADASYN oversampling. The feature panel was built by recursive feature elimination on 33 candidate variables, then augmented with age and SpO2 on expert advice; the network's hidden layers (128-64-32-16) with ReLU activations let it fit nonlinear interactions among these labs and vitals, while L2 regularization and grid search control overfitting. ADASYN supplies synthetic minority-class examples in regions where readmitted patients are underrepresented, which is what the authors credit for the high recall. SHAP values then provide the attribution story: the same 12-input network doubles as an interpretability device, ranking age, chloride, MCHC, and monocytes as the main drivers of readmission risk.","core_discovery":"On its own terms, the paper's discovery is that a small artificial neural network—four hidden layers of 128, 64, 32, and 16 units with ReLU activations, a sigmoid output, L2 regularization tuned by grid search, and ADASYN oversampling to balance the minority readmission class—predicts ICU readmission in ICH patients better than the baselines it was tested against. Trained on 80% and tested on 20% of a merged cohort of 2,316 patients (871 from MIMIC-III and 1,445 from MIMIC-IV), the model achieved accuracy 0.881, sensitivity 0.893, specificity 0.796, and AUROC 0.899 (95% CI 0.860–0.911). SHAP analysis on the test set identifies age, chloride, MCHC, and monocytes as the strongest predictors, with older age, abnormal chloride, and low SpO2 pushing risk upward. The authors conclude that the model offers a simpler, more sensitive alternative to the 44-variable LightGBM model from the prior literature.","pith_inferences":["The paper never defines the time window for 'ICU readmission,' so the outcome could mix returns within the same hospitalization, returns within 30 days, and returns at any later time; standardizing to a 30-day unplanned-readmission definition would make the result comparable to most readmission literature.","A sentence in the Discussion describes the model as predicting ICU mortality in mechanically ventilated patients, which does not match the readmission outcome analyzed in the Methods and Results; the target outcome should be confirmed before the reported comparison is interpreted.","Because the gain over XGBoost is modest (0.899 vs 0.870), a plausible alternative explanation is that ADASYN and the 12-feature selection—not depth of the ANN—carry most of the performance; re-running XGBoost with the same oversampling and features would settle that.","If ADASYN was applied before the 80/20 split, as the Methods may imply, synthetic examples could appear in the test set and inflate the AUROC; checking whether any test rows are synthetic is a quick way to test the headline number."],"forward_implications":["A 12-variable panel could be turned into a discharge checklist that flags ICH patients at high readmission risk without waiting for imaging or clinical notes.","At 0.893 sensitivity, the model would catch roughly 9 of 10 true readmissions; the 0.796 specificity means about 1 in 5 flagged patients would not actually return, a trade-off many ICU triage settings could tolerate.","The reported AUROC gap over the prior LightGBM model (0.899 vs 0.736) implies that the combination of feature selection and ANN may be extracting predictive signal that the earlier 44-variable model missed.","If the model is deployed, its SHAP-identified drivers—age, chloride, MCHC, monocytes, SpO2, neutrophils, and sodium—could be monitored longitudinally, not just at admission.","The lightweight architecture and small feature set would make the model cheaper to embed in an electronic health record than models requiring dozens of variables."],"supporting_citations":[{"why":"Supplies the MIMIC-III critical-care cohort from which 871 ICH patients were extracted.","marker":"[20]"},{"why":"Supplies the MIMIC-IV cohort contributing 1,445 ICH patients to the merged training and test sets.","marker":"[21]"},{"why":"Defines the prior LightGBM benchmark (AUROC 0.736) that the paper claims to outperform.","marker":"[22]"},{"why":"Supplies clinical evidence on predictors of 30-day readmission after ICH, grounding the initial feature set.","marker":"[23]"},{"why":"Provides the recursive feature elimination approach used to reduce the initial 33 features to 10.","marker":"[27]"},{"why":"Defines ADASYN, the oversampling method the authors apply to correct class imbalance.","marker":"[33]"},{"why":"Provides the SHAP technique used to rank feature importance and interpret the ANN's predictions.","marker":"[38]"}],"fun_headline_variants":["ANN scores 0.899 AUROC for brain-bleed ICU readmissions","Small neural net beats baselines for ICH ICU readmission risk","12-feature ANN predicts ICH ICU readmission with 89% sensitivity","MIMIC-trained ANN flags ICU readmission risk in ICH patients","ICH readmission predictor: ANN outperforms LightGBM with 0.899 AUROC"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline AUROC depends on the assumption that every patient's 'ICU readmission' was labeled with the same, unspecified time window and that ADASYN oversampling was applied only to the training set; the paper does not state the readmission window or whether the sampling happened before or after the 80/20 split, so if test information leaked into training, the 0.899 figure is inflated.","fun_headline_variants_meta":{"raw":{"variants":["ANN scores 0.899 AUROC for brain-bleed ICU readmissions","Small neural net beats baselines for ICH ICU readmission risk","12-feature ANN predicts ICH ICU readmission with 89% sensitivity","MIMIC-trained ANN flags ICU readmission risk in ICH patients","ICH readmission predictor: ANN outperforms LightGBM with 0.899 AUROC"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000527,"raw_usage":{"total_tokens":2586,"prompt_tokens":1029,"completion_tokens":1557,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":645,"completion_tokens_details":{"reasoning_tokens":1455}},"tokens_in":645,"tokens_out":1557,"duration_ms":11786,"temperature":1.0,"reasoning_tokens":1455,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:32:55.677210+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the published pipeline with ADASYN applied strictly after the train-test split, a fixed readmission window of 30 days, and the same 12 features; if the test AUROC no longer beats XGBoost's 0.870 or falls outside the reported 95% confidence interval of 0.860–0.911, the claimed superiority of the ANN is not supported.","supporting_citations":[{"cited_title":"Scientific data 3(1), 1–9 (2016)","cited_arxiv_id":null,"evidence_quote":"Supplies the MIMIC-III critical-care cohort from which 871 ICH patients were extracted."},{"cited_title":"Scientific data 10(1), 1 (2023)","cited_arxiv_id":null,"evidence_quote":"Supplies the MIMIC-IV cohort contributing 1,445 ICH patients to the merged training and test sets."},{"cited_title":"Journal of the Neurological Sciences 456, 122849 (2024)","cited_arxiv_id":null,"evidence_quote":"Defines the prior LightGBM benchmark (AUROC 0.736) that the paper claims to outperform."},{"cited_title":"Critical care medicine 41(12), 2762–2769 (2013)","cited_arxiv_id":null,"evidence_quote":"Supplies clinical evidence on predictors of 30-day readmission after ICH, grounding the initial feature set."},{"cited_title":"Expert Systems 38(3), e12658 (2021)","cited_arxiv_id":null,"evidence_quote":"Provides the recursive feature elimination approach used to reduce the initial 33 features to 10."},{"cited_title":"In: 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence)","cited_arxiv_id":null,"evidence_quote":"Defines ADASYN, the oversampling method the authors apply to correct class imbalance."},{"cited_title":"Computer Methods and Programs in Biomedicine 204, 106040 (2021)","cited_arxiv_id":null,"evidence_quote":"Provides the SHAP technique used to rank feature importance and interpret the ANN's predictions."}],"review_version":1}