{"id":"f7493f00-07af-4fcf-ae7c-dfe96e4a9d4a","arxiv_id":"2508.00117","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"StackLiverNet, a stacked XGBoost/KNN/LightGBM ensemble, reports 99.89% test accuracy on the Kaggle Liver Disease Patient Dataset, with feature selection performed before the train/test split.","lead":"This paper builds a stacked machine learning model (XGBoost and k-nearest neighbors feeding a LightGBM meta-model) to classify liver disease from a public 26,000-row clinical dataset, reporting 99.89% test accuracy. The result would matter if it held up, because it suggests routine blood-test features can separate liver disease cases with almost no errors, but the evaluation pipeline has methodological flaws.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Pre-split feature selection and scaling leak test information, invalidating the reported 99.89% test accuracy as an unbiased generalization estimate.","rationale":"The reader's weakest-assumption analysis correctly identifies the pre-split application of ANOVA and RFE-CV as the load-bearing weakness. The paper's own Methods sections describe feature selection and standardization before the train/test split, which is textbook data leakage. This is not a disagreement about model-selection philosophy; it is a direct threat to the validity of the headline accuracy. The reader's verdict of REJECT is therefore well-supported. My stress-test pass found no countervailing evidence: the paper provides no code, no data, and no external validation, and the internal sample counts (26,303 vs. 23,370 implied by Table II) are unexplained. Even if the feature-selection leakage turns out to be small in practice, the reported 99.89% figure is not a trustworthy estimate, which is sufficient to reject the paper's central claim. No change to the reader's verdict is needed.","tokens_in":9344,"tokens_out":5603,"duration_ms":52977,"concrete_test":"Re-run the full StackLiverNet pipeline on the same Kaggle dataset with a strict leakage-free protocol: (1) split the data into train/test before any preprocessing; (2) fit ANOVA, RFE-CV, standardization, and the stacking ensemble exclusively on the training split; (3) apply the fitted transformations to the test split; (4) report test accuracy. If accuracy drops materially below 99.89% (e.g., by more than one percentage point), the reported number is inflated by leakage. Independently, remove exact duplicate rows before splitting and repeat the protocol; if accuracy falls sharply, duplicate leakage is also present.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is the 99.89% held-out test accuracy with only 5 misclassifications. The paper's pipeline applies one-way ANOVA (Section III-C) and RFE-CV feature selection (Section III-D) to the entire dataset before the 80/20 split (Section III-E), and standardization is performed during global preprocessing (Section III-B) prior to the split. Consequently, the test split participates in deciding which features are retained and how features are scaled, so the reported test accuracy is not an unbiased estimate of generalization. The paper nowhere states that feature selection or scaling was refit inside the training folds, and no code or data artifacts are provided to rule this out. A secondary concern is that the paper never reports the post-cleaning sample size; the 26,303-sample dataset description is hard to reconcile with the 23,370 samples implied by Table II, and no deduplication step is mentioned, so if this public Kaggle dataset contains duplicate records the split may place identical rows in both train and test. The load-bearing assumption is that the test set is information-free with respect to model construction, and that assumption is violated by the described methodology.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes StackLiverNet, a stacked ensemble (XGBoost and KNN base classifiers with a LightGBM meta-model) for binary liver disease detection on a Kaggle clinical dataset. The pipeline includes missing-value handling, Winsorization, standardization, one-way ANOVA feature filtering, RFE-CV feature selection, random undersampling, hyperparameter-tuned base models, and post-hoc interpretability with LIME, SHAP, and the Morris method. The central claim is near-perfect held-out performance: 99.89% test accuracy, Cohen's Kappa 0.9974, AUC 0.9993, and only 5 misclassifications on a test set of 4,674 samples.","tokens_in":9608,"tokens_out":3012,"duration_ms":30459,"significance":"If the reported results were valid, the contribution would be practically attractive: a fast, interpretable model that reaches essentially perfect discrimination from five clinical laboratory features, with training time around 4 seconds and inference around 0.1 seconds. The paper also integrates three XAI techniques, which is a strength for clinical acceptability. However, the central numerical claim is undermined by the described evaluation protocol: feature selection and standardization are performed on the entire dataset before the train/test split, so the reported test accuracy is not an unbiased generalization estimate. The paper provides no code or data artifacts to allow independent verification. Because the headline result is the main contribution, the validity concern is load-bearing.","major_comments":[{"comment":"The experimental pipeline leaks test information into model construction. Section III-B standardizes all numerical features, Section III-C performs one-way ANOVA filtering, and Section III-D runs RFE-CV feature selection on the full dataset, all before the 80/20 split described in Section III-E. The test split therefore contributes to deciding which five features are retained and to the values of the scaling parameters. The reported 99.89% test accuracy and the 'only 5 misclassifications' claim are consequently not unbiased estimates of generalization. The paper does not state that feature selection or standardization was refit inside the training folds, and Algorithm 1, which operates on the full X and y, makes no provision for a split. This leakage alone invalidates the paper's central empirical claim.","section":"III-B, III-C, III-D, III-E"},{"comment":"The data-set description and the class-frequency tables do not reconcile. Section III-A states that the dataset contains 26,303 samples (16,749 liver patients). Table II implies 13,414 + 5,282 = 18,696 samples in the original training set and 3,335 + 1,339 = 4,674 samples in the test set, for a total of 23,370 samples. This leaves 2,933 samples unaccounted for, and no post-cleaning sample count or deduplication step is reported. The cited Kaggle dataset is known to contain duplicate rows; if duplicates are present and not removed, identical or near-identical records may appear in both training and test partitions, further inflating the held-out accuracy. The manuscript must report the exact sample size after each preprocessing stage and explain the discrepancy.","section":"III-A and Table II"},{"comment":"The claim that StackLiverNet is 'slightly but significantly better' than the baseline models is not supported by the reported numbers or by any significance test. Table IV shows StackLiverNet test AUC 0.9993, which is lower than XGBoost's 0.9995 and essentially tied with KNN's 0.9992; the test accuracy margin over KNN is 0.0008. No confidence intervals, statistical tests, or repeated-split analyses are provided. Given the feature-selection leakage described above, the comparative superiority claim is also not valid. The authors should either provide a proper statistical comparison under a leak-free protocol or temper the claim.","section":"IV, Table IV"}],"minor_comments":[{"comment":"Algorithm 2 is incomplete: the pseudocode ends abruptly after the meta-training-set construction and does not show the actual training of the meta-model or the prediction step on the test set, and the line numbering with '0:' prefixes is nonstandard.","section":"III-G, Algorithm 2"},{"comment":"The header for the last column appears as a stray 'µ*' and is not defined; the table also reports a confidence interval column that is never explained in the text. The values for SGOT and ALB (µ* = 0.0008 and 0.0000) sit oddly with the LIME discussion in Section IV-B, which describes SGOT as an important feature; the authors should reconcile the sensitivity results with the local explanations.","section":"Table VII"},{"comment":"The caption contains typos: 'eft' should be 'left' and one panel label is truncated as 'Curve (Right)'. Please also number the subfigures consistently.","section":"Figure 3"},{"comment":"Reference formatting is inconsistent (e.g., [4] is a self-citation to a different conference paper, [3] is a preprint without a DOI, and several entries mix capitalization styles). The authors should verify that all references are relevant and complete.","section":"References"},{"comment":"There are numerous typographical and grammatical issues (e.g., 'Cohen s Kappa', 'ModelAnagnostic', 'spliting') and inconsistent use of variable names (e.g., 'Alkphos' vs. 'ALP' vs. 'Alkaline Phosphatase'). A careful proofreading pass is needed.","section":"Throughout"}],"recommendation":"reject","confidential_remarks":"The central empirical claim is invalid as reported because the evaluation protocol leaks test information through pre-split feature selection and standardization, and the dataset sample-size discrepancy in Section III-A/Table II is unresolved. Correcting these issues would require re-running the entire experimental pipeline under a nested resampling protocol, which is beyond the scope of a revision. I also note that no code or data is provided, which is a reproducibility concern for a methods paper. The interpretability analysis is a worthwhile component, but it cannot rescue the headline accuracy claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, short version: this is a competent stacking-ensemble write-up that gets its central number from a pipeline that leaks test information. The 99.89% test accuracy is not an unbiased generalization estimate because ANOVA filtering, RFE-CV feature selection, and standardization all happen before the 80/20 split. I'd treat the headline result as unvalidated.\n\nWhat's genuinely here: the exact stack (XGBoost + KNN, LightGBM meta-learner) appears to be new to this dataset, though stacking on this dataset is not new. The paper reports hyperparameters, training and inference times, and a clean description of the preprocessing steps. The LIME, SHAP, and Morris sensitivity analyses are a nice addition, and the explanations are consistent with known liver markers (bilirubin, alkaline phosphatase, transaminases). Those parts are fine.\n\nThe soft spots are load-bearing, not cosmetic. First, the leakage: Section III-C and III-D run feature significance and RFE-CV on the full dataset, and Section III-B standardizes all numeric features before the split. The test set therefore influences which features survive and how features are scaled. The paper never says these steps were refit inside cross-validation folds, and there is no code to check. This alone invalidates the reported test accuracy as an honest generalization number.\n\nSecond, the counts don't reconcile. The data collection reads 26,303 total, with 16,749 patients and 9,554 non-patients. Table II implies 18,696 + 4,674 = 23,370 after cleaning. The paper doesn't state the post-cleaning sample size, and no deduplication is mentioned — and this Kaggle dataset is known to contain duplicates. So even the split's integrity is uncertain.\n\nThird, the 'slightly but significantly better' claim for StackLiverNet over KNN and XGBoost looks overstated. The cross-validation means differ by 0.0002 and 0.0009 with overlapping standard deviations, and no statistical test is reported. Given the saturated benchmark (prior models at 99.72–99.78%), a 0.0002 improvement is noise without proper error analysis.\n\nWhat this paper would need to be salvageable: move feature selection and scaling inside the CV loop (or refit them on training folds), report the cleaned sample size and handle duplicates, and run a proper significance test. Without that, the central claim collapses.\n\nWho this is for: anyone tracking applied ML on the LDPD dataset, or teaching a graduate class on data leakage pitfalls. It is a useful negative example. As a peer-reviewed contribution, I would not send it to referees in its current form; the flaw is fundamental and the incremental value small. Recommend decline.","headline":"A textbook data-leakage error – feature selection and scaling before the split – invalidates the headline 99.89% test accuracy, so this stacking ensemble is not what it claims.","tokens_in":10113,"tokens_out":3219,"would_cite":false,"duration_ms":28037,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a five-biomarker stacked ensemble of XGBoost and k-nearest neighbors with a LightGBM meta-model detects liver disease with 99.89% held-out accuracy and per-patient explanations.","keywords":["liver disease classification","StackLiverNet","stacked ensemble learning","explainable AI","LIME","SHAP","feature selection","class imbalance"],"falsifier":"Run the exact StackLiverNet pipeline again, but refit the ANOVA threshold, the RFE-CV selector, and the standard scaler inside each cross-validation fold on the training fold only, then apply the fitted transformations to the held-out fold. If the resulting test accuracy falls well below 99.89%, the reported figure was inflated by data leakage; if it stays near 99.9%, the claim survives.","tokens_in":9192,"feed_emoji":"🩺","tokens_out":8341,"duration_ms":75487,"temperature":0.7,"pith_summary":"This paper aims to establish that liver disease can be detected almost perfectly from routine blood-test values by a stacked ensemble it calls StackLiverNet, and that the model is fast and transparent enough for clinical screening. The authors train XGBoost and k-nearest neighbors as base classifiers, feed their predictions to a LightGBM meta-model, and report 99.89% accuracy, 0.9974 Cohen's kappa, and 0.9993 AUC on a held-out test set, with only five misclassified samples. They also report that the entire pipeline trains in about 4.3 seconds and classifies a batch in about 0.11 seconds. The practical stake is that, if the estimate is unbiased, a cheap five-feature blood panel could flag likely liver disease with near-perfect accuracy while explaining each decision in terms clinicians can review.","feed_headline":"StackLiverNet hits 99.89% accuracy on liver-disease screening","feed_subtitle":"A five-biomarker blood panel and a two-classifier stack give near-perfect detection with per-patient explanations.","key_machinery":"The carrying object is StackLiverNet itself, a two-level stacking ensemble: the predictions of XGBoost and KNN on each training sample are concatenated into a new two-column feature matrix, and a LightGBM classifier is trained on that matrix to produce the final label. Around this core sits a preprocessing chain that does much of the work: mode imputation and Winsorization tame missing values and outliers, one-way ANOVA drops the two non-significant features, recursive feature elimination with cross-validation keeps five lab biomarkers, and random undersampling equalizes the training classes. The ensemble mechanism is what lets the model combine complementary decision patterns, while the preprocessing chain is what keeps the learning signal clean.","core_discovery":"In the paper's own terms, the central discovery is that a stacking ensemble built from just two diverse base learners—XGBoost and k-nearest neighbors—combined by a LightGBM meta-classifier outperforms individual gradient-boosting and neural-network baselines on the liver patient dataset. After one-way ANOVA removes Gender and Age as non-significant, recursive feature elimination with cross-validation retains five features—Total Bilirubin, Alkaline Phosphatase, Aspartate Aminotransferase, Alanine Aminotransferase, and Albumin—and random undersampling balances the training classes. The resulting StackLiverNet reaches a mean 5-fold cross-validation accuracy of 99.95% and a test accuracy of 99.89% with AUC 0.9993, with LIME, SHAP, and Morris sensitivity analysis all pointing to alkaline phosphatase, bilirubin, and the transaminases as the decision-driving biomarkers. The authors interpret these results as evidence that the model is accurate, stable, and explainable enough to support early liver-disease screening.","pith_inferences":["An inference the paper leaves implicit: the near-perfect separation on five routine lab values suggests the dataset's classes are strongly separable, so the accuracy ceiling is not evidence about harder real-world populations with similar lab panels; an independent hospital cohort is needed.","A caution implied by the reported pipeline: because feature selection and scaling appear to be applied before the single 80/20 split, the headline accuracy is best treated as an estimate that should be rechecked with selection nested inside cross-validation.","A testable extension: comparing StackLiverNet against the same five features with no feature selection at all would show how much of the gain comes from the ensemble versus the preprocessing.","A neighboring problem to watch: the same stacking recipe—two diverse base learners plus a fast gradient-boosting meta-model—transfers naturally to other tabular medical screening tasks where interpretability matters."],"forward_implications":["If StackLiverNet's estimate is unbiased, a five-biomarker blood panel—total bilirubin, alkaline phosphatase, AST, ALT, and albumin—is enough to separate liver disease from non-disease in this patient population with near-perfect accuracy.","The reported training time of 4.28 seconds and inference time of 0.11 seconds imply the full pipeline can run on modest hardware without delaying a clinical screening workflow.","The LIME and SHAP findings tie the model's decisions to known liver biomarkers, which means a clinician can audit individual predictions rather than treating the model as a black box.","Because the stacking design beats both XGBoost and KNN individually, the result suggests that complementary base learners plus a small meta-model can outperform either family alone on this task."],"supporting_citations":[{"why":"Supplies the 26,303-sample liver patient dataset used for all training, testing, and explanation experiments.","marker":"[11]"},{"why":"Supplies the one-way ANOVA procedure used to test each feature against the target and to drop Gender and Age.","marker":"[12]"},{"why":"Supplies the recursive feature elimination with cross-validation method that selects the five retained biomarkers.","marker":"[13]"},{"why":"Provides the gradient-boosting baseline at 98.80% accuracy that StackLiverNet is compared against on the same dataset.","marker":"[5]"},{"why":"Provides the stacked random forest with SHAP and LIME result at 99.72% that is the closest interpretable-ensemble comparison.","marker":"[7]"},{"why":"Provides the Explainable Boosting Machine comparison at roughly 99.78% against which StackLiverNet's 99.89% is measured.","marker":"[9]"}],"fun_headline_variants":["Stacked model hits 99.89% accuracy in liver screening","Liver disease detection: 99.89% with interpretable stack","StackLiverNet: near-perfect liver diagnosis with explainability","Interpretable ensemble achieves 99.89% on liver disease"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline accuracy depends on the assumption that every preprocessing step that touches the target variable—especially the ANOVA filter and RFE-CV feature selection—was fitted only on the training part and never saw the test split before the final evaluation.","fun_headline_variants_meta":{"raw":{"variants":["Stacked model hits 99.89% accuracy in liver screening","Liver disease detection: 99.89% with interpretable stack","StackLiverNet: near-perfect liver diagnosis with explainability","Interpretable ensemble achieves 99.89% on liver disease"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000599,"raw_usage":{"total_tokens":2851,"prompt_tokens":1048,"completion_tokens":1803,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":664,"completion_tokens_details":{"reasoning_tokens":1730}},"tokens_in":664,"tokens_out":1803,"duration_ms":12064,"temperature":1.0,"reasoning_tokens":1730,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T10:21:12.450960+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the exact StackLiverNet pipeline again, but refit the ANOVA threshold, the RFE-CV selector, and the standard scaler inside each cross-validation fold on the training fold only, then apply the fitted transformations to the held-out fold. If the resulting test accuracy falls well below 99.89%, the reported figure was inflated by data leakage; if it stays near 99.9%, the claim survives.","supporting_citations":[{"cited_title":"Liver disease patient dataset,","cited_arxiv_id":null,"evidence_quote":"Supplies the 26,303-sample liver patient dataset used for all training, testing, and explanation experiments."},{"cited_title":"Analysis of variance (anova),","cited_arxiv_id":null,"evidence_quote":"Supplies the one-way ANOVA procedure used to test each feature against the target and to drop Gender and Age."},{"cited_title":"Recursive feature elimination with cross - validation with decision tree: Feature selection method for machine learning-based intrusion detection systems,","cited_arxiv_id":null,"evidence_quote":"Supplies the recursive feature elimination with cross-validation method that selects the five retained biomarkers."},{"cited_title":"Improved liver disease prediction from clinical data through an evaluation of ensemble learning approaches,","cited_arxiv_id":null,"evidence_quote":"Provides the gradient-boosting baseline at 98.80% accuracy that StackLiverNet is compared against on the same dataset."},{"cited_title":"Explainability enhanced liver disease diagnosis technique using tree selection and stacking ensemble -based random forest model,","cited_arxiv_id":null,"evidence_quote":"Provides the stacked random forest with SHAP and LIME result at 99.72% that is the closest interpretable-ensemble comparison."},{"cited_title":"A comparative study of machine learning algorithms using explainable artificial intelligence system for predicting liver disease,","cited_arxiv_id":null,"evidence_quote":"Provides the Explainable Boosting Machine comparison at roughly 99.78% against which StackLiverNet's 99.89% is measured."}],"review_version":1}