{"id":"e296628a-07d2-4885-b073-bd6eb8685432","arxiv_id":"2412.21156","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"A machine-learning pipeline claims 98.31% accuracy on liver disease prediction, but the result is obtained on a dataset augmented with 1000 synthetic samples and does not reflect real ILPD performance.","lead":"The paper tests a pipeline of dimensionality reduction methods (LDA, FA, t-SNE, UMAP) plus standard classifiers on the Indian Liver Patient Dataset, reporting 98.31% accuracy for Random Forest. The evaluation is invalid because the authors concatenate 1000 synthetic samples to the 583 real patient records and train and test on the combined data.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Synthetic-data concatenation and pre-split oversampling invalidate the reported accuracies as estimates of ILPD performance.","rationale":"The reader's weakest assumption identifies exactly the load-bearing issue. Even granting the algorithmic pipeline as described, the reported numbers are not measurements of classifier performance on the ILPD patient population. The concatenation of 1000 synthetic samples (Section 3.2.2) and the use of RandomOverSampler before splitting (Section 3.2.1) mean that test folds can contain synthetic rows and duplicated real rows. The confusion matrix total of 666 test instances confirms that the train-test evaluation did not use a 25% split of the original 583-row dataset. A model can achieve very high accuracy by learning the label structure of synthetically generated rows, especially when those rows are also present in training folds through duplication. No machine-checked proof, released code, or reproducible data accompanies the paper to mitigate this. The paper's own literature review records a prior 98.46% accuracy on ILPD (Kumar and Sahoo, 2013), so the claim would not be state-of-the-art even if the evaluation were valid; but the decisive problem is validity, not novelty. The proposed concrete test would settle the concern by re-running the pipeline on real ILPD data only, with the oversampler confined to training folds.","tokens_in":16341,"tokens_out":3132,"duration_ms":36638,"concrete_test":"Obtain the original 583-row ILPD file, remove all synthetic rows and any RandomOverSampler duplication from the test partition, and rerun the Random Forest pipeline with the same 75/25 stratified split and random_state=42, fitting the oversampler inside the training folds only. If the test accuracy falls materially below 95.79% (or CV below 98.31%), the headline result depends on synthetic/duplicate leakage. Optionally, request the synthetic data generator from the authors and test whether synthetic samples are statistically distinguishable from real ILPD rows using a classifier trained to separate the two sources.","verdict_should_be":"REJECT","load_bearing_attack":"Reported accuracies are not valid estimates of ILPD performance. Section 3.2.2 creates 1000 synthetic samples with undisclosed distributions and label rule; Section 3.2.3 concatenates them with the 583 real records; Section 3.2.1 applies RandomOverSampler before the evaluation splits of Section 3.5.1/3.5.2. The test set is therefore not composed of real patient records: the confusion matrix in Figure 12 sums to 666 test instances (313+329+5+19), which is consistent with a 25% split of the combined/oversampled dataset (2664 rows) and not with a 25% split of the original 583-row ILPD (~146 rows). Hence the 98.31% CV and 95.79% train-test accuracies can be inflated by classifiers exploiting synthetic label structure or duplicated rows, and they do not measure generalization to chronic liver disease patients. For the central claim to hold, the synthetic 1000 rows would have to be drawn from the same feature-label joint distribution as ILPD; the paper provides no distribution, no parameters, no code, and no data to support that equivalence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a pipeline for chronic liver disease detection on the Indian Liver Patient Dataset (ILPD) that combines linear and nonlinear dimensionality reduction techniques (LDA, FA, t-SNE, UMAP) with standard classifiers. The authors report that Random Forest achieves 98.31% accuracy in 10-fold cross-validation and 95.79% accuracy in a train-test split, which they claim outperforms prior state-of-the-art results on ILPD. The methodology includes imputation, random oversampling, generation and concatenation of 1000 synthetic samples, IQR-based outlier replacement, feature integration, and scaling.","tokens_in":16538,"tokens_out":4766,"duration_ms":45889,"significance":"If the reported results were valid, the paper would offer a useful empirical comparison of combined linear/nonlinear dimensionality reduction techniques on a widely used clinical dataset, and the proposed pipeline would merit attention. The manuscript provides mathematical formulations for the dimensionality reduction methods and describes the preprocessing steps in detail. However, the central evaluation is compromised: the test data are not drawn from the real ILPD patient distribution because 1000 synthetic samples are concatenated with the real records and random oversampling is applied before the data splits. Consequently, the reported accuracies do not measure generalization to real chronic liver disease patients. The paper would need a complete re-analysis on the original ILPD, with oversampling confined to the training folds and without synthetic concatenation, to support its claims.","major_comments":[{"comment":"The evaluation dataset is contaminated by synthetic data. Section 3.2.2 describes generating 1000 samples with 10 features (5 informative, 3 redundant, 2 repeated) and Section 3.2.3 concatenates them with the 583 real ILPD records. No generative distribution, class-balance rule, or parameters are given for this synthetic dataset, and no argument is provided that these points come from the same feature-label joint distribution as ILPD. The reported 98.31% and 95.79% accuracies in Tables 2 and 3 are therefore computed on a dataset in which roughly 63% of rows are fabricated, so they cannot be interpreted as measurements of performance on real chronic liver disease patients. This invalidates the paper's central claim.","section":"§3.2.2–3.2.3, Eqs. (1)–(2)"},{"comment":"Oversampling is applied before the train-test split and cross-validation, which creates leakage through duplicated rows that can appear in both training and test sets. The confusion matrix in Fig. 12 sums to 666 instances (313+329+5+19), which is exactly 25% of 2664 rows, matching a 25% stratified split on the combined, oversampled dataset; a 25% split of the original 583-record ILPD would contain about 146 instances. Thus the test set contains synthetic and duplicated examples, and the reported generalization measures do not reflect the original ILPD distribution. The evaluation protocol must be redone without pre-split augmentation.","section":"§3.2.1, §3.5.1–3.5.2, Fig. 12"},{"comment":"The comparative analysis is not valid evidence of state-of-the-art performance. Prior results listed in Table 4 were obtained on the original ILPD or its standard preprocessed versions, whereas the proposed method is evaluated on a dataset augmented with 1000 synthetic rows and balanced via oversampling. Because the evaluation set differs, the accuracy advantage over the cited baselines is not attributable to the proposed dimensionality-reduction pipeline. A fair comparison would require the same protocol on the same data.","section":"§4.3, Table 4"}],"minor_comments":[{"comment":"The abstract and introduction claim performance on the Indian Liver Patient Dataset but do not disclose that the evaluation included 1000 synthetic samples; this should be prominently stated.","section":"Abstract, §1"},{"comment":"The text says k (1.5 or 3) determines sensitivity for the IQR bounds but never states which value was used in the experiments, and the phrase 'second quartile values' is an unclear way to refer to the median.","section":"§3.2.4"},{"comment":"The mapping of the target variable is ambiguous: the text says 'the values 0 and 1 were mapped to represent the presence and absence of liver conditions, respectively,' but ILPD's original labels are 1 and 2; the exact mapping should be stated explicitly.","section":"§3.1"},{"comment":"Figure 9 is described in the text as a calibration curve, but the caption reads 'Learning curve for different algorithms in train-test split'; the caption and the referenced section should be consistent.","section":"§4.1.5, Fig. 9"},{"comment":"The confusion matrix discussion says the model identified '313 cases of liver disease (LD) and 329 cases of non-liver disease (NLD),' which is inconsistent with the 0/1 mapping defined in Section 3.1; clarify which class label corresponds to disease.","section":"§4.1.7"},{"comment":"Hyperparameters for t-SNE (perplexity, number of iterations) and UMAP (n_neighbors, min_dist) are not reported, even though fixed random states are mentioned; this hampers reproducibility.","section":"§3.3.3–3.3.4"},{"comment":"There is a grammatical error: '10-fold cross-validation was also used to assess the of the model' should read 'to assess the model.'","section":"§3.5.2"}],"recommendation":"reject","confidential_remarks":"The synthetic data description (5 informative, 3 redundant, 2 repeated features, binary classification) strongly resembles scikit-learn's make_classification, which would mean the generated rows are not liver-patient data at all; no tool or parameters are cited. The claim that 'data will be made available on request' is insufficient for reproducibility, and no code is released. If the authors resubmit, they should re-run all experiments on the original ILPD with oversampling confined to the training folds and disclose the complete preprocessing and hyperparameter settings."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: don't trust the headline numbers. The 98.31% CV and 95.79% train-test accuracies are not estimates of performance on real ILPD patients, because the evaluation set is contaminated. Section 3.2.2 creates 1000 synthetic rows, Section 3.2.3 concatenates them with the 583 real records, and Section 3.2.1 applies RandomOverSampler before the Section 3.5 splits. The confusion matrix in Figure 12 sums to 666 test instances, which matches a 25% split of the combined/oversampled 2664 rows, not roughly 146 from the original 583. So the test set contains synthetic rows and duplicated real rows. The synthetic generation and label assignment are never specified, so there is no basis for claiming those rows resemble the ILPD distribution. The central result falls.\n\nWhat is good: the paper is clearly written about what it does, the literature survey is broad, and it includes prior ILPD results that are comparable or better (Kumar & Sahoo at 98.46%, Sivasangari at 95.18%). It does not hide those. The idea of chaining LDA/FA with t-SNE/UMAP before classification is a routine combination, not a new technique, and the authors overstate it as novel. But the pipeline is at least described in enough detail to reproduce approximately, apart from missing hyperparameters and synthetic-data details.\n\nThe soft spots beyond the contamination: no code or data released; free parameters like FA factor count, t-SNE dimensions, UMAP components, and the IQR multiplier are only partially specified; the comparison table omits the Kumar & Sahoo result that appears in the text; and claiming \"state-of-the-art\" while citing a higher prior accuracy is sloppy. These are secondary. The synthetic-data concatenation is the load-bearing flaw. Even if the synthetic rows were drawn perfectly from the real distribution—which is not shown—pre-split oversampling alone creates train/test leakage, and the reader cannot verify anything without the code or data.\n\nThis paper is for someone who wants a catalog of ILPD results; the comparative table has some use. As a research claim, it is not worth a full referee process. I would desk reject. If the authors rerun everything on the real 583 records, oversample inside cross-validation folds, release code and data, and stop calling a routine pipeline novel, there might be a modest benchmark paper in there. As submitted, the headline accuracy is an artifact.","headline":"The reported accuracies are artifacts of synthetic-data contamination and pre-split oversampling, not measures of real ILPD performance.","tokens_in":17131,"tokens_out":2072,"would_cite":false,"duration_ms":23764,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T05","62H30"],"pacs":[],"model":"deepseek-v4-flash","headline":"The authors report that a unified stack of LDA, FA, t-SNE and UMAP followed by Random Forest achieves 98.31% accuracy on the ILPD dataset, outperforming earlier published results.","keywords":["chronic liver disease","ILPD","dimensionality reduction","LDA","factor analysis","t-SNE","UMAP","Random Forest"],"falsifier":"Re-run the pipeline on the original 583-patient ILPD without the 1,000 synthetic records; if accuracy drops from 98.31% toward the 88–95% range of prior published ILPD results, then the synthetic samples, not the reduction stack, are responsible for the claimed performance.","tokens_in":16067,"feed_emoji":"🩺","tokens_out":8316,"duration_ms":69827,"temperature":0.7,"pith_summary":"This paper proposes a single integrated pipeline for predicting chronic liver disease from the Indian Liver Patient Dataset (ILPD): replace outliers by quartile values, balance classes by random oversampling, apply four dimensionality-reduction methods in sequence (LDA, then factor analysis, then t-SNE, then UMAP), scale the result, and feed it to a classifier. The authors' central claim is that this unified reduction stack lets a plain Random Forest reach 98.31% accuracy in 10-fold cross-validation and 95.79% accuracy in a train-test split, both better than previously published results on the same dataset. The authors also report that Random Forest beats logistic regression, multi-layer perceptron, and k-nearest neighbours on all metrics in both protocols. The contribution is presented as the ordering and integration of the four reduction methods rather than a new classifier.","feed_headline":"Dimension-reduction stack hits 98.31% accuracy on liver data","feed_subtitle":"Combining LDA, FA, t-SNE and UMAP before Random Forest beats earlier ILPD benchmarks, the authors report.","key_machinery":"The load-bearing object is the sequential feature-integration pipeline: LDA ($S_w^{-1}S_b w = \\lambda w$) and factor analysis ($S = LL^T + \\Psi$) are applied first to condense the 10 raw features; their outputs are concatenated, then t-SNE (minimizing the KL divergence between high- and low-dimensional neighbor distributions) and a three-component UMAP embedding are applied, and finally StandardScaler normalizes the result. The authors' stated rationale is that removing outliers before, rather than after, dimensionality reduction protects t-SNE's local-structure preservation from extreme values, and that the linear-then-nonlinear order captures both global class structure and local neighborhood structure.","core_discovery":"The central claim is that a carefully ordered combination of linear and nonlinear dimensionality reduction — LDA and FA first, then t-SNE and UMAP, followed by standard scaling — transforms the ILPD feature space into a representation on which a plain Random Forest achieves near-perfect separation of liver-disease from non-liver-disease patients. In the authors' framing, LDA maximizes class separability, FA recovers latent factors, t-SNE preserves local neighborhoods, and UMAP preserves local and global manifold structure; together they are said to yield a unified low-dimensional view in which the classifier generalizes well. The paper reports 98.31% mean accuracy, 98.20% precision, 98.42% recall, and 98.315% F1 in 10-fold cross-validation, and 95.79% accuracy, 97.79% precision, 93.67% recall, and 99.53% AUC in the train-test split, with Random Forest the best of four classifiers in both protocols.","pith_inferences":["Because the 1,000 synthetic rows are concatenated before the train-test split, their structure is likely embedded in both training and test folds; a reader should treat the 98.31% figure as a property of the combined synthetic-plus-real population, not of the ILPD population alone.","The paper does not report the distribution, separability, or label balance of the synthetic samples; if those samples are trivially separable, they would inflate accuracy regardless of the dimensionality-reduction stack.","A direct ablation — running Random Forest on the raw scaled features without LDA/FA/t-SNE/UMAP, and on each reduction method alone — would test whether the four-stage stack is necessary or whether one nonlinear method such as UMAP suffices; the paper does not include such an ablation.","The confusion matrix reported for the train-test split contains 666 test instances, which matches a 75/25 split of the oversampled real data (2,664 rows) rather than of the combined set with synthetic rows; the paper does not clarify which rows were in the test set."],"forward_implications":["Random Forest on the unified reduction representation is the best of the four classifiers tested in both evaluation protocols.","The ordering of preprocessing matters: outlier replacement before dimensionality reduction is claimed to improve t-SNE stability and is a deliberate point of divergence from prior work.","The combined linear-then-nonlinear reduction is claimed to capture both global and local structure, which the authors expect to transfer to other high-dimensional clinical tabular datasets.","The high cross-validation and train-test accuracies are offered as evidence that the model resists overfitting and is suitable for real-world screening applications."],"supporting_citations":[{"why":"Supplies the t-SNE algorithm and cost function the pipeline applies to the concatenated LDA/FA features.","marker":"van der Maaten et al. (2008)"},{"why":"Supplies the UMAP algorithm and topological embedding step used after t-SNE.","marker":"McInnes et al. (2018)"},{"why":"Supplies the LDA scatter-matrix formulation that the pipeline uses for linear class-separability reduction.","marker":"Tharwat et al. (2017)"},{"why":"Supplies the factor-analysis covariance decomposition $S = LL^T + \\Psi$ used to extract latent factors.","marker":"Johnson & Wichern (2007)"},{"why":"Provides the strongest prior cross-validation baseline (88.10% with Random Forest) that the proposed method claims to surpass.","marker":"Amin et al. (2023)"},{"why":"Provides the prior SVM baseline (95.18% accuracy) on ILPD that the proposed method claims to exceed.","marker":"Sivasangari et al. (2020)"},{"why":"Provides the autoencoder baseline (92.1% accuracy) that the proposed method claims to outperform.","marker":"Bhupathi et al. (2022)"}],"fun_headline_variants":["DR stack hits 98.31% on chronic liver data","Unified DR + RF reaches 98.31% on liver disease","Four DR methods combine for 98.31% liver detection","98.31% liver accuracy via four-way DR and Random Forest"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported accuracies rest on the assumption that training and evaluating on a dataset that includes 1,000 synthetic patient records, generated with undisclosed distributions and labels alongside the real 583-record ILPD, measures how well the models would perform on real liver-disease patients.","fun_headline_variants_meta":{"raw":{"variants":["DR stack hits 98.31% on chronic liver data","Unified DR + RF reaches 98.31% on liver disease","Four DR methods combine for 98.31% liver detection","98.31% liver accuracy via four-way DR and Random Forest"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001492,"raw_usage":{"total_tokens":6023,"prompt_tokens":1011,"completion_tokens":5012,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":627,"completion_tokens_details":{"reasoning_tokens":4938}},"tokens_in":627,"tokens_out":5012,"duration_ms":35336,"temperature":1.0,"reasoning_tokens":4938,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:00:36.008542+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the pipeline on the original 583-patient ILPD without the 1,000 synthetic records; if accuracy drops from 98.31% toward the 88–95% range of prior published ILPD results, then the synthetic samples, not the reduction stack, are responsible for the claimed performance.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the t-SNE algorithm and cost function the pipeline applies to the concatenated LDA/FA features."},{"cited_title":"A., & Wichern, D","cited_arxiv_id":null,"evidence_quote":"Supplies the factor-analysis covariance decomposition $S = LL^T + \\Psi$ used to extract latent factors."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the prior SVM baseline (95.18% accuracy) on ILPD that the proposed method claims to exceed."},{"cited_title":"(2022) Liver Disease Detection using Machine Learning Techniques","cited_arxiv_id":null,"evidence_quote":"Provides the autoencoder baseline (92.1% accuracy) that the proposed method claims to outperform."}],"review_version":1}