{"id":"c0bd7489-bfe6-4318-b77c-0229a7aff5ac","arxiv_id":"2411.13326","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A GA-MLP hybrid is claimed to classify colon microarray samples with 99.87% accuracy using two genes, but the stated test protocol makes that figure impossible.","lead":"This paper combines a genetic algorithm for gene selection with a neural network to classify colon cancer versus normal tissue from microarray data, claiming 99.87% accuracy using only two genes. The reported evaluation is internally inconsistent and lacks the validation detail needed to support the claim.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported 99.87% accuracy is arithmetically impossible under the stated 90/10 split of the 62-sample colon dataset.","rationale":"The reader's verdict of REJECT is well-founded. The most load-bearing flaw is the arithmetic impossibility of the headline accuracy. Beyond this, the protocol tunes the MLP hidden-layer size (3 to 15) by comparing test accuracy, which is a form of test-set leakage; but the core claim fails regardless. There are no machine-checked proofs or reproducible code to offset this. The paper also contains text artifacts (e.g., Table 1 referencing Ling-Spam dataset) that suggest sloppy presentation, but the internal inconsistency of the reported number is decisive. My stress-test does not change the verdict.","tokens_in":3814,"tokens_out":4922,"duration_ms":48423,"concrete_test":"Brute-force the stated protocol: for a = 0..20 runs with 6 test samples (and 20−a runs with 7), compute every possible integer correct count per run; for each combination compute pooled accuracy C/T and unweighted mean of per-run accuracies. List all attainable values and verify that 0.9987 (and its rounded versions) never appears. If the authors supply per-run test sizes and correct counts, recompute directly from those numbers.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III.B states the colon dataset has 62 samples (40 tumor, 22 normal); Section IV says 20 runs are done, with 90% of the data for training and 10% for testing each run. Thus each test set contains either 6 or 7 samples. The reported accuracy of 99.87% cannot arise from this protocol. For pooled accuracy, total test samples T = 6a + 7(20−a), and total correct C is an integer between 0 and T, so accuracy is C/T. For T from 120 to 140, 0.9987·T is never an integer, so 99.87% is unreachable. For an unweighted mean of per-run accuracies, each run's accuracy is j/6 or j/7, so the average is a rational with denominator dividing 840; 0.9987, even after rounding to four decimal places, is not of that form. The claim that the proposed classifier achieved 99.87% accuracy is therefore not supported by the described evaluation and is internally inconsistent.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a hybrid genetic-algorithm (GA) feature selector combined with a multi-layer perceptron (MLP) classifier for binary classification of microarray gene-expression data. The method is applied to the public colon cancer dataset (62 samples, 2000 genes), with the claim that it reduces the feature set to two genes and achieves 99.87% accuracy, compared to 93.55% for SVM and Naive Bayes. The manuscript contains no theoretical derivations; its contribution is entirely empirical. The evaluation, however, is described in an internally inconsistent way, and the headline accuracy figure is arithmetically impossible under the stated 90/10 split on 62 samples.","tokens_in":4005,"tokens_out":4987,"duration_ms":50421,"significance":"If the reported result were valid, a two-gene signature with near-perfect classification accuracy on a well-known colon cancer microarray dataset would be a practically interesting contribution to the feature-selection and classification literature. The paper also has the merit of attempting to combine evolutionary search with neural-network classification for high-dimensional, small-sample data. However, the central empirical claim is not supported by the described protocol: the arithmetic of the reported accuracy is incompatible with the stated train/test split, the evaluation is not independent because the hidden-layer size is chosen on the test set, and the baseline comparisons are not reproducible. As presented, the contribution is not established.","major_comments":[{"comment":"The reported accuracy of 99.87% (final results table in Section V) is arithmetically incompatible with the evaluation protocol described in Section IV. The colon dataset contains 62 samples (Section III.B), and Section IV states that 20 runs are performed with a 90/10 train/test split, so each test set has either 6 or 7 samples. If accuracy is pooled over runs, the total number of test samples is T = 6a + 7(20-a) for some integer a between 0 and 20, and the total number of correct classifications C is an integer; thus accuracy is C/T. No such fraction equals 0.9987 (the achievable values near 0.9987 are 0.9917 and 1.0000). If instead the reported accuracy is the unweighted mean of per-run accuracies, each per-run accuracy is a multiple of 1/6 or 1/7, so the mean has a denominator dividing 840, which 0.9987 does not. The central numeric claim therefore cannot arise from the stated experiment and must be re-derived or corrected.","section":"Section IV"},{"comment":"The experimental protocol is internally inconsistent. The text first says '20 runs are done' with 90% training and 10% testing, but later in the same section it states 'All these results are obtained by using 10-fold validation and 50 to 100 features'. Table 1 is said to show performances on the 'Ling-Spam dataset', whereas the paper is about the colon cancer dataset. These statements cannot all describe the experiments that produced the final comparison table, and the manuscript does not clarify which protocol, dataset, and feature counts were actually used.","section":"Section IV"},{"comment":"The evaluation procedure is not independent. Section IV says 'The nodes of hidden layer are tested from 3 to 15'; selecting the hidden-layer size that maximizes test accuracy on the same 10% test sets used to report the final accuracy means the reported accuracy is a selected maximum rather than an unbiased estimate. No nested cross-validation, separate validation set, or correction for multiple comparisons is described. Additionally, no variance, confidence intervals, or per-run accuracies are reported for the 20 runs, despite the tiny test sets (6 or 7 samples each), which make the per-run estimates extremely noisy.","section":"Section IV"},{"comment":"The comparison with SVM and Naive Bayes is not reproducible from the information given. The final results table reports SVM at 93.55% with 2 features and Naive Bayes at 93.55% with 3 features, but the manuscript does not state the feature-selection method, hyperparameters, or train/test protocol for these baselines, nor whether they used the same splits as the proposed method. The column labels 'SVM_KEGG', 'NB_KEGG', and 'MLP_GA' and the phrase 'declined concentration feature vector' in the conclusion refer to concepts not defined in the paper. Without these details, the claimed superiority is unsupported.","section":"Section V"}],"minor_comments":[{"comment":"There are multiple incomplete or malformed citations (e.g., '[]', '[..]', '[...]') and references cited as [5-9] or [6] that cannot be resolved from the reference list; the bibliography needs to be completed.","section":"Sections I and II"},{"comment":"In the definitions of the confusion-matrix terms, TN and FP are both described as 'normal tissues which are predicted as Cancerous'; TN should be normal tissues correctly predicted as normal. This typo makes the accuracy formula ambiguous.","section":"Section III.E"},{"comment":"The GA parameters (population size, crossover rate, mutation rate, and number of generations) are mentioned as important but their values are never reported; the feature-selection procedure cannot be reproduced without them.","section":"Section III.C"},{"comment":"The phrase 'declined concentration feature vector' appears in the conclusion but is never defined in the methods; either define the term or remove it.","section":"Section VI"},{"comment":"Table numbering is inconsistent: the dataset description is 'Table I', then 'Table 1' appears in Section IV, and the final comparison table is unlabeled; the text should refer to tables consistently.","section":"General"}],"recommendation":"reject","confidential_remarks":"This manuscript appears to be at an early draft stage: the experimental section is internally inconsistent, the headline accuracy is arithmetically impossible under the stated protocol, and many references are missing. The paper does not meet the standard for publication in a serious journal. I recommend rejection; the authors would need to supply a corrected, reproducible evaluation protocol and raw results to make the claims credible."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the headline result cannot be true as stated. The colon dataset has 62 samples; a 90/10 split gives test sets of 6 or 7. Per-run accuracy must be a multiple of 1/6 or 1/7, and no weighted or unweighted average of those values rounds to 99.87%. That arithmetic check alone sinks the central claim.\n\nWhat is actually new: not much. A GA for gene selection feeding an MLP is a standard hybrid, and the paper's own related work lists similar GA+SVM and GA+NN papers. Applying that to the public colon cancer dataset is a routine extension. The one sensible angle is the desire for a minimal gene panel, but that is not enough to carry the paper.\n\nThe paper does a few things well. The motivation is clearly stated, the related work is reasonably assembled, and reporting both accuracy and number of selected genes is a sensible evaluation style. But the experimental section is where it falls apart. No error bars or variance are reported for the 20 runs. The MLP hidden layer size is tested from 3 to 15, presumably on the same test fold used to report accuracy, which leaks information. The comparison with SVM and Naive Bayes is underspecified: the text contains a paragraph that appears copied from a different paper (Ling-Spam, 256 features), and the final comparison table is missing. There are also internal inconsistencies, such as the performance measurement section swapping the definitions of false positive and false negative.\n\nIn short, the paper is not publishable in its current form. The impossible accuracy number, missing baseline details, and text artifacts all point to experiments that were either not run as described or not reported carefully. There is no salvageable core beyond the generic idea, which is not new. If this crosses your desk, I would desk reject it, or at most send it back for a complete redo of the experiments with honest reporting and proper nested validation. It does not deserve serious referee time as is.","headline":"The reported 99.87% accuracy is arithmetically impossible under the paper's own 90/10 split, and the evaluation has too many unsupported details to trust any of the numbers.","tokens_in":4557,"tokens_out":2313,"would_cite":false,"duration_ms":27524,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper's GA-MLP classifier reports 99.87% accuracy on colon microarray data using two selected genes.","keywords":["Microarray classification","Genetic algorithm feature selection","Multi-layer perceptron","Colon cancer gene expression","Dimensionality reduction","Cancer diagnosis","Gene signature","Hybrid evolutionary neural network"],"falsifier":"Re-run the experiment with the MLP hidden-layer size selected inside the training folds only and evaluate a fixed two-gene signature on a held-out test set. The paper's own numbers point to a check: with 62 samples split 90/10, each test set has about six samples, so per-run accuracy is a multiple of roughly one-sixth, and no average of 20 such runs can equal 99.87%.","tokens_in":3572,"feed_emoji":"🧬","tokens_out":9192,"duration_ms":91152,"temperature":0.7,"pith_summary":"This paper proposes a hybrid classifier for DNA microarray data in which a genetic algorithm first reduces thousands of gene-expression features to two informative genes, and a multi-layer perceptron then classifies colon tissue as cancerous or normal. The authors report 99.87% accuracy on the public colon cancer dataset using only two selected features, compared with 93.55% for SVM and Naive Bayes, and conclude that the method achieves both high accuracy and the smallest number of selected genes. The claim, if correct, would mean a very small gene signature can separate cancer from normal tissue in microarray data without using the full 2,000-gene profile.","feed_headline":"Two genes put colon cancer at 99.87% accuracy","feed_subtitle":"A genetic algorithm plus a neural net reaches that number with two genes, topping SVM and Naive Bayes on the colon dataset.","key_machinery":"The load-bearing machinery is the GA-MLP pipeline. The genetic algorithm encodes candidate gene subsets as binary chromosomes (bit 1 lets a gene participate, bit 0 excludes it) and searches the 2,000-dimensional space down to two genes; the MLP then takes the two-element concentration feature vector, which the paper calls the declined concentration feature vector, as input, tries hidden-layer sizes from 3 to 15, and uses backpropagation for up to 60 epochs to an error goal of 0.01. The GA is what makes the classification tractable, and the MLP is what converts the reduced representation into a cancer/normal decision.","core_discovery":"The paper's central discovery is that a binary-encoded genetic algorithm can shrink a 2,000-gene colon cancer expression set to exactly two genes, and a three-layer MLP trained on the resulting declined concentration feature vector classifies the 62 samples into tumor and normal classes at 99.87% accuracy over 20 runs of a 90/10 split. The paper presents this result as beating SVM and Naive Bayes, both reported at 93.55%, with the same or fewer selected features.","pith_inferences":["Beyond the paper: the stated protocol cannot produce the reported number, since a 90/10 split of 62 samples yields test sets of about six samples and per-run accuracy is a multiple of one-sixth; an independent evaluation would be needed to see whether a two-gene signature really generalizes.","Beyond the paper: using the same test set to choose the MLP hidden-layer size (3 to 15) and then to report accuracy risks an optimistically biased estimate; nested cross-validation would give a more trustworthy figure.","Beyond the paper: the gene identities selected by the GA should be reported and checked for stability across runs; without that, it is unclear whether the result is a fixed diagnostic signature or a run-specific artifact.","Beyond the paper: if a fixed pair of genes were re-tested on an independent colon cancer cohort, the actual accuracy would settle whether the claim extends beyond this 62-sample dataset."],"forward_implications":["A two-gene expression signature would be enough to separate colon cancer from normal tissue, shrinking a diagnostic assay from whole-chip profiling to two measurements.","The same GA+MLP pipeline could be applied to other cancer microarray datasets as a template for dimensionality reduction and classification.","On the colon dataset, the GA-selected two-feature MLP would outperform the compared SVM and Naive Bayes classifiers by about 6 percentage points.","Using two genes instead of 2,000 reduces computational time and complexity during both training and testing."],"supporting_citations":[],"fun_headline_variants":["GA slims 2,000 genes to two for 99.87% cancer accuracy","Neural net plus GA: 99.87% colon cancer with just 2 genes","Colon cancer: 2 gene signature hits 99.87% accuracy","2 genes to 99.87%: colon cancer classification","Two-gene test achieves 99.87% accuracy in colon cancer"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the reported 99.87% accuracy is a valid out-of-sample estimate, which requires that the test data never influenced the choice of hidden-layer size or any other model decision.","fun_headline_variants_meta":{"raw":{"variants":["GA slims 2,000 genes to two for 99.87% cancer accuracy","Neural net plus GA: 99.87% colon cancer with just 2 genes","Colon cancer: 2 gene signature hits 99.87% accuracy","2 genes to 99.87%: colon cancer classification","Two-gene test achieves 99.87% accuracy in colon cancer"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001064,"raw_usage":{"total_tokens":4384,"prompt_tokens":792,"completion_tokens":3592,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":408,"completion_tokens_details":{"reasoning_tokens":3489}},"tokens_in":408,"tokens_out":3592,"duration_ms":23130,"temperature":1.0,"reasoning_tokens":3489,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:32:13.384645+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the experiment with the MLP hidden-layer size selected inside the training folds only and evaluate a fixed two-gene signature on a held-out test set. The paper's own numbers point to a check: with 62 samples split 90/10, each test set has about six samples, so per-run accuracy is a multiple of roughly one-sixth, and no average of 20 such runs can equal 99.87%.","supporting_citations":[],"review_version":1}