REVIEW 3 major objections 7 minor 29 references
The efficacy of various machine learning models for multi-class classification of RNA-seq expression data
T0 review · 3 major / 7 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Standard ensemble classifiers reach about 99.9% accuracy in identifying 17 cancer types from RNA-seq expression profiles, and a 20-gene panel keeps accuracy above 95%.
desk verdict A useful but methodologically loose benchmark of five classifiers on 17 cancer types; near-perfect accuracy is plausible, but the single split and unclear train/test separation keep me from trusting the headline number. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The engine of the result is the ensemble-of-decision-trees family: random forest grows many deep trees on bootstrap samples and averages their votes; gradient boosting adds weak learners sequentially to correct residual errors; random ferns uses a constrained tree ensemble where splits are fixed and features are randomly chosen per fern. These methods are suited to data with tens of thousands of features and comparatively few samples because they can pick out informative gene combinations while ignoring noise. The other load-bearing component is feature selection: after training, the important variables from random forest and gradient boosting are combined to form ranked gene lists of size 80 down to 10, and models are retrained on those subsets.
What would settle it
Re-run the pipeline with feature selection and z-score normalisation performed separately inside each cross-validation fold, and check whether any sample identifier appears in both training and test splits; also shuffle the batch labels and see if the top genes still separate the 17 classes. If accuracy collapses or duplicates are found, the near-100% figure is an artefact of leakage rather than biological signal.
Extended reading notes
Core claim
The central claim is that tumour type can be read from gene-expression levels with near-perfect reliability using ensemble decision-tree methods. Trained on 4,221 samples (75%) and tested on 1,408 withheld samples (25%), the random forest classifies 17 cancer types with 99.89% accuracy and the gradient boosting machine with 99.68%; both ensembles classify 14 of the 17 classes with 100% accuracy, and those classes span most of the major tissue sites in the study. The paper attributes the gap between ensembles and simpler models to the high-dimensional noise of RNA-seq data, where the number of genes far exceeds the number of samples. A follow-up feature-reduction experiment, using the top genes identified jointly by random forest and gradient boosting, shows that 20 genes are enough to keep accuracy above 95% for the ensembles, with defensin, keratin-associated, interferon, actin, ribonuclease, and olfactory-receptor genes among the most influential.
Load-bearing premise
The claim of near-perfect accuracy depends on the 1,408 held-out test samples being truly independent of the 4,221 training samples, with no duplicate tumour samples, no technical batch that correlates with cancer type, and no gene-selection or normalisation step that used the test data.
Editorial extensions
If this is right
- RNA-seq expression profiles contain enough tissue-specific signal that a decision-tree ensemble can separate 17 cancer types on held-out samples with near-perfect accuracy.
- A roughly 20-gene expression panel preserves most of the classification performance, pointing toward cheaper diagnostic assays than whole-transcriptome sequencing.
- On high-dimensional expression data, support vector machines and k-nearest neighbours need feature selection to be competitive; without it their accuracy is 47.18% and 75.43%.
- Because the dataset has no healthy samples, the models can only distinguish cancer type from other cancer types, not detect cancer against normal tissue.
- The near-100% class-level results imply that the genetic mutations driving these tumour types produce expression signatures consistent enough across samples to be learned by a single model.
Reading between the lines
- If the result survives a stricter protocol where feature selection and normalisation are computed only inside training folds, a clinically usable minimal gene panel could be developed and tested on independent cohorts.
- The appearance of olfactory-receptor genes in the top-20 list, whose link to cancer is disputed in the literature, suggests the feature ranking may capture tissue-specific expression patterns that are not all causally tied to carcinogenesis; that distinction matters for biomarker interpretation.
- A direct extension would be to test the same ensembles on samples from external hospitals or different sequencing platforms; if accuracy drops, batch effects rather than biology are driving part of the signal.
- Comparing the 20-gene panel against random 20-gene panels would quantify how much of the accuracy comes from the specific selected genes versus the mere reduction in dimensionality.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper compares five machine learning models (random forest, gradient boosting machine, random ferns, support vector machine, and k-nearest neighbors) for multi-class classification of 17 cancer types from RNA-seq expression data obtained from COSMIC/TCGA. Using a single 75/25 train-test split of 5,629 samples and the full gene expression matrix, the authors report near-perfect accuracy for the ensemble methods (RF 99.89%, GBM 99.68%) and substantially lower accuracy for SVM (47.18%) and KNN (75.43%). They also perform feature reduction based on ensemble variable importance and report that a 20-gene panel maintains ensemble accuracy above 95%. The paper concludes that ensemble algorithms, particularly RF and GBM, are well suited to this classification task.
Significance. If the reported accuracies are obtained from a genuinely independent test set, the paper provides a useful data point that RNA-seq expression profiles contain strong cancer-type-specific signals that standard ensemble classifiers can exploit, and it identifies a compact gene panel worthy of further investigation. The study is commendable for comparing five algorithms on a single large dataset with per-class breakdowns, and the feature-selection analysis is a constructive step toward clinical translation. However, the current manuscript does not demonstrate the independence of the test set, and the feature-selection protocol is underspecified, so the headline accuracies are not yet fully supported.
major comments (3)
- [Section 3, 'tested with 1,408 samples' and Table 2] The manuscript does not describe how the 75/25 split was performed; if the random split was applied at the row level without checking for duplicate or multiple samples from the same TCGA patient, near-identical expression profiles can appear in both training and test sets, which would make the reported 99.89% and 99.68% accuracies reflect sample identity rather than biological signal. Please state the splitting procedure, whether any de-duplication by patient or sample identifier was performed, and if necessary, redo the evaluation with a patient-level split.
- [Section 3, 'features were reduced based on a combination of the important variables selected by both ensembles' and…] Variable importance for feature selection must come from the training set only, but the paper does not state this. If the importance scores were computed using the full 5,629 samples or the ensembles were refit on the whole dataset before selection, the reduced-feature results (including the 20-gene >95% accuracy) are optimistically biased. Please specify the exact timing of feature selection relative to the split, or repeat the procedure within each training fold.
- [Sections 2-3 and the abstract] The feature count is stated as 18,019 in Section 2, 18,015 in the abstract and Section 3, and 16,718 after filtering in Section 3, and the text says the models were tested 'with the full 18,015 genes' despite the earlier filtering to 16,718. This inconsistency prevents the reader from knowing what feature matrix was actually used for training and testing; please reconcile these numbers and state the exact input dimensions for each experiment.
minor comments (7)
- [Table 2 and the abstract] The abstract says '100% accuracy in the classification of 14 out of 17 types of cancer,' but the text lists 15 classes (including Lung Squamous cell Carcinoma) that were classified with 100% accuracy by one of the ensemble algorithms; please verify and correct the count.
- [Conclusion] The conclusion reports KNN accuracy as 75.13% while Table 2 shows an average of 75.43%; please reconcile the discrepancy.
- [Table 2] In the Lung Squamous cell Carcinoma row, the KNN accuracy value 76.03 appears twice; please remove the duplicate.
- [Section 2, reference [11]] Reference [11] is cited for the statement that TCGA expression data are z-score normalized, but reference [11] is a random-forest gene-selection paper; please cite the appropriate TCGA documentation instead.
- [Figure 4] Figure 4 lacks a legend and has minimal axis labels; please label the model lines and provide clear axis titles so the reader can distinguish the five algorithms.
- [Section 2, paragraph on RF] The sentence 'RF builds classification trees are using a bootstrap sample of the dataset' contains a grammatical error; please proofread the manuscript.
- [Section 2, paragraph on RF] The statement 'This ensemble is not prone to over-fitting since splitting points are randomly chosen' is an oversimplification; consider rephrasing to reflect that random feature selection reduces variance but does not eliminate overfitting risk.
Circularity Check
No circularity: the reported accuracies are direct held-out measurements, not quantities derived from fitted inputs or self-citations.
full rationale
The paper's central claims are empirical measurements. Table 2 reports the test-set accuracy of each algorithm after training on 75% of the COSMIC expression matrix and testing on the remaining 25%; these accuracies are observed counts of correct classifications, not values obtained by substitution into an equation or by renaming a fitted parameter. The reduced-feature experiments select genes by variable importance from the trained RF and GBM ensembles and then retrain on those genes, but the paper does not define the predicted accuracy as equal to the importance ranking; any bias from using the full dataset for feature selection would be data leakage (a correctness/validation risk), not a circular derivation in which the output is identical to the input by construction. No load-bearing premise is justified by a self-citation: the references are external algorithm descriptions (Breiman, Friedman, et al.) and domain studies, and none of the paper's conclusions reduce to an author's own prior result. The batch-bias rebuttal is weak scientifically, but weakness of evidence is not circularity. Therefore the circularity score is 0.
Assumptions & free parameters
free parameters (2)
- Model hyperparameters (RF, GBM, RFERN, SVM, KNN) =
not reported
- Feature subset sizes (80, 60, 40, 20, 10 genes) =
null
assumptions (4)
- domain assumption Cancer-type labels in the COSMIC/TCGA sample metadata are accurate and are the appropriate ground truth.
- domain assumption Samples in the COSMIC v80 gene expression dataset are independent and no sample appears in both training and test splits.
- domain assumption z-score normalized expression values from TCGA are comparable across cancer types and do not encode batch or technical artifacts that leak labels.
- ad hoc to paper The top-gene ranking used for feature reduction was computed without using the test set.
Cite this review
Pith. "Pith review of The efficacy of various machine learning models for multi-class classification of RNA-seq expression data." pith.science (2026). https://pith.science/paper/XICJ5DNR
@misc{pith2026190806817,
author = {Pith},
title = {Pith review of: The efficacy of various machine learning models for multi-class classification of RNA-seq expression data},
year = {2026},
howpublished = {\url{https://pith.science/paper/XICJ5DNR}},
note = {Machine review of arXiv:1908.06817}
}
read the original abstract
Late diagnosis and high costs are key factors that negatively impact the care of cancer patients worldwide. Although the availability of biological markers for the diagnosis of cancer type is increasing, costs and reliability of tests currently present a barrier to the adoption of their routine use. There is a pressing need for accurate methods that enable early diagnosis and cover a broad range of cancers. The use of machine learning and RNA-seq expression analysis has shown promise in the classification of cancer type. However, research is inconclusive about which type of machine learning models are optimal. The suitability of five algorithms were assessed for the classification of 17 different cancer types. Each algorithm was fine-tuned and trained on the full array of 18,015 genes per sample, for 4,221 samples (75 % of the dataset). They were then tested with 1,408 samples (25 % of the dataset) for which cancer types were withheld to determine the accuracy of prediction. The results show that ensemble algorithms achieve 100% accuracy in the classification of 14 out of 17 types of cancer. The clustering and classification models, while faster than the ensembles, performed poorly due to the high level of noise in the dataset. When the features were reduced to a list of 20 genes, the ensemble algorithms maintained an accuracy above 95% as opposed to the clustering and classification models.
Figures
Reference graph
Works this paper leans on
-
[1]
Computer methods and programs in biomedicine 146, 11-24 (2017)
Al-Rajab, M., Lu, J., Xu, Q.: Examining applying high performance genetic data feature selection and classification algorithms for colon cancer diagnosis. Computer methods and programs in biomedicine 146, 11-24 (2017)
work page 2017
-
[2]
Bioinformatics 21, 631-643 (2005)
Statnikov, A., Aliferis, C.F., Tsamardinos, I., Hardin, D., Levy, S.: A comprehensive evaluation of multicategory classification methods for microarray gene expression cancer diagnosis. Bioinformatics 21, 631-643 (2005)
work page 2005
-
[3]
Computer methods and programs in biomedicine 113, 465-473 (2014)
Azar, A.T., Elshazly, H.I., Hassanien, A.E., Elkorany, A.M.: A random forest classifier for lymph diseases. Computer methods and programs in biomedicine 113, 465-473 (2014)
work page 2014
-
[4]
The Journal of urology 195, 493-498 (2016)
Bartsch, G., Mitra, A.P., Mitra, S.A., Almal, A.A., Ste ven, K.E., Skinner, D.G., Fry, D.W., Lenehan, P.F., Worzel, W.P., Cote, R.J.: Use of artificial intelligence and machine learning algorithms with gene expression profiling to predict recurrent nonmuscle invasive urothelial carcinoma of the bladder. The Journal of urology 195, 493-498 (2016)
work page 2016
-
[5]
Machine Learning 45, 5-32 (2001)
Breiman, L.: Random Forests. Machine Learning 45, 5-32 (2001)
work page 2001
-
[6]
CRC press (1984)
Breiman, L., Friedman, J., Stone, C.J., Olshen, R.A.: Classification and regression trees. CRC press (1984)
1984
-
[7]
Human molecular genetics 23, 5866-5878 (2014)
Ezkurdia, I., Juan, D., Rodriguez, J.M., Frankish, A., Diekhans, M., Harrow, J., Vazquez, J., Valencia, A., Tress, M.L.: Multiple evidence strands suggest that there may be as few as 19 000 human protein-coding genes. Human molecular genetics 23, 5866-5878 (2014)
work page 2014
-
[8]
Nature genetics 45, 1113 (2013)
Weinstein, J.N., Collisson, E.A., Mills, G.B., Shaw, K.R.M., Ozenberger, B.A., Ellrott, K., Shmulevich, I., Sander, C., Stuart, J.M., Network, C.G.A.R.: The cancer genome atlas pan - cancer analysis project. Nature genetics 45, 1113 (2013)
work page 2013
Show all 29 references
-
[9]
Asian Pacific Journal of Cancer Prevention 17, 835-838 (2016)
Podolsky, M.D., Barchuk, A.A., Kuznetcov, V.I ., Gusarova, N.F., Gaidukov, V.S., Tarakanov, S.A.: Evaluation of machine learning algorithm utilization for lung cancer classification based on gene expression levels. Asian Pacific Journal of Cancer Prevention 17, 835-838 (2016)
2016
-
[10]
Egyptian Informatics Journal 18, 151-159 (2017)
Tarek, S., Elwahab, R .A., Shoman, M.: Gene expression based cancer classification. Egyptian Informatics Journal 18, 151-159 (2017)
2017
-
[11]
BMC bioinformatics 7, 3 (2006)
Díaz-Uriarte, R., De Andres, S.A.: Gene selection and classification of microarray data using random forest. BMC bioinformatics 7, 3 (2006)
2006
-
[12]
Computational Biology and Chemistry 28, 235-243 (2004)
Tan, Y., Shi, L., Tong, W., Hwang, G.G., Wang, C.: Multi -class tumor classification by discriminant partial least squares using microarray gene expression data and assessment of classification models. Computational Biology and Chemistry 28, 235-243 (2004)
2004
-
[13]
R: A language and environment for statistical computing
Team, R.C.: R Development Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria; 2014. (2014)
2014
-
[14]
Computers in biology and medicine 81, 139-147 (2017)
Khalilabad, N.D., Hassanpour, H.: Employing image processing techniques for cancer detection using microarray images. Computers in biology and medicine 81, 139-147 (2017)
2017
-
[15]
arXiv preprint arXiv:1202.1121 (2012)
Kursa, M.B.: rFerns: An Implementation of the Random Ferns Method for General - Purpose Machine Learning. arXiv preprint arXiv:1202.1121 (2012)
2012 arXiv
-
[16]
-S., He, X
Meng, J., Zhang, J., Luan, Y. -S., He, X. -Y., Li, L. -S., Zhu, Y. -F.: Parallel gene selection and dynamic ensemble pruning based on Affinity Propagation. Computers in Biology and Medicine 87, 8-21 (2017)
2017
-
[17]
Machine learning 63, 3-42 (2006)
Geurts, P., Ernst, D., Wehenkel, L.: Extremely randomized trees. Machine learning 63, 3-42 (2006)
2006
-
[18]
Pattern Recognition 45, 3141-3153 (2012)
Villamizar, M., Andrade -Cetto, J., Sanfeliu, A., Moreno -Noguer, F.: Bootstrapping boosted random ferns for discriminative and efficient object classification. Pattern Recognition 45, 3141-3153 (2012)
2012
-
[19]
, Wang, Z., Ding, W.: Support vector machine classifier for prediction of the metastasis of colorectal cancer
Zhi, J., Sun, J. , Wang, Z., Ding, W.: Support vector machine classifier for prediction of the metastasis of colorectal cancer. International journal of molecular medicine 41, 1419-1426 (2018)
2018
-
[20]
PloS one 12, e0189875 (2017)
Perez-Riverol, Y., Kuhn, M., Vizcaíno, J.A., Hitz, M.-P., Audain, E.: Accurate and fast feature selection workflow for high-dimensional omics data. PloS one 12, e0189875 (2017)
2017
-
[21]
Optics & Laser Technology 102, 233-239 (2018)
Li, X., Yang, S., Fan, R., Yu, X., Chen, D.: Discrimination of soft tissues using laser - induced breakdown spectroscopy in combination with k nearest ne ighbors (kNN) and support vector machine (SVM) classifiers. Optics & Laser Technology 102, 233-239 (2018)
2018
-
[22]
Scientific reports 4, 4184 (2014)
Shang, Y., Bouffanais, R.: Influence of the number of topologically interacting neighbors on swarm dynamics. Scientific reports 4, 4184 (2014)
2014
-
[23]
Springer series in statistics Springer, Berlin (2001)
Friedman, J., Hastie, T., Tibshirani, R.: The elements of statistical learning. Springer series in statistics Springer, Berlin (2001)
2001
-
[24]
IEEE Transactions on Information Theory 13, 21-27 (1967)
Cover, T., Hart, P.: Nearest neighbor pattern classification. IEEE Transactions on Information Theory 13, 21-27 (1967)
1967
-
[25]
Cl inical Cancer Research (2018)
Ye, Z., Dong, H., Li, Y., Ma, T., Huang, H., Leong, H.S., Eckel -Passow, J., Kocher, J.-P.A., Liang, H., Wang, L.: Prevalent Homozygous Deletions of Type I Interferon and Defensin Genes in Human Cancers Associate with Immunotherapy Resistance. Cl inical Cancer Research (2018)
2018
-
[26]
Cancer research canres
Rhee, H., Kim, H.-Y., Choi, J.-H., Woo, H.G., Yoo, J.E., Nahm, J.H., Choi, J.S., Park, Y.N.: Keratin 19 expression in hepatocellular carcinoma is regulated by fibroblast-derived HGF via a MET-ERK1/2-AP1 and SP1 axis. Cancer research canres. 0988.2017 (2018)
2018
-
[27]
British journal of haematology (2018)
Bram Ednersson, S., Stenson, M., Stern, M., Enblad, G., Fagman, H., Nilsson‐Ehle, H., Hasselblom, S., Andersson, P.O.: Expression of ribosomal and actin network proteins and immunochemotherapy resistance in diffuse l arge B cell lymphoma patients. British journal of haematology (2018)
2018
-
[28]
PloS one 9, e85110 (2014)
Sanz, G., Leray, I., Dewaele, A., Sobilo, J., Lerondel, S., Bouet, S., Grébert, D., Monnerie, R., Pajot-Augy, E., Mir, L.M.: Promotion of cancer cell invasiveness and metastasis emergence caused by olfactory receptor stimulation. PloS one 9, e85110 (2014)
2014
-
[29]
Nature 499, 214 (2013)
Lawrence, M.S., Stojanov, P., Polak, P., Kryukov, G.V., Cibulskis, K., Sivachenko, A., Carter, S.L., Stewart, C., Mermel, C.H., Roberts, S.A.: Mutational heterogeneity in cancer and the search for new cancer-associated genes. Nature 499, 214 (2013)
2013
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.