Pith. sign in

REVIEW 3 major objections 7 minor 29 references

The efficacy of various machine learning models for multi-class classification of RNA-seq expression data

T0 review · 3 major / 7 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read Standard ensemble classifiers reach about 99.9% accuracy in identifying 17 cancer types from RNA-seq expression profiles, and a 20-gene panel keeps accuracy above 95%.

desk verdict A useful but methodologically loose benchmark of five classifiers on 17 cancer types; near-perfect accuracy is plausible, but the single split and unclear train/test separation keep me from trusting the headline number. read the letter →

arxiv 1908.06817 v1 pith:XICJ5DNR submitted 2019-08-19 cs.LG q-bio.GNstat.ML

classification cs.LGq-bio.GNstat.ML
keywords cancerclassificationRNA-seqgeneexpressionmulti-classrandomforestgradientboostingmachinefernsfeatureselectionensemblelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that standard ensemble classifiers, trained on whole-transcriptome RNA-seq expression profiles, can identify the type of cancer a tumour sample came from almost perfectly. On a public pan-cancer expression dataset of 5,629 samples spanning 17 cancer classes and more than 18,000 measured genes, the random forest and gradient boosting machine reach 99.89% and 99.68% accuracy on a withheld quarter of the data, with 14 of 17 classes classified at 100%. The authors further claim that reducing the features to a 20-gene panel keeps ensemble accuracy above 95%, which suggests a small, inexpensive diagnostic test might be possible. The paper also shows that support vector machines and k-nearest neighbours do far worse on the full gene set, but improve after feature selection.

What carries the argument

The engine of the result is the ensemble-of-decision-trees family: random forest grows many deep trees on bootstrap samples and averages their votes; gradient boosting adds weak learners sequentially to correct residual errors; random ferns uses a constrained tree ensemble where splits are fixed and features are randomly chosen per fern. These methods are suited to data with tens of thousands of features and comparatively few samples because they can pick out informative gene combinations while ignoring noise. The other load-bearing component is feature selection: after training, the important variables from random forest and gradient boosting are combined to form ranked gene lists of size 80 down to 10, and models are retrained on those subsets.

What would settle it

Re-run the pipeline with feature selection and z-score normalisation performed separately inside each cross-validation fold, and check whether any sample identifier appears in both training and test splits; also shuffle the batch labels and see if the top genes still separate the 17 classes. If accuracy collapses or duplicates are found, the near-100% figure is an artefact of leakage rather than biological signal.

Watch

Extended reading notes

Core claim

The central claim is that tumour type can be read from gene-expression levels with near-perfect reliability using ensemble decision-tree methods. Trained on 4,221 samples (75%) and tested on 1,408 withheld samples (25%), the random forest classifies 17 cancer types with 99.89% accuracy and the gradient boosting machine with 99.68%; both ensembles classify 14 of the 17 classes with 100% accuracy, and those classes span most of the major tissue sites in the study. The paper attributes the gap between ensembles and simpler models to the high-dimensional noise of RNA-seq data, where the number of genes far exceeds the number of samples. A follow-up feature-reduction experiment, using the top genes identified jointly by random forest and gradient boosting, shows that 20 genes are enough to keep accuracy above 95% for the ensembles, with defensin, keratin-associated, interferon, actin, ribonuclease, and olfactory-receptor genes among the most influential.

Load-bearing premise

The claim of near-perfect accuracy depends on the 1,408 held-out test samples being truly independent of the 4,221 training samples, with no duplicate tumour samples, no technical batch that correlates with cancer type, and no gene-selection or normalisation step that used the test data.

Editorial extensions

If this is right

  • RNA-seq expression profiles contain enough tissue-specific signal that a decision-tree ensemble can separate 17 cancer types on held-out samples with near-perfect accuracy.
  • A roughly 20-gene expression panel preserves most of the classification performance, pointing toward cheaper diagnostic assays than whole-transcriptome sequencing.
  • On high-dimensional expression data, support vector machines and k-nearest neighbours need feature selection to be competitive; without it their accuracy is 47.18% and 75.43%.
  • Because the dataset has no healthy samples, the models can only distinguish cancer type from other cancer types, not detect cancer against normal tissue.
  • The near-100% class-level results imply that the genetic mutations driving these tumour types produce expression signatures consistent enough across samples to be learned by a single model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the result survives a stricter protocol where feature selection and normalisation are computed only inside training folds, a clinically usable minimal gene panel could be developed and tested on independent cohorts.
  • The appearance of olfactory-receptor genes in the top-20 list, whose link to cancer is disputed in the literature, suggests the feature ranking may capture tissue-specific expression patterns that are not all causally tied to carcinogenesis; that distinction matters for biomarker interpretation.
  • A direct extension would be to test the same ensembles on samples from external hospitals or different sequencing platforms; if accuracy drops, batch effects rather than biology are driving part of the signal.
  • Comparing the 20-gene panel against random 20-gene panels would quantify how much of the accuracy comes from the specific selected genes versus the mere reduction in dimensionality.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper compares five machine learning models (random forest, gradient boosting machine, random ferns, support vector machine, and k-nearest neighbors) for multi-class classification of 17 cancer types from RNA-seq expression data obtained from COSMIC/TCGA. Using a single 75/25 train-test split of 5,629 samples and the full gene expression matrix, the authors report near-perfect accuracy for the ensemble methods (RF 99.89%, GBM 99.68%) and substantially lower accuracy for SVM (47.18%) and KNN (75.43%). They also perform feature reduction based on ensemble variable importance and report that a 20-gene panel maintains ensemble accuracy above 95%. The paper concludes that ensemble algorithms, particularly RF and GBM, are well suited to this classification task.

Significance. If the reported accuracies are obtained from a genuinely independent test set, the paper provides a useful data point that RNA-seq expression profiles contain strong cancer-type-specific signals that standard ensemble classifiers can exploit, and it identifies a compact gene panel worthy of further investigation. The study is commendable for comparing five algorithms on a single large dataset with per-class breakdowns, and the feature-selection analysis is a constructive step toward clinical translation. However, the current manuscript does not demonstrate the independence of the test set, and the feature-selection protocol is underspecified, so the headline accuracies are not yet fully supported.

major comments (3)
  1. [Section 3, 'tested with 1,408 samples' and Table 2] The manuscript does not describe how the 75/25 split was performed; if the random split was applied at the row level without checking for duplicate or multiple samples from the same TCGA patient, near-identical expression profiles can appear in both training and test sets, which would make the reported 99.89% and 99.68% accuracies reflect sample identity rather than biological signal. Please state the splitting procedure, whether any de-duplication by patient or sample identifier was performed, and if necessary, redo the evaluation with a patient-level split.
  2. [Section 3, 'features were reduced based on a combination of the important variables selected by both ensembles' and…] Variable importance for feature selection must come from the training set only, but the paper does not state this. If the importance scores were computed using the full 5,629 samples or the ensembles were refit on the whole dataset before selection, the reduced-feature results (including the 20-gene >95% accuracy) are optimistically biased. Please specify the exact timing of feature selection relative to the split, or repeat the procedure within each training fold.
  3. [Sections 2-3 and the abstract] The feature count is stated as 18,019 in Section 2, 18,015 in the abstract and Section 3, and 16,718 after filtering in Section 3, and the text says the models were tested 'with the full 18,015 genes' despite the earlier filtering to 16,718. This inconsistency prevents the reader from knowing what feature matrix was actually used for training and testing; please reconcile these numbers and state the exact input dimensions for each experiment.
minor comments (7)
  1. [Table 2 and the abstract] The abstract says '100% accuracy in the classification of 14 out of 17 types of cancer,' but the text lists 15 classes (including Lung Squamous cell Carcinoma) that were classified with 100% accuracy by one of the ensemble algorithms; please verify and correct the count.
  2. [Conclusion] The conclusion reports KNN accuracy as 75.13% while Table 2 shows an average of 75.43%; please reconcile the discrepancy.
  3. [Table 2] In the Lung Squamous cell Carcinoma row, the KNN accuracy value 76.03 appears twice; please remove the duplicate.
  4. [Section 2, reference [11]] Reference [11] is cited for the statement that TCGA expression data are z-score normalized, but reference [11] is a random-forest gene-selection paper; please cite the appropriate TCGA documentation instead.
  5. [Figure 4] Figure 4 lacks a legend and has minimal axis labels; please label the model lines and provide clear axis titles so the reader can distinguish the five algorithms.
  6. [Section 2, paragraph on RF] The sentence 'RF builds classification trees are using a bootstrap sample of the dataset' contains a grammatical error; please proofread the manuscript.
  7. [Section 2, paragraph on RF] The statement 'This ensemble is not prone to over-fitting since splitting points are randomly chosen' is an oversimplification; consider rephrasing to reflect that random feature selection reduces variance but does not eliminate overfitting risk.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the reported accuracies are direct held-out measurements, not quantities derived from fitted inputs or self-citations.

full rationale

The paper's central claims are empirical measurements. Table 2 reports the test-set accuracy of each algorithm after training on 75% of the COSMIC expression matrix and testing on the remaining 25%; these accuracies are observed counts of correct classifications, not values obtained by substitution into an equation or by renaming a fitted parameter. The reduced-feature experiments select genes by variable importance from the trained RF and GBM ensembles and then retrain on those genes, but the paper does not define the predicted accuracy as equal to the importance ranking; any bias from using the full dataset for feature selection would be data leakage (a correctness/validation risk), not a circular derivation in which the output is identical to the input by construction. No load-bearing premise is justified by a self-citation: the references are external algorithm descriptions (Breiman, Friedman, et al.) and domain studies, and none of the paper's conclusions reduce to an author's own prior result. The batch-bias rebuttal is weak scientifically, but weakness of evidence is not circularity. Therefore the circularity score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper's central claim rests on the trustworthiness of the COSMIC/TCGA labels, on sample independence, on the absence of batch leakage, and on the undocumented isolation of the feature-selection step. It introduces no new theoretical entities. The main unstated assumptions are empirical and data-related rather than mathematical.

free parameters (2)
  • Model hyperparameters (RF, GBM, RFERN, SVM, KNN) = not reported
    Each algorithm was 'fine-tuned and trained' on the training data (Section 3), but the actual hyperparameter values, tuning grid, and selection procedure are not given, so their effect on accuracy cannot be audited or reproduced.
  • Feature subset sizes (80, 60, 40, 20, 10 genes) = null
    These counts are chosen experimental levels reported in Figure 4, not values fitted to optimize a target. They are included because the 20-gene result becomes a highlighted claim, and the selection process is not fully described.
assumptions (4)
  • domain assumption Cancer-type labels in the COSMIC/TCGA sample metadata are accurate and are the appropriate ground truth.
    The entire accuracy calculation compares model predictions to these labels after combining the expression dataset with the sample metadata in Section 2.
  • domain assumption Samples in the COSMIC v80 gene expression dataset are independent and no sample appears in both training and test splits.
    The 75/25 split in Section 3 assumes independent samples; duplicate or same-patient samples across splits would inflate held-out accuracy.
  • domain assumption z-score normalized expression values from TCGA are comparable across cancer types and do not encode batch or technical artifacts that leak labels.
    Section 2 states all expression data are presented as z-scores, and the analysis does not correct for batch effects, which can produce spurious classification accuracy.
  • ad hoc to paper The top-gene ranking used for feature reduction was computed without using the test set.
    Section 3 says features were reduced 'based on a combination of the important variables selected by both ensembles' but does not state the ranking was restricted to training data. If the full dataset was used, the reduced-feature accuracies leak.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The efficacy of various machine learning models for multi-class classification of RNA-seq expression data." pith.science (2026). https://pith.science/paper/XICJ5DNR

@misc{pith2026190806817,
  author       = {Pith},
  title        = {Pith review of: The efficacy of various machine learning models for multi-class classification of RNA-seq expression data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XICJ5DNR}},
  note         = {Machine review of arXiv:1908.06817}
}
read the original abstract

Late diagnosis and high costs are key factors that negatively impact the care of cancer patients worldwide. Although the availability of biological markers for the diagnosis of cancer type is increasing, costs and reliability of tests currently present a barrier to the adoption of their routine use. There is a pressing need for accurate methods that enable early diagnosis and cover a broad range of cancers. The use of machine learning and RNA-seq expression analysis has shown promise in the classification of cancer type. However, research is inconclusive about which type of machine learning models are optimal. The suitability of five algorithms were assessed for the classification of 17 different cancer types. Each algorithm was fine-tuned and trained on the full array of 18,015 genes per sample, for 4,221 samples (75 % of the dataset). They were then tested with 1,408 samples (25 % of the dataset) for which cancer types were withheld to determine the accuracy of prediction. The results show that ensemble algorithms achieve 100% accuracy in the classification of 14 out of 17 types of cancer. The clustering and classification models, while faster than the ensembles, performed poorly due to the high level of noise in the dataset. When the features were reduced to a list of 20 genes, the ensemble algorithms maintained an accuracy above 95% as opposed to the clustering and classification models.

Figures

Figures reproduced from arXiv: 1908.06817 by the authors.

Figure 2
Figure 2. Layout of the cosmic_sample dataset displaying the primary site and histology subtype of each sample. Supervised machine learning models require the data to be labelled and thus, a combination of both datasets is needed for further analysis. The format of the processed [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Matrix representation of the pre-processed RNA-seq expression analysis, where rows represent samples, and columns represent the z-scores of each gene. RF builds classification trees are using a bootstrap sample of the dataset [6, 14, 15]. Each split is derived by searching a random subset (chosen by varying split points) of the given variables (16,718 genes) as the candidate set [16]. Although memory intensive, all … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 28 canonical work pages

  1. [1]

    Computer methods and programs in biomedicine 146, 11-24 (2017)

    Al-Rajab, M., Lu, J., Xu, Q.: Examining applying high performance genetic data feature selection and classification algorithms for colon cancer diagnosis. Computer methods and programs in biomedicine 146, 11-24 (2017)

  2. [2]

    Bioinformatics 21, 631-643 (2005)

    Statnikov, A., Aliferis, C.F., Tsamardinos, I., Hardin, D., Levy, S.: A comprehensive evaluation of multicategory classification methods for microarray gene expression cancer diagnosis. Bioinformatics 21, 631-643 (2005)

  3. [3]

    Computer methods and programs in biomedicine 113, 465-473 (2014)

    Azar, A.T., Elshazly, H.I., Hassanien, A.E., Elkorany, A.M.: A random forest classifier for lymph diseases. Computer methods and programs in biomedicine 113, 465-473 (2014)

  4. [4]

    The Journal of urology 195, 493-498 (2016)

    Bartsch, G., Mitra, A.P., Mitra, S.A., Almal, A.A., Ste ven, K.E., Skinner, D.G., Fry, D.W., Lenehan, P.F., Worzel, W.P., Cote, R.J.: Use of artificial intelligence and machine learning algorithms with gene expression profiling to predict recurrent nonmuscle invasive urothelial carcinoma of the bladder. The Journal of urology 195, 493-498 (2016)

  5. [5]

    Machine Learning 45, 5-32 (2001)

    Breiman, L.: Random Forests. Machine Learning 45, 5-32 (2001)

  6. [6]

    CRC press (1984)

    Breiman, L., Friedman, J., Stone, C.J., Olshen, R.A.: Classification and regression trees. CRC press (1984)

  7. [7]

    Human molecular genetics 23, 5866-5878 (2014)

    Ezkurdia, I., Juan, D., Rodriguez, J.M., Frankish, A., Diekhans, M., Harrow, J., Vazquez, J., Valencia, A., Tress, M.L.: Multiple evidence strands suggest that there may be as few as 19 000 human protein-coding genes. Human molecular genetics 23, 5866-5878 (2014)

  8. [8]

    Nature genetics 45, 1113 (2013)

    Weinstein, J.N., Collisson, E.A., Mills, G.B., Shaw, K.R.M., Ozenberger, B.A., Ellrott, K., Shmulevich, I., Sander, C., Stuart, J.M., Network, C.G.A.R.: The cancer genome atlas pan - cancer analysis project. Nature genetics 45, 1113 (2013)

Show all 29 references
  1. [9]

    Asian Pacific Journal of Cancer Prevention 17, 835-838 (2016)

    Podolsky, M.D., Barchuk, A.A., Kuznetcov, V.I ., Gusarova, N.F., Gaidukov, V.S., Tarakanov, S.A.: Evaluation of machine learning algorithm utilization for lung cancer classification based on gene expression levels. Asian Pacific Journal of Cancer Prevention 17, 835-838 (2016)

  2. [10]

    Egyptian Informatics Journal 18, 151-159 (2017)

    Tarek, S., Elwahab, R .A., Shoman, M.: Gene expression based cancer classification. Egyptian Informatics Journal 18, 151-159 (2017)

  3. [11]

    BMC bioinformatics 7, 3 (2006)

    Díaz-Uriarte, R., De Andres, S.A.: Gene selection and classification of microarray data using random forest. BMC bioinformatics 7, 3 (2006)

  4. [12]

    Computational Biology and Chemistry 28, 235-243 (2004)

    Tan, Y., Shi, L., Tong, W., Hwang, G.G., Wang, C.: Multi -class tumor classification by discriminant partial least squares using microarray gene expression data and assessment of classification models. Computational Biology and Chemistry 28, 235-243 (2004)

  5. [13]

    R: A language and environment for statistical computing

    Team, R.C.: R Development Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria; 2014. (2014)

  6. [14]

    Computers in biology and medicine 81, 139-147 (2017)

    Khalilabad, N.D., Hassanpour, H.: Employing image processing techniques for cancer detection using microarray images. Computers in biology and medicine 81, 139-147 (2017)

  7. [15]

    arXiv preprint arXiv:1202.1121 (2012)

    Kursa, M.B.: rFerns: An Implementation of the Random Ferns Method for General - Purpose Machine Learning. arXiv preprint arXiv:1202.1121 (2012)

  8. [16]

    -S., He, X

    Meng, J., Zhang, J., Luan, Y. -S., He, X. -Y., Li, L. -S., Zhu, Y. -F.: Parallel gene selection and dynamic ensemble pruning based on Affinity Propagation. Computers in Biology and Medicine 87, 8-21 (2017)

  9. [17]

    Machine learning 63, 3-42 (2006)

    Geurts, P., Ernst, D., Wehenkel, L.: Extremely randomized trees. Machine learning 63, 3-42 (2006)

  10. [18]

    Pattern Recognition 45, 3141-3153 (2012)

    Villamizar, M., Andrade -Cetto, J., Sanfeliu, A., Moreno -Noguer, F.: Bootstrapping boosted random ferns for discriminative and efficient object classification. Pattern Recognition 45, 3141-3153 (2012)

  11. [19]

    , Wang, Z., Ding, W.: Support vector machine classifier for prediction of the metastasis of colorectal cancer

    Zhi, J., Sun, J. , Wang, Z., Ding, W.: Support vector machine classifier for prediction of the metastasis of colorectal cancer. International journal of molecular medicine 41, 1419-1426 (2018)

  12. [20]

    PloS one 12, e0189875 (2017)

    Perez-Riverol, Y., Kuhn, M., Vizcaíno, J.A., Hitz, M.-P., Audain, E.: Accurate and fast feature selection workflow for high-dimensional omics data. PloS one 12, e0189875 (2017)

  13. [21]

    Optics & Laser Technology 102, 233-239 (2018)

    Li, X., Yang, S., Fan, R., Yu, X., Chen, D.: Discrimination of soft tissues using laser - induced breakdown spectroscopy in combination with k nearest ne ighbors (kNN) and support vector machine (SVM) classifiers. Optics & Laser Technology 102, 233-239 (2018)

  14. [22]

    Scientific reports 4, 4184 (2014)

    Shang, Y., Bouffanais, R.: Influence of the number of topologically interacting neighbors on swarm dynamics. Scientific reports 4, 4184 (2014)

  15. [23]

    Springer series in statistics Springer, Berlin (2001)

    Friedman, J., Hastie, T., Tibshirani, R.: The elements of statistical learning. Springer series in statistics Springer, Berlin (2001)

  16. [24]

    IEEE Transactions on Information Theory 13, 21-27 (1967)

    Cover, T., Hart, P.: Nearest neighbor pattern classification. IEEE Transactions on Information Theory 13, 21-27 (1967)

  17. [25]

    Cl inical Cancer Research (2018)

    Ye, Z., Dong, H., Li, Y., Ma, T., Huang, H., Leong, H.S., Eckel -Passow, J., Kocher, J.-P.A., Liang, H., Wang, L.: Prevalent Homozygous Deletions of Type I Interferon and Defensin Genes in Human Cancers Associate with Immunotherapy Resistance. Cl inical Cancer Research (2018)

  18. [26]

    Cancer research canres

    Rhee, H., Kim, H.-Y., Choi, J.-H., Woo, H.G., Yoo, J.E., Nahm, J.H., Choi, J.S., Park, Y.N.: Keratin 19 expression in hepatocellular carcinoma is regulated by fibroblast-derived HGF via a MET-ERK1/2-AP1 and SP1 axis. Cancer research canres. 0988.2017 (2018)

  19. [27]

    British journal of haematology (2018)

    Bram Ednersson, S., Stenson, M., Stern, M., Enblad, G., Fagman, H., Nilsson‐Ehle, H., Hasselblom, S., Andersson, P.O.: Expression of ribosomal and actin network proteins and immunochemotherapy resistance in diffuse l arge B cell lymphoma patients. British journal of haematology (2018)

  20. [28]

    PloS one 9, e85110 (2014)

    Sanz, G., Leray, I., Dewaele, A., Sobilo, J., Lerondel, S., Bouet, S., Grébert, D., Monnerie, R., Pajot-Augy, E., Mir, L.M.: Promotion of cancer cell invasiveness and metastasis emergence caused by olfactory receptor stimulation. PloS one 9, e85110 (2014)

  21. [29]

    Nature 499, 214 (2013)

    Lawrence, M.S., Stojanov, P., Polak, P., Kryukov, G.V., Cibulskis, K., Sivachenko, A., Carter, S.L., Stewart, C., Mermel, C.H., Roberts, S.A.: Mutational heterogeneity in cancer and the search for new cancer-associated genes. Nature 499, 214 (2013)

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.