REVIEW 5 major objections 5 minor 9 references
SEdroid: A Robust Android Malware Detector using Selective Ensemble Learning
T0 review · 5 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read SEdroid uses selective ensemble learning to keep Android malware detection at 98.3% precision even when 10% of training labels are flipped.
desk verdict The robustness claim in Table 2 is arithmetically incompatible with the stated 10% test-set label swap, so the paper's central evidence doesn't hold as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the selective ensemble: many component deep belief networks are trained on bootstrap samples, a 0/1 weight vector chooses which components vote, and the weights are evolved by a genetic algorithm whose fitness function is $F(x) = \text{Accuracy} \times D$, where $D$ is the average pairwise Euclidean distance between component predictions. The diversity factor prevents the ensemble from collapsing into identical networks, while the accuracy factor keeps the selected subset useful. The final detector is a majority vote over the selected components.
What would settle it
Re-run the same 30-run comparison with label noise correlated to app families, or at noise levels of 5% and 20%, or with naturally noisy labels from multiple antivirus scanners; if SEdroid's average precision and recall fall to DroidDeep's level or below in any of these conditions, the robustness claim is falsified.
Extended reading notes
Core claim
SEdroid's central claim is that a selective ensemble of deep belief networks, with component selection driven by a genetic algorithm that maximizes accuracy times diversity, is more robust to training-label noise than a single deep network. The paper supports this by comparing average performance over 30 runs on a dataset with 10% of labels swapped: DroidDeep degrades to 93.8% precision and 94.3% recall, with its worst run at 89.7% and 90.3%, while SEdroid holds 98.3% and 98.1% on average. On the clean dataset, the selective ensemble also improves over single-component classifiers, which the authors attribute to the diversity term in the fitness function.
Load-bearing premise
The robustness claim rests on the assumption that randomly swapping 10% of the malicious and benign labels mimics real-world dataset noise; if real mislabeling is correlated with app families or features, or occurs at a different rate, SEdroid's reported advantage may not hold.
Editorial extensions
If this is right
- Android malware detectors could be trained from automatically collected, uncurated app datasets without a full manual audit, since 10% label noise no longer degrades accuracy sharply.
- The accuracy-times-diversity fitness gives a practical recipe for choosing ensemble members in other security classifiers where label noise is expected.
- A selective ensemble can recover performance that a single deep network loses under noise, suggesting diversity is a usable robustness lever, not just an accuracy booster.
- Because the genetic algorithm selects a subset of trained networks, robustness does not require paying the full ensemble's compute or memory cost at inference.
Reading between the lines
- The same selective-ensemble recipe likely transfers to other malware-detection feature sets, such as dynamic API call sequences or graph embeddings, since nothing in the argument depends on the three specific static feature types.
- The paper only tests one noise level; a natural extension would vary the swap rate to see whether SEdroid's advantage grows or shrinks as noise becomes more or less severe.
- The diversity term is computed on final predictions, not on internal representations; testing diversity based on hidden-layer activations or gradients might yield even stronger ensembles under label noise.
- If real-world mislabeling is correlated with app families or specific features, SEdroid's advantage could change; measuring it on naturally noisy multi-scanner labels would be a useful stress test.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SEdroid, an Android malware detector that builds an ensemble of deep belief network (DBN) component learners trained on bootstrap samples and then uses a genetic algorithm to select a subset of these learners by a fitness function combining accuracy and diversity. The evaluation uses a dataset of 16,000 Android apps (8,000 malicious and 8,000 benign) and compares SEdroid against SVM, DroidDeep, and an ensemble of SVM classifiers on an original dataset and on a modified dataset in which 10% of malicious and benign labels are swapped. The paper reports that SEdroid achieves 98.3% precision and 98.1% recall on the modified dataset, versus 93.8% and 94.3% for DroidDeep on average, and interprets this as evidence of the superiority and robustness of selective ensembling.
Significance. If the reported results were sound, the paper would make a useful contribution by showing that selective ensemble learning can improve Android malware detection under label noise. The general idea of jointly optimizing accuracy and diversity with a genetic algorithm is reasonable, and the use of 30 repeated experiments is a good instinct. However, the central robustness claim is not adequately supported as presented: the evaluation protocol for the modified dataset is ambiguous and appears arithmetically inconsistent with the reported numbers, and the paper lacks the statistical detail, baseline comparisons, and hyperparameter information needed to confirm that selective ensembling is the source of the improvement. The contribution is therefore promising but unverified.
major comments (5)
- [Section 3.1 and Table 2] The evaluation protocol for the modified dataset is ambiguous and, under the natural reading, arithmetically inconsistent. Section 3.1 states that 10% of malicious and benign apps are swapped to create a modified dataset and that SEdroid is evaluated on both datasets, with the 60/20/20 split described afterward. If the swap is performed before splitting, then the test partition contains corrupted labels, and no true-label classifier can exceed 90% precision and recall on a balanced test set: with 10% of each class relabeled, precision = recall = 7,200/8,000 = 0.90. Table 2 reports SEdroid at 0.983 precision and 0.981 recall on the modified dataset, which is impossible under that reading. The manuscript never states whether the swap reaches the test partition or how the metrics are computed relative to the corrupted labels. Please clarify the protocol and, if the test set is corrupted, recompute the reported metrics on clean test labels; as written, the main robustness result is not supported.
- [Section 3.3] The 30 repeated experiments are summarized only by worst, best, and average values for DroidDeep and by a single average for SEdroid. No standard deviation, confidence interval, or paired significance test is reported. Given that DroidDeep's best precision and recall are 0.967 and 0.960, respectively, and SEdroid's average is 0.983 and 0.981, the claimed superiority could be within run-to-run variation. Please report the dispersion of SEdroid's results across the 30 runs and a paired statistical comparison against DroidDeep.
- [Section 2.3 and Algorithm 1] The genetic algorithm is not specified enough to reproduce the experiments or to judge whether the selection process is responsible for the reported improvement. The manuscript omits the population size, the number of iterations, crossover and mutation probabilities, the selection strategy, and the number N of bootstrap component learners. In addition, the fitness function F(x) = Accuracy x D in Eq. (2) uses accuracy on the training data, but the paper does not state whether the validation split is used to guide selection or to avoid overfitting to the noisy training labels. These details are essential for a claim about robustness.
- [Section 3.3 and Table 1] There is no comparison with a full ensemble baseline that does not perform selection. Table 1 reports an ensemble of SVM learners and SEdroid, which is an ensemble of DBN learners, but there is no 'ensemble of all DBN learners' row. Consequently, the improvement over DroidDeep could be due to ensembling itself rather than to the selective mechanism, especially because Table 1 already shows that ensembling SVM improves over single SVM. The paper should directly compare SEdroid against its own full ensemble with all component learners included.
- [Section 3.1 and Section 3.3] The noise model is limited to a single level of label swapping, and the structure of the simulated noise is not analyzed. Real-world mislabeling may be correlated with application features, concentrated in particular malware families, or present at rates different from 10%, and the paper provides no evidence that the reported advantage persists under such conditions. The conclusion that SEdroid is robust to 'low-quality datasets' in general is broader than what the current experiment can support.
minor comments (5)
- [Section 3.3] The text refers to 'Fig.6', but the manuscript contains only Figure 1; please correct the cross-reference.
- [Equation (3)] In Eq. (3), the symbol N is used for both the number of component learners and the normalization factor in the diversity measure, which is confusing; please clarify and define the range of the outer sum explicitly.
- [Table 1 and Table 2] The column headers 'Precision(M)' and 'Recall(M)' are not defined; please state explicitly that M denotes the malware class and describe whether the reported metrics are per-class or macro-averaged values.
- [References] Several references have incomplete or inconsistent bibliographic information, such as reference [9], which lists both 'Sarnoff Symposium, 2016 IEEE' and the year 2017; please unify the reference format.
- [Throughout] The manuscript would benefit from language editing; for example, Section 3.2 contains the phrase 'almostly correct', and several sentences in the introduction and conclusion are awkwardly constructed.
Circularity Check
No circularity: SEdroid is an empirical evaluation with no derivation-level input/output identity; the genetic algorithm uses only training-set accuracy and diversity, and no result is forced by construction.
full rationale
The paper's central claim is an experimental robustness comparison, not a derivation. The selective ensemble is optimized by a genetic algorithm whose fitness function combines training-set accuracy and a diversity term (Eqs. 2-3); the subsequent precision, recall, and F1 numbers are measured on a held-out test partition. No parameter is fitted to the test labels, no test-set information enters the ensemble selection, and the paper does not rely on prior work by the same authors as a load-bearing premise. The only notable concern is experimental protocol: Section 3.1 describes swapping 10% of malicious and benign labels before describing the 60/20/20 split, so if the label swap reaches the test set, a perfect true-label classifier would be capped near 90% precision and recall against the corrupted test labels, making Table 2's 98.3%/98.1% figures arithmetically questionable. That is a correctness or reporting problem, not circularity, because the reported numbers are not equivalent to the training objective by construction. Hence the circularity score is 0.
Assumptions & free parameters
free parameters (4)
- Label noise level =
10%
- Number of component learners N
- Genetic algorithm parameters
- DBN architecture
assumptions (5)
- domain assumption The three static features (permission, intent-action, API calls) are sufficient for Android malware detection.
- domain assumption 10% swapped labels is a realistic simulation of real-world dataset noise.
- domain assumption The dataset labels are basically correct after VirusTotal and manual checking.
- standard math Ensemble accuracy and pairwise Euclidean diversity are good fitness signals for selecting an ensemble.
- domain assumption Bootstrap sampling with replacement creates usefully diverse component learners.
Cite this review
Pith. "Pith review of SEdroid: A Robust Android Malware Detector using Selective Ensemble Learning." pith.science (2026). https://pith.science/paper/KS4ZHCOG
@misc{pith2026190903837,
author = {Pith},
title = {Pith review of: SEdroid: A Robust Android Malware Detector using Selective Ensemble Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/KS4ZHCOG}},
note = {Machine review of arXiv:1909.03837}
}
read the original abstract
For the dramatic increase of Android malware and low efficiency of manual check process, deep learning methods started to be an auxiliary means for Android malware detection these years. However, these models are highly dependent on the quality of datasets, and perform unsatisfactory results when the quality of training data is not good enough. In the real world, the quality of datasets without manually check cannot be guaranteed, even Google Play may contain malicious applications, which will cause the trained model failure. To address the challenge, we propose a robust Android malware detection approach based on selective ensemble learning, trying to provide an effective solution not that limited to the quality of datasets. The proposed model utilizes genetic algorithm to help find the best combination of the component learners and improve robustness of the model. Our results show that the proposed approach achieves a more robust performance than other approaches in the same area.
Figures
Reference graph
Works this paper leans on
-
[1]
A machine learning approach to android malware detection
Justin Sahs and Latifur Khan. A machine learning approach to android malware detection. In Intelligence and Security Informatics Conference , pages 141–147, 2012
work page 2012
-
[2]
Dali Zhu, Hao Jin, Ying Yang, Di Wu, and Weiyi Chen. Deepflow: Deep learning- based malware detection by mining android application for abnormal usage of sensitive data. pages 438–443, 2017
work page 2017
-
[3]
A deep learning approach to android malware feature learning and detection
Xin Su, Dafang Zhang, Wenjia Li, and Kai Zhao. A deep learning approach to android malware feature learning and detection. In Trustcom/bigdatase/iâĂŃspa, pages 244–251, 2017
work page 2017
-
[4]
Android malware detection using deep learning on api method sequences
El Mouatez Billah Karbab, Mourad Debbabi, Abdelouahid Derhab, and Djedjiga Mouheb. Android malware detection using deep learning on api method sequences. 2017
work page 2017
-
[5]
Ensembling neural networks: Many could be better than all
Zhi Hua Zhou, Jianxin Wu, and Wei Tang. Ensembling neural networks: Many could be better than all. In ARTIFICIAL INTELLIGENCE, 2002
work page 2002
-
[6]
G Carney, P Cunningham, and U Bhagwan
J. G Carney, P Cunningham, and U Bhagwan. Confidence and prediction intervals for neural network ensembles. InInternational Joint Conference on Neural Networks, pages 1215–1218 vol.2, 1999
work page 1999
-
[7]
David W. Opitz and Jude W. Shavlik. Generating accurate and diverse members of a neural-network ensemble. Advances in Neural Information Processing Systems , 8:535–541, 1996
work page 1996
-
[8]
Design and implementation of a genetic-based algorithm for data mining
Sunil Choenni. Design and implementation of a genetic-based algorithm for data mining. In VLDB 2000, Proceedings of International Conference on Very Large Data Bases, September 10-14, 2000, Cairo, Egypt , pages 33–42, 2000
work page 2000
Show all 9 references
-
[9]
Droiddeeplearner: Identifying android malware using deep learning
Zi Wang, Juecong Cai, Sihua Cheng, and Wenjia Li. Droiddeeplearner: Identifying android malware using deep learning. In Sarnoff Symposium, 2016 IEEE , pages 160–165, 2017
2016
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.