REVIEW 5 major objections 6 minor 2 cited by
Deep Learning-Driven Malware Classification with API Call Sequence Analysis and Concept Drift Handling
T0 review · 5 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read The paper claims that adding genetic-algorithm mutants of API n-gram features raises malware classification accuracy on data from a different time period.
desk verdict The concept-drift claim is unsupported because Tables 11 and 12 compare static feature sets on the same fixed dataset; the GA-augmented features show a feature-engineering gain, not drift adaptation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying mechanism is the genetic algorithm operating on n-gram strings in Phase 5 of the framework. It mutates sub-parts of API sequence features, defines a fitness score as the number of positions in which the mutant differs from a target string (a Hamming/edit distance), and uses that score to select the most distant mutants as new features. These selected mutants are appended to the original corpus, so the deep networks train on both the original 2023-era features and synthetic variants, which the paper argues allows the classifier to recognize samples from an older, drifted distribution.
What would settle it
Train the models on a dataset from one period (e.g., 2023) and test on a later period (e.g., 2024), with and without the 10,500 GA-mutated features. If the accuracy gain over the un-augmented model disappears on this genuinely future test set, the GA mutation is not handling concept drift; it is only helping on the particular older test set used in the paper.
Extended reading notes
Core claim
The central discovery claim is that GA-generated mutations of API n-gram sequences constitute an effective concept-drift handling mechanism. Starting from n-grams extracted from sandbox API call reports, the algorithm mutates the secondary and tertiary parts of the sequence strings while leaving the primary API name untouched, scores each mutant by its edit distance to a target string, and selects the top 1,500 mutants per malware family (10,500 total) to augment the existing feature corpus. Retraining the same ANN, CNN, and RNN architectures with these added features on a 2020 malware dataset produced higher accuracy and lower loss than training without them, which the paper presents as evidence of adaptability to concept drift.
Load-bearing premise
The load-bearing premise is that mutating strings of existing API n-gram features produces features that represent how malware behavior drifts over time; if that equivalence is false, the experiments measure feature augmentation on a fixed distribution rather than drift handling.
Editorial extensions
If this is right
- If the GA-mutated features truly emulate drift, then a model trained on current features plus mutants should generalize to past or future distributions without collecting new samples.
- The method adds only about 1% more features (10,500 of 88,972), so drift handling can be achieved at a marginal feature-cost increase.
- The edit-distance fitness selection offers a principled way to choose which mutants to retain, favoring strings most different from the originals.
- The effect appeared in both ANN and CNN architectures, suggesting the augmentation may transfer to other deep classifiers or feature-based detectors.
Reading between the lines
- An implicit, testable extension would be to check whether the GA-generated mutant n-grams actually occur in malware released after the training period; the paper does not show that the synthetic strings match real drifted behavior.
- The comparison between two fixed datasets does not by itself establish drift handling; stronger evidence would come from a time-ordered evaluation where the model trains on one period and is tested on a later period, with and without mutants.
- The fitness function rewards distance from a target string, which may select for highly unusual strings rather than representative ones; an alternative would be to generate mutants from drift-detection signals such as a validation-window performance drop.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a hybrid deep-learning and genetic-algorithm framework for malware classification from API call n-grams, with the stated goal of handling concept drift. The method extracts unigrams, bigrams, and trigrams from Cuckoo Sandbox reports, applies term-frequency filtering, and then generates additional features by mutating existing API n-grams with a genetic algorithm. The GA selects 10,500 mutants by an edit-distance fitness score, and these are added to the feature set. The authors evaluate ANN, CNN, and RNN classifiers on two datasets collected from VirusShare (Dataset 1: January–June 2023; Dataset 2: January–May 2020), comparing results without and with the GA-generated features. They report that including the GA features on Dataset 2 improves accuracy and reduces loss, and they attribute this improvement to concept-drift handling. The paper also includes a comparison with two related works and a brief threats-to-validity section.
Significance. If the central claim were valid, a method that demonstrably improves malware classification under concept drift by augmenting API n-gram features would be of practical interest to the malware-detection community. The paper also makes a concrete attempt to combine evolutionary feature generation with deep classifiers, and it provides a detailed data-collection pipeline. However, the significance is severely limited by the experimental design: the comparison claimed to demonstrate concept-drift handling is conducted on a single static dataset with no temporal or distributional shift, so the reported gains are equally consistent with ordinary feature augmentation. The GA fitness function is underspecified, and the RNN results indicate a non-learning model, further weakening the empirical claims. The authors do provide a public code link and a dataset-availability statement, but no machine-checked proofs or reproducible experiments beyond what is described.
major comments (5)
- [Section 6.2, Tables 11 and 12] The central claim that "utilising concept drift in the malware dataset can lead to improved testing accuracy and reduced loss" is not supported by the experiments. Tables 11 and 12 both report results on Dataset 2 (Table 10), a single static collection from January to May 2020. The only reported difference between the two conditions is the presence of the GA-generated features; there is no temporal split, no train-on-old/test-on-new evaluation, and no distributional shift between the two settings. Under the paper's own definition in Section 2.1, concept drift is a change between successive time steps P_t(X,y) and P_{t+1}(X,y), but no time steps are used in the evaluation. The improvements in Table 12 over Table 11 are therefore equally explained by static feature augmentation on a fixed distribution, and the concept-drift claim is not established.
- [Section 5, Phase 5 and Equation (1)] The GA fitness function is underspecified in a way that directly affects the validity of the feature-selection step. Equation (1) defines Fitness(individual) as the number of mismatched characters against a "target string," but the manuscript never defines what the target string is, how it is chosen per class, or how it relates to the malware families. The selection of the "top 1500" features per class based on this fitness score is therefore not reproducible and may introduce arbitrary class-specific bias. Moreover, the fitness score is computed on the same malware corpus that is later used for training and testing, so the feature-selection process is circular with respect to the reported accuracy gain.
- [Section 6.2, Tables 7 and 11/12] The RNN results are not interpretable as a meaningful comparison. Table 7 shows that the RNN's loss and accuracy remain nearly constant (loss around 1.7–1.9, accuracy around 0.39) across all epochs, which strongly suggests the model is not learning. The architecture in Table 3 uses an input shape of (88972, 1), treating all 88,972 features as a single time step, which is not a meaningful sequence representation for API call n-grams. The RNN rows in Tables 11 and 12 show slightly different but still near-constant values, so any "improvement" attributed to the GA for the RNN is not credible as evidence of concept-drift handling.
- [Section 6.2, Tables 11 and 12] No error bars, repeated runs, or statistical significance tests are reported. The accuracy differences between Table 11 and Table 12 are small (e.g., CNN training accuracy improves from 0.9250 to 0.9314 and validation accuracy from 0.9430 to 0.9459), and the ANN validation accuracy improves from 0.8946 to 0.9184 while its training loss slightly increases from 0.6250 to 0.6314. Without variance estimates or multiple trials, the claim of "significant improvements" is not justified, and the observed differences could be within run-to-run variability.
- [Section 8, Threats to Validity] The manuscript itself concedes that the concept-drift handling approach "may not fully capture the complexity and dynamics of concept drift in real-world malware datasets" (Section 8, External Validity). This admission directly undermines the abstract and Section 6.2's conclusion that the GA-augmented features demonstrate adaptability to concept drift. Since the evaluation never introduces a temporal or distributional shift, this limitation is not a secondary caveat but a statement that the central phenomenon the paper claims to address has not been measured.
minor comments (6)
- [Abstract and Section 2.3] Typographical errors include "handeling" (abstract), "handelling" (Section 2.3), and "paramount are dynamic environments" (Section 2.1). These should be corrected.
- [Section 5, Phase 5] The manuscript refers to Figure 5 as "Mutated Features" but the figure appears after Figure 3 and the caption is uninformative; the figure numbering and captions should be revised for clarity.
- [Table 7] The row numbering in Table 7 is inconsistent: rows 5 and 6 are labeled "3" and "4" again after rows 3 and 4 have already used those numbers. The epoch values also skip from 60 to 80 but the labels repeat.
- [Equation (1)] Equation (1) defines the fitness as a sum over positions i of the target string, but it is not specified how the fitness is computed when the individual and the target string have different lengths, which is likely given that API n-gram strings vary in length. This should be clarified.
- [Section 6.1 and Table 5] The text states that "two lakhs of a diverse set of malware samples" were collected, but the reported datasets contain only about 22,000 samples each. The discrepancy between the claimed collection size and the actual dataset sizes should be explained.
- [Section 7 and Table 13] Table 13 compares the proposed work with only two prior studies, and the comparison is not quantitative because the datasets and evaluation protocols differ. The table would be more informative if it included a direct performance comparison or a clear statement that no direct comparison is possible.
Circularity Check
The concept-drift claim reduces to GA feature augmentation: Tables 11 and 12 differ only in added mutant features on the same static Dataset 2, with no temporal drift, so the reported gains do not demonstrate drift handling.
-
self definitional
[Section 2.1 (drift definition) vs. Section 5, Phase 5 (drift-handling implementation) vs. Section 6.2 (claim)]
"Formally, concept drift occurs when: Pt (X,y) ̸= Pt+1(X,y), where t and t + 1 denote successive time steps. ... Phase 5: Concept Drift Handling Phase ... mutation occurs in the sub-parts of API sequence features ... These mutants serve as features, adding approximately ≈ 1% to our existing feature corpus. ... Table 12 demonstrates the impact of concept drift on the same old dataset (Dataset 2) ..."
The paper defines concept drift formally as a change between successive time steps P_t(X,y) vs P_{t+1}(X,y), but the experiment never creates successive time steps: Dataset 2 (Table 10) is a single static collection (January to May 2020), and Tables 11 and 12 differ only in whether the GA-mutated n-gram features are added. Therefore 'concept drift handling' is operationalized entirely as the GA mutation/addition step, and the reported accuracy gain is attributed to 'concept drift' even though drift itself was never varied. The conclusion 'concept drift handling improves accuracy' is equivalent by construction to 'adding GA-mutated features to the same corpus changes accuracy', which is the only manipulated input.
-
fitted input called prediction
[Section 5, Phase 5, Eq. (1) and Table 1; Section 6.2, Tables 10-12]
"The fitness score [19] is calculated as the edit distance from the target string. ... Fitness(individual ) = n ∑ i=1 1ai̸=bi (1) ... From this pool, we selected 10,500 mutants, with the top 1500 from each malware category based on their fitness scores as shown in Table 1. These mutants serve as features, adding approximately ≈ 1% to our existing feature corpus."
The 10,500 mutant features are derived from the same malware corpus (the n-gram patterns of Dataset 2) whose accuracy is later reported as the 'with concept drift handling' result. The fitness function is an edit-distance count against an unspecified target string; it is not computed on any future or held-out drift period. Thus the improvement in Table 12 over Table 11 is an in-sample feature-augmentation effect measured on the very corpus that supplied the mutants. Calling this improvement a prediction about concept-drift robustness is a fitted-input-called-prediction step: the input features are constructed from the evaluation data, and no independent temporal split exists to validate drift adaptation.
1 more flagged steps
-
other
[Section 8, External Validity]
"Concept Drift Representation: The concept drift handling approach may not fully capture the complexity and dynamics of concept drift in real-world malware datasets, affecting the effectiveness of the proposed approach in practical scenarios."
This is the paper's own admission that the GA-mutation procedure is not validated as a representation of concept drift. Since the experimental comparison (Tables 11 vs 12) measures only the addition of GA-mutated features, and the paper concedes this may not capture drift, the central claim that the results show 'the effectiveness of the concept drift malware classifier' lacks an independent referent. The admission reinforces that the claimed drift-handling effect is attached by definition to the feature-augmentation step rather than demonstrated against a drifting distribution.
full rationale
The paper has no problematic self-citation chain: its references are external and the GA fitness formula is attributed to an independent source. The circularity is instead in the construction of the central evaluation. The formal definition in Section 2.1 requires P_t(X,y) != P_{t+1}(X,y) for successive time steps, but no time steps appear in the experiments. Dataset 1 and Dataset 2 are two static collections from different periods; the 'without' and 'with concept drift' conditions are both applied to the same Dataset 2, with the only stated difference being the presence of 10,500 GA-mutated n-gram features. Consequently, the accuracy improvements in Table 12 over Table 11 are explained equally well, and more parsimoniously, by static feature augmentation on a fixed distribution. The GA fitness score is an edit distance to an undefined target string, and the mutants are selected from and evaluated on the same corpus, so no held-out future distribution tests drift adaptation. Section 8 explicitly concedes that the approach 'may not fully capture the complexity and dynamics of concept drift in real-world malware datasets.' The central claim therefore reduces by construction to the operational definition of drift handling as GA feature mutation; the reported numbers may be real, but they do not measure drift adaptation. This is partial circularity rather than a fully empty derivation, because the feature augmentation effect itself is an empirical observation, so the score is 6 rather than 8-10.
Assumptions & free parameters
free parameters (5)
- GA mutation and crossover rates
- Fitness target string per class
- Number of mutant features selected per class =
1500
- Term frequency filtering threshold
- Neural network training hyperparameters
assumptions (5)
- ad hoc to paper The target string in Equation (1) is a meaningful and well-defined prototype for fitness.
- domain assumption Mutated API n-grams represent future or drifted malware behaviors.
- domain assumption The VirusTotal majority label is the correct ground truth for the malware family.
- domain assumption Using the same Dataset 2 for both the no-drift and 'drift handling' conditions is a valid way to measure concept drift.
- ad hoc to paper The RNN architecture treating the 88,972 features as time steps is a valid model for this data.
Cite this review
Pith. "Pith review of Deep Learning-Driven Malware Classification with API Call Sequence Analysis and Concept Drift Handling." pith.science (2026). https://pith.science/paper/ICKEEWLO
@misc{pith2026250208679,
author = {Pith},
title = {Pith review of: Deep Learning-Driven Malware Classification with API Call Sequence Analysis and Concept Drift Handling},
year = {2026},
howpublished = {\url{https://pith.science/paper/ICKEEWLO}},
note = {Machine review of arXiv:2502.08679}
}
read the original abstract
Malware classification in dynamic environments presents a significant challenge due to concept drift, where the statistical properties of malware data evolve over time, complicating detection efforts. To address this issue, we propose a deep learning framework enhanced with a genetic algorithm to improve malware classification accuracy and adaptability. Our approach incorporates mutation operations and fitness score evaluations within genetic algorithms to continuously refine the deep learning model, ensuring robustness against evolving malware threats. Experimental results demonstrate that this hybrid method significantly enhances classification performance and adaptability, outperforming traditional static models. Our proposed approach offers a promising solution for real-time malware classification in ever-changing cybersecurity landscapes.
Figures
Figures from the paper (3 more)
Forward citations
Cited by 2 Pith papers
-
Understanding Concept Drift with Deprecated Permissions in Android Malware Detection
Removing deprecated and restricted Android permissions barely changes malware detection accuracy, but makes year-to-year model drift easier to detect.
-
System Calls for Malware Detection and Classification: Methodologies and Applications
A survey of system call and API call based malware detection that consolidates known methodologies and repeats the authors' previously published accuracy results.
Reference graph
Works this paper leans on
-
[7]
Fesad ran- somware detection framework with machine learning using adap- tion to concept drift
Damien Warren Fernando and Nikos Komninos. Fesad ran- somware detection framework with machine learning using adap- tion to concept drift. Computers & Security, 137:103629, 2024
work page 2024
-
[1]
The problem of concept drift: definitions and related work
Alexey Tsymbal. The problem of concept drift: definitions and related work. Computer Science Department, Trinity College Dublin, 106(2):58, 2004
work page 2004
-
[2]
Learning under concept drift: A review
Jie Lu, Anjin Liu, Fan Dong, Feng Gu, Joao Gama, and Guangquan Zhang. Learning under concept drift: A review. IEEE transactions on knowledge and data engineering , 31(12):2346– 2363, 2018
work page 2018
-
[3]
Mining data streams with concept drifts using genetic algorithm
Periasamy Vivekanandan and Raju Nedunchezhian. Mining data streams with concept drifts using genetic algorithm. Artificial In- telligence Review, 36:163–178, 2011
work page 2011
-
[4]
Is it overkill? analyzing feature-space con- cept drift in malware detectors
Zhi Chen, Zhenning Zhang, Zeliang Kan, Limin Yang, Jacopo Cortellazzi, Feargus Pendlebury, Fabio Pierazzi, Lorenzo Caval- laro, and Gang Wang. Is it overkill? analyzing feature-space con- cept drift in malware detectors. In 2023 IEEE Deep Learning Se- curity and Privacy Workshop (DLSP). IEEE, 2023
work page 2023
-
[5]
A critical review on adverse effects of concept drift over machine learning classifica- tion models
Syed Muslim Jameel, Manzoor Ahmed Hashmani, Hitham Alhus- sain, Mobashar Rehman, and Arif Budiman. A critical review on adverse effects of concept drift over machine learning classifica- tion models. International Journal of Advanced Computer Science and Applications, 11(1), 2020
work page 2020
-
[6]
An effectiveness analysis of transfer learning for the concept drift problem in malware detection
David Escudero García, Noemí DeCastro-García, and Angel Luis Muñoz Castañeda. An effectiveness analysis of transfer learning for the concept drift problem in malware detection. Ex- pert Systems with Applications, 212:118724, 2023
work page 2023
-
[8]
A survey on machine learning for recurring concept drift- ing data streams
Andrés L Suárez-Cetrulo, David Quintana, and Alejandro Cer- vantes. A survey on machine learning for recurring concept drift- ing data streams. Expert Systems with Applications, 213:118934, 2023
work page 2023
Show all 19 references
-
[9]
Co- variate shift: A review and analysis on classifiers
Nimisha G Nair, Pallavi Satpathy, Jabez Christopher, et al. Co- variate shift: A review and analysis on classifiers. In 2019 Global Conference for Advancement in Technology (GCAT) , pages 1–6. IEEE, 2019
2019
-
[10]
Classifier ensembles for changing environ- ments
Ludmila I Kuncheva. Classifier ensembles for changing environ- ments. In International workshop on multiple classifier systems , pages 1–15. Springer, 2004
2004
-
[11]
Practical malware analysis based on sandboxing
Mihai Vasilescu, Laura Gheorghe, and Nicolae Tapus. Practical malware analysis based on sandboxing. In 2014 RoEduNet Con- ference 13th Edition: Networking in Education and Research Joint Event RENAM 8th Conference, pages 1–6. IEEE, 2014
2014
-
[12]
Malware dynamic analysis evasion techniques: A survey
Amir Afianian, Salman Niksefat, Babak Sadeghiyan, and David Baptiste. Malware dynamic analysis evasion techniques: A survey. ACM Computing Surveys (CSUR), 52(6):1–28, 2019
2019
-
[13]
Dynamic malware analy- sis using cuckoo sandbox
Sainadh Jamalpur, Yamini Sai Navya, Perla Raja, Gampala Tagore, and G Rama Koteswara Rao. Dynamic malware analy- sis using cuckoo sandbox. In 2018 Second international confer- ence on inventive communication and computational technologies (ICICCT), pages 1056–1060. IEEE, 2018
2018
-
[14]
https://github.com/cuckoosandbox/cuckoo
Cuckoo sandbox. https://github.com/cuckoosandbox/cuckoo. Ac- cessed: 10th Feb, 2024
2024
-
[15]
Concept drift evolution in machine learning approaches: a systematic literature review
Manzoor Ahmed Hashmani, Syed Muslim Jameel, Mobashar Rehman, and Atsushi Inoue. Concept drift evolution in machine learning approaches: a systematic literature review. International Journal on Smart Sensing and Intelligent Systems , 13(1):1–16, 2020
2020
-
[16]
An adaptive ensemble classifier for mining con- cept drifting data streams
Dewan Md Farid, Li Zhang, Alamgir Hossain, Chowd- hury Mofizur Rahman, Rebecca Strachan, Graham Sexton, and Keshav Dahal. An adaptive ensemble classifier for mining con- cept drifting data streams. Expert Systems with Applications , 40(15):5895–5906, 2013
2013
-
[17]
https://virusshare.com/
Virusshare. https://virusshare.com/. Accessed: 10th Feb, 2024
2024
-
[18]
https://www.virustotal.com/gui/home/upload
Virustotal. https://www.virustotal.com/gui/home/upload. Ac- cessed: 10th Feb, 2024
2024
-
[19]
Genetic algorithms for credit scoring: Alternative fitness function performance comparison
Vaclav Kozeny. Genetic algorithms for credit scoring: Alternative fitness function performance comparison. Expert Systems with ap- plications, 42(6):2998–3004, 2015
2015
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.