REVIEW 4 major objections 4 minor 21 references
Mal-D2GAN: Double-Detector based GAN for Malware Generation
T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A GAN with two detectors generates adversarial malware that drives eight machine-learning detectors' true positive rate to near zero, with random forest falling from 97.25% to 2.28% on the training set.
desk verdict The paper's headline claim is likely an artifact of an uncontrolled detector comparison; the double-detector idea is worth a look but needs a valid evaluation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the double-detector design: a substitute detector $D_1$ trained on black-box labels supplies a differentiable surrogate of the target detector, while an additional detector $D_2$ is trained on benign samples and the generator's adversarial outputs. Their losses combine as $L_D = \alpha L_{D_1} + (1-\alpha) L_{D_2}$, and the generator minimizes the combined detectors' confidence that a generated sample is malware. The generator uses a smooth output $G_{\theta_g}(m,z) = \max(m,o)$, then binarizes at the 0.5 threshold, so gradients flow through the substitute detector while the final adversarial sample stays a binary feature vector that only adds unrelated features.
What would settle it
Train all three GANs against one fixed set of frozen black-box detector weights on the same train/test split and compare adversarial TPR on the same test set; the concrete check is whether the original TPR for each classifier is identical across the three model columns, since Tables IV and V currently show different original TPRs for the same classifier (e.g., RF 97.25 for Mal-D2GAN versus 97.75 for Mal-LSGAN on the training set).
Extended reading notes
Core claim
The central claim is that the additional detector is what makes the adversarial examples effective: an extra detector trained on benign data and on the generator's adversarial output sharpens the generator's ability to fool the black-box detector, while the substitute detector keeps gradients flowing. The generator maps a 160-dimensional API feature vector plus noise to a modified binary vector by adding unrelated features through a bitwise OR operation, leaving malware functionality intact. Detector losses are combined as $L_D = \alpha L_{D_1} + (1-\alpha) L_{D_2}$ with $\alpha = 0.5$, and the generator is trained against the combined detector output with a mean-square-error loss. The reported outcome is adversarial true positive rates near zero for all eight detectors, and after black-box detectors are retrained, retraining Mal-D2GAN produces new examples that remain undetected.
Load-bearing premise
The comparison assumes the eight black-box detectors were identical, with the same trained weights and data split, across the Mal-D2GAN, Mal-LSGAN, and MalGAN runs, so that differences in true positive rates come only from the GAN model; if the detectors differed between runs, the claimed advantage is not established.
Editorial extensions
If this is right
- Adversarial malware generated by Mal-D2GAN drives adversarial true positive rates below a few percent for RF, LR, DT, SVM, MLP, AB, GB, and KNN, while MalGAN and Mal-LSGAN leave most of those detectors with materially higher true positive rates.
- The additional detector, not just the least-squares loss, is credited with the improvement, since Mal-LSGAN uses the same loss family with a single detector and achieves worse evasion on most classifiers.
- Retraining the black-box detectors five times on the captured adversarial examples initially detects all of them, but retraining Mal-D2GAN against the updated detectors yields new adversarial examples that remain undetected for RF, DT, AB, GB, and KNN.
- Because the attack adds unrelated API features rather than removing malware features, the generated variants are intended to preserve malware functionality, which matters for whether such evasion transfers to real-world detection pipelines.
Reading between the lines
- A controlled ablation that disables the additional detector (setting $\alpha = 1$) while keeping all other training details fixed would isolate how much of the TPR drop comes from the second detector; the paper's comparisons against MalGAN and Mal-LSGAN are not fully controlled ablations.
- The reported drops across eight different model families suggest the generated adversarial examples transfer across detectors, so a natural test is whether examples generated against one detector also fool the other seven without any retraining.
- If the result transfers to real executables, defenses should consider feature-space regularization or behavior-based sandboxing rather than relying on retraining, because the paper shows retraining alone can be bypassed once the attacker retrains the GAN.
- The same double-detector recipe could be tried on other malware feature representations, such as static PE headers, network flows, or Android permissions, where a binary feature vector can be modified by adding irrelevant features.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes Mal-D2GAN, a GAN-based adversarial malware example generator that extends the MalGAN substitute-detector architecture with an additional detector and least-squares losses. The authors report experiments on 20,000 Portable Executable samples against eight black-box machine-learning classifiers, claiming that adversarial examples produced by Mal-D2GAN reduce the true positive rate of all eight detectors to near zero and that the method outperforms MalGAN and Mal-LSGAN, including in a retraining-based defense setting. The paper presents the network structures, loss equations, training algorithm, and three result tables.
Significance. If the comparison were controlled, the paper would provide a useful data point: a simple double-detector modification that reduces detector TPR more effectively than two existing GAN-based generators. The use of a real malware dataset and eight common classifiers is appropriate, and the retraining defense experiment addresses an important practical question. However, because the evaluation does not hold the black-box detectors fixed across the compared models and does not report run-to-run variability, the headline comparative claim is not currently supported. The significance of the contribution is therefore conditional on a re-executed, controlled experiment; as written, the evidence does not establish that the additional detector is the cause of the observed TPR reductions.
major comments (4)
- [IV.B, Tables IV and V] The 'Original' TPR values for the same classifier differ across the three compared models; for example, on the training set RF is 97.25 for Mal-D2GAN, 97.75 for Mal-LSGAN, and 97.02 for MalGAN, and similar discrepancies appear for LR, DT, and other classifiers. Because the original TPR is determined by the original malware samples and the trained black-box detector, these differences imply that the black-box detector was not identical across the compared models. The reported adversarial TPR differences (e.g., RF 2.28 vs 20.14 vs 31.68) are therefore confounded by differences in the detector itself rather than being attributable solely to the GAN architecture. The authors must rerun all three models with one fixed black-box detector per classifier, using the same training split and the same detector initialization and training procedure, and report the original TPR in a single shared column.
- [III.C.3, Eq. (5)] The generator loss uses the notation D_{θd1 d2}(G_{θg}(m, z)), but no definition of this combined detector output is given. Equations (2)-(4) define LD1, LD2, and the weighted total LD, but they do not specify how the outputs of the substitute detector D1 and the additional detector D2 are combined to produce the scalar D_{θd1 d2} that the generator minimizes. This is a load-bearing detail because this combined value is the training signal for the generator; the authors should state the combination rule explicitly, for example as an average, a product, or a concatenation followed by a shared output layer.
- [IV.C, Table VI] The retraining protocol is described too loosely to be interpreted. The table contains 'Before' and 'After' columns for each model, and the text says the black-box detector is retrained five times, after which the GAN is retrained, but it is unclear what each column represents. In particular, the 'After' columns for Mal-D2GAN are claimed to demonstrate that new adversarial examples remain undetected after GAN retraining, while the 'After' columns for Mal-LSGAN are a mixture of 100% and non-100% values. The authors should specify exactly when each TPR is measured, which dataset each column uses, and how many retraining rounds were applied to the detector and to the generator.
- [IV.A and IV.B] No repeated runs or variance statistics are reported for any of the TPR numbers. GAN training is stochastic, and the differences between the compared models are sometimes small relative to the variation that random seeds and initialization can introduce. The authors should run each model with multiple random seeds and report the mean and standard deviation, or confidence intervals, for both the original and adversarial TPR values. This is necessary to assess whether the observed differences between Mal-D2GAN, Mal-LSGAN, and MalGAN are statistically meaningful.
minor comments (4)
- [III.C.3, Algorithm 1, line 6] The instruction 'Label SBenign and GAdversarial using additional detector' appears to be a typo, since the additional detector is trained on these sets with known labels rather than used to label them; please correct Algorithm 1 to say that the labels are assigned by the black-box detector or are known benign/generated labels.
- [Table III] Table III has the caption 'NETWORK STRUCTURE OF THE SUBSTITUTE DETECTOR' even though it describes the additional detector; the caption should be changed to 'NETWORK STRUCTURE OF THE ADDITIONAL DETECTOR'.
- [IV.A] The paper fixes the balance weight α at 0.5 but provides no sensitivity analysis; since α controls the relative contribution of LD1 and LD2 in Eq. (4), the authors should justify the choice or show that the conclusions are insensitive to it.
- [IV.A] No code or data repository is provided. For reproducibility, the authors should release the implementation, the exact feature extraction pipeline, and the train/test split used in the experiments.
Circularity Check
The paper trains the generator to minimize the detector's output and then reports the resulting training-set TPR drop as empirical evidence; this part of the claim reduces to the optimization objective, while the test-set results retain independent content and the cross-model comparison is additionally confounded.
-
fitted input called prediction
[Section III.C.3, Eq. (5); Section IV.B, Table IV]
"Minimizing LG reduces the probability of the generated malware being predicted as harmful, thereby encouraging the substitute detector to classify malware as benign. ... Overall, the results in Table IV and Table V show that the Mal-D2GAN model reduced the detection accuracy (true positive rate) across 8 malware detection algorithms."
The generator weights θg are updated by descending along ∇θ LG (Algorithm 1, line 8), where LG = ½ E[(D(G(m,z)))²] (Eq. 5). Table IV then reports the black-box TPR of the generated samples on the same training set. Because the substitute detector is trained on black-box labels to approximate the black-box detector, minimizing Eq. (5) directly drives the training-set adversarial TPR toward zero; for example, RF drops from 97.25% to 2.28%. This reported training-set result is the value of the optimized objective evaluated at the fitted generator, not an independent prediction.
full rationale
The central empirical claim is that Mal-D2GAN outperforms MalGAN and Mal-LSGAN in reducing detector TPR. For this comparison to be valid, the black-box detector for each classifier must be identical across the three GAN models. Tables IV and V violate this: the 'Original' TPR for the same classifier differs across models—e.g., RF Original is 97.25 for Mal-D2GAN, 97.75 for Mal-LSGAN, 97.02 for MalGAN on the training set; similar discrepancies appear for LR, DT, and others. Since the original samples and the trained detector determine the original TPR, these numbers cannot differ if the detector is shared. The most plausible explanation is that each model was evaluated with a separately trained black-box detector, or on different data splits. In that case, the adversarial TPR differences (e.g., RF 2.28 vs 20.14 vs 31.68) reflect differences in the detectors themselves, not the GAN architectures. The paper provides no indication that detectors were held fixed, nor does it report variance across runs. This confound is sufficient to invalidate the headline comparison. A secondary issue is the undefined combined detector loss in Eq. (5), but the detector inconsistency alone is decisive. The training-set TPR reduction is also the direct optimization objective (Eq. 5), so the reported near-zero TPR on the training set is expected from the loss rather than an external empirical finding. The test-set TPRs (Table V) are more meaningful because they evaluate on samples not directly optimized, but they face the same detector-fixedness problem. Overall, the paper has partial circularity: the headline training-set results reduce to the fitted objective, and the cross-model comparison is confounded. There is no self-citation chain or uniqueness import; the paper's own equations and tables supply the evidence.
Assumptions & free parameters
free parameters (6)
- alpha (balance weight) =
0.5
- noise dimension Z =
10
- hidden units =
256
- binarization threshold =
0.5
- training epochs =
20
- retraining epochs =
5
assumptions (4)
- domain assumption The substitute detector approximates the black-box detector closely enough for gradient transfer.
- domain assumption Adding unrelated API features to malware preserves its malicious functionality.
- domain assumption The 160 selected dynamic features are sufficient to represent malware behavior.
- ad hoc to paper The combined detector output D_{θd1 d2} in Eq. (5) is a well-defined training signal.
invented entities (1)
-
Additional detector (D2)
Cite this review
Pith. "Pith review of Mal-D2GAN: Double-Detector based GAN for Malware Generation." pith.science (2026). https://pith.science/paper/RKKO6Y7O
@misc{pith2026250518806,
author = {Pith},
title = {Pith review of: Mal-D2GAN: Double-Detector based GAN for Malware Generation},
year = {2026},
howpublished = {\url{https://pith.science/paper/RKKO6Y7O}},
note = {Machine review of arXiv:2505.18806}
}
read the original abstract
Machine learning (ML) has been developed to detect malware in recent years. Most researchers focused their efforts on improving the detection performance but ignored the robustness of the ML models. In addition, many machine learning algorithms are very vulnerable to intentional attacks. To solve these problems, adversarial malware examples are generated by GANs to enhance the robustness of the malware detector. However, since current GAN models suffer from limitations such as unstable training and weak adversarial examples, we propose the Mal-D2GAN model to address these problems. Specifically, the Mal-D2GAN architecture was designed with double-detector and a least square loss function and tested on a dataset of 20,000 samples. The results show that the Mal-D2GAN model reduced the detection accuracy (true positive rate) in 8 malware detectors. The performance was then compared with that of the existing MalGAN and Mal- LSGAN models.
Figures
Reference graph
Works this paper leans on
-
[1]
Ray, Cybersecurity for Connected Medical Devices
A. Ray, Cybersecurity for Connected Medical Devices . Elsevier Inc, 2022
work page 2022
-
[2]
The pipeline process of signature-based and behavior-based malware detection,
M. Goyal and R. Kumar, “The pipeline process of signature-based and behavior-based malware detection,” in 2020 IEEE 5th International Conference on Computing Communication and Automation (ICCCA) , India, 2020
work page 2020
-
[3]
T. M. Mitchell, Machine Learning . McGraw-Hill Sci- ence/Engineering/Math, 1997
work page 1997
-
[4]
Adversarial attacks against windows pe malware detection: A survey of the state-of-the-art,
X. Ling et al. , “Adversarial attacks against windows pe malware detection: A survey of the state-of-the-art,” https://arxiv.org/abs/2112. 12310v5, 2023, arXiv:2112.12310v5 [cs.CR], 17 Feb 2023
arXiv 2023
-
[5]
I. Goodfellow et al. , “Generative adversarial nets,” in Proceedings of the 27th International Conference on Neural Information Processing Systems (NeurIPS), vol. 2, 2014
work page 2014
-
[6]
The threat of adversarial attacks against machine learn- ing,
O. Ibitoye et al., “The threat of adversarial attacks against machine learn- ing,” https://arxiv.org/abs/1911.02621v3, 2023, arXiv:1911.02621v3 [cs.CR], 21 Mar 2023
arXiv 1911
-
[7]
Generating adversarial malware examples for black-box attacks based on gan,
W. Hu and Y . Tan, “Generating adversarial malware examples for black-box attacks based on gan,” in Data Mining and Big Data: 7th International Conference (DMBD 2022) , 2022, pp. 409–423
work page 2022
-
[8]
Mal- lsgan: An effective adversarial malware example generation model,
J. Wang, X. Chang, J. Mi ˇsi´c, V . B. Miˇsi´c, Y . Wang, and J. Zhang, “Mal- lsgan: An effective adversarial malware example generation model,” in 2021 IEEE Global Communications Conference (GLOBECOM) , 2021, pp. 1–6
work page 2021
Show all 21 references
-
[9]
LSGAN- AT: enhancing malware detector robustness against adversarial examples,
J. Wang, X. Chang, Y . Wang, R. J. Rodr ´ıguez, and J. Zhang, “LSGAN- AT: enhancing malware detector robustness against adversarial examples,” Cybersecurity, vol. 4, no. 1, p. 38, 2021. [Online]. Available: https://doi.org/10.1186/s42400-021-00102-9
2021 doi
-
[10]
Least squares generative adversarial networks,
X. Mao et al. , “Least squares generative adversarial networks,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2794–2802
2017
-
[11]
Aagan: Android malware generation system based on generative adversarial network,
V . H. Trung et al., “Aagan: Android malware generation system based on generative adversarial network,” Vietnam Journal of Computer Science , vol. 11, no. 2, pp. 275–299, 2024
2024
-
[12]
Generative adversarial networks: An overview,
A. Creswell et al. , “Generative adversarial networks: An overview,” https://arxiv.org/abs/1710.07035, 2017, arXiv:1710.07035v1 [cs.CV], 19 Oct 2017
2017 arXiv
-
[13]
Adversarial attacks on mobile malware detec- tion,
M. Shahpasand et al. , “Adversarial attacks on mobile malware detec- tion,” in 2019 IEEE 1st International Workshop on Artificial Intelligence for Mobile (AI4Mobile) , February 2019, pp. 24–24
2019
-
[14]
Unsupervised representation learning with deep con- volutional generative adversarial networks,
A. Radford et al., “Unsupervised representation learning with deep con- volutional generative adversarial networks,” https://arxiv.org/abs/1511. 06434, 2015
2015
-
[15]
Virusshare.com - because sharing is caring,
“Virusshare.com - because sharing is caring,” https://virusshare.com/, accessed 1 Jan 2024
2024
-
[16]
Cuckoo sandbox - automated malware analysis,
“Cuckoo sandbox - automated malware analysis,” https://github.com/ cuckoosandbox, accessed 1 Jan 2024
2024
-
[17]
Anaconda,
“Anaconda,” https://www.anaconda.com/
-
[18]
Foster, Generative Deep Learning
D. Foster, Generative Deep Learning . O’Reilly Media, 2019
2019
-
[19]
Generative adversarial networks for mal- ware detection: a survey,
A. Dunmore et al. , “Generative adversarial networks for mal- ware detection: a survey,” https://arxiv.org/abs/2302.08558, 2023, arXiv:2302.08558v2 [cs.CR], 24 Feb 2023
2023 arXiv
-
[20]
Alazab, Deep Learning Applications for Cyber Security
M. Alazab, Deep Learning Applications for Cyber Security . Springer, 2019
2019
-
[21]
A. Y . Mahgoub et al., Machine Learning Techniques for Cybersecurity . Springer, 2023
2023
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.