REVIEW 2 major objections 5 minor 19 references
Empirical Analysis of Evasion and Poisoning Against Malware Data Drift Detection
T0 review · 2 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Data drift detectors with contrastive autoencoders make malware attacks non-monotone: larger perturbations and more poisoning samples can reduce attack success.
desk verdict First systematic attack surface analysis for malware drift detectors; the evasion non-monotonicity is solid, but the poisoning curves rest on an unverified top-k insertion precondition. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is CADE's contrastive autoencoder (CAE), which maps inputs into a low-dimensional embedding space with a contrastive loss that pulls same-class samples together and a reconstruction loss that preserves input detail. Drift is measured by the Euclidean distance from a sample to its class centroid, converted into an anomaly score and an OOD score; samples with high OOD scores are treated as drifted. Transcendent instead builds an SVM on the CAE embedding (or MLP latent) space and uses distances to the SVM decision boundary to compute credibility, confidence, and cred+conf scores. The argument turns on how these two metrics respond to attack perturbations: pushing a sample toward the benign region in classification terms can simultaneously push it away from the CAE benign centroid, increasing its drift score; and adding more poisoned samples crowds the embedding space near the centroid, forcing the CAE to move triggered samples outward during reconstruction.
What would settle it
Run the poisoning scenario without assuming all inserted samples are attacker-controlled: let the victim's drift detector pick the top-k drifted samples from a mix of ordinary benign samples, ordinary malware, and poisoned candidates, and check whether the poisoned samples are actually inserted. If they are not, the reported poisoning ASR curves would not reproduce in practice.
Extended reading notes
Core claim
The paper's central claim is that data drift detectors built on contrastive autoencoders do not behave like the malware classifiers they guard: a larger evasion perturbation, which monotonically improves evasion of the MLP classifier alone, can decrease overall attack success when CADE is in the pipeline, and adding more poisoning samples can similarly decrease attack success against CADE and Transcendent with CAE. The mechanism is that both attack levers push crafted samples away from the benign centroid in the CAE embedding space, where they become highly drifted and are selected for human inspection. The paper also finds that the attacker's surrogate model and loss term must be matched to the detector's geometry: perturbations tuned to an MLP's cross-entropy loss drift worse against CADE than perturbations tuned to CAE embedding distance.
Load-bearing premise
The poisoning results assume every sample inserted into the retraining set is an attacker-controlled poisoning sample, so the paper never tests whether poisoned files actually rank among the top drifted samples the victim's detector would select for inspection.
Editorial extensions
If this is right
- For a fixed detector budget k, attack success is not monotone in perturbation size; attackers must find a sweet spot between enough perturbation to fool the classifier and too much perturbation that triggers drift detection.
- The surrogate model and loss function matter: MLP cross-entropy perturbations transfer worse to CADE than CAE-distance perturbations at large perturbation magnitudes, so attack transferability depends on matching the detector's geometry.
- Transcendent with a CAE is more permissive than CADE under large perturbations, because its SVM-boundary metric treats confidently benign samples as non-drifting even when they are far from the benign centroid.
- Poisoning more samples can backfire against CAE-based detectors because the reconstruction loss runs out of embedding capacity, pushing triggered samples away from the benign centroid and into the drifted region.
- The combined pipeline of classifier plus drift detector is a strictly harder target than the classifier alone, and attack success rates that look high against the classifier can drop substantially once drift detection is included.
Reading between the lines
- Editorial inference: in a real deployment the victim's drift detector would select the top-k drifted samples from a mix of benign and poisoned candidates, so the paper's assumption that every inserted sample is attacker-controlled likely overstates poisoning attack success; a testable extension is to let the detector choose from mixed candidates and measure how many poisoned samples actually enter
- Editorial inference: the non-monotonicity suggests defenders could deliberately add a reconstruction-capacity bottleneck or a stronger contrastive loss to CAE-based drift detectors, forcing poisoned or over-perturbed samples into the drifted region as a cheap defense-in-depth layer.
- Editorial inference: the results point toward a 'drift-aware' attack objective that directly constrains the OOD score to stay below the top-k threshold while minimizing distance to the benign centroid, instead of maximizing classifier confidence.
- Editorial inference: the central mechanism—embedding distance and reconstruction capacity—may generalize to other embedding-based drift detectors or to raw-byte malware models, but that extension is not tested in this paper.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents an empirical study of evasion and poisoning attacks against malware classifiers coupled with data drift detectors (CADE and Transcendent). The authors train victim MLP classifiers and CAE- or MLP-based drift detectors on the Android (Androzoo) and Windows (Bodmas) datasets, then craft universal perturbations using surrogate MLP or surrogate CAE models. They report an overall attack success rate (ASR overall) that requires the crafted malware to be misclassified as benign and not among the top-k drifted samples selected by the drift detector. The central finding is that, unlike the monotonic success against the MLP classifier alone, larger perturbations and more poisoning samples can decrease ASR overall when a CAE-based drift detector is present: large perturbations push samples away from the benign centroid of the CAE, and poisoning fills the limited embedding space near the benign centroid, so the CAE learns to move triggered samples further away. The paper claims these effects are driven by the architecture and loss terms of the drift detector, and that the choice of surrogate loss (cross-entropy vs. CAE embedding distance) is decisive.
Significance. If the findings hold, they are a useful contribution to the adversarial machine-learning-for-malware literature: they identify a concrete way in which drift detectors with contrastive autoencoders change the attack surface, and they show that attacker effort (perturbation size, poisoning budget) does not monotonically improve success. The 30-run averaged experiments, the appendix tables linking CAE distance and anomaly scores to poisoning behavior (Tables 5 and 6), and the use of two realistic datasets with feature-manipulation constraints are strengths. The mechanism explanations are post hoc but are corroborated by measured distances and anomaly scores. However, the poisoning claim currently rests on an unverified selection assumption, and the lack of a no-attack baseline limits the interpretation of the reported ASR values.
major comments (2)
- [Section 3 and Section 4.2 (Poisoning Attack)] The paper assumes that all samples inserted to form the new training set are attacker-chosen poisoning samples, but under the stated threat model, a poisoning sample is inserted only if the victim's drift detector ranks it among the top k most drifted samples of the future selection set. The paper never verifies that the backdoor trigger (the universal perturbation of Algorithm 1) applied to benign samples produces drift scores (CADE OOD or Transcendent cred+conf) high enough to enter that top-k set. If the triggered benign samples are not among the top k, the poisoning ASR curves in Figures 3-6 and the supporting distance/anomaly tables (Tables 5-6) correspond to training sets the victim would never form, and the conclusion that 'more poisoning samples can inhibit attack success' would not be a property of the attack pipeline under the threat model. Please report the drift-score percentiles of the candidate poisoning samples relative to the future selection set, and the fraction of candidates actually selected at k=200 and k=500; if they are not selected, the poisoning scenario needs to be re-framed as an oracle insertion model.
- [Section 5.1 (Evaluation Metrics) and Figures 1-6] ASR overall is reported only for attacked malware samples; there is no no-attack baseline showing the proportion of unmodified malware samples that are both misclassified as benign and not in the top-k drifted set. Because the victim pipeline's base rate of 'success' (i.e., malware evading both components) is unknown, the reported ASR values cannot be interpreted as attack effectiveness. For example, a non-monotonic ASR curve could partly reflect the drift detector's intrinsic detection rate rather than the attack's interaction with it. Please report the no-attack ASR overall for each dataset, detector, and k, and discuss how the attack changes this baseline.
minor comments (5)
- [Figures 1-6] The figures plot averaged ASR overall over 30 runs without error bars or confidence bands, even though Tables 2 and 3 report standard deviations; adding error bars or shaded intervals would make the trends and their variability clearer.
- [Section 5.2 (Evasion), Androzoo results] In the Androzoo experiments, the 'perturbation size' sweep is confounded: increasing from 50 to 100 features uses the 200-feature significant set, while 200 and 500 use the 1000-feature set. This should be stated explicitly in the figure captions or main text, as the trend is not a pure magnitude sweep.
- [Section 3 and Algorithm 1] The phrase 'apply perturbation limit(u)' is vague; please specify exactly how the per-feature clipping is performed for the min-max scaled Bodmas features and for the binary Androzoo features, including how the 0-to-1 bound interacts with negative updates.
- [Section 5 (Evaluation)] The paper states that experiments applying universal perturbations to malware samples in the future selection set have 'similar results' but provides no supporting data; please include an appendix table or figure to substantiate this claim.
- [Section 3] There is a typo in the first paragraph: 'evalution' should be 'evaluation'.
Circularity Check
No significant circularity: the attacks are evaluated against independently trained victim models and the non-monotone ASR curves are measured, not derived from the attack objectives by construction.
full rationale
The paper's central claims (non-monotone attack success under evasion and poisoning against CAE-based drift detectors) are empirical measurements on independently trained victim models. The surrogate models are trained on a separate 10% subset of the initial training set, and the victim CADE and Transcendent detectors are pre-existing systems trained without the attacker's perturbation objectives. The non-monotone ASR curves in Figures 1-4 are measured, not imposed: the loss functions in Equations (1) and (2) optimize different objectives, and the observed rollover (larger perturbation no longer evades the drift detector) is exactly a case where the attack optimization does not match the detector's scoring rule. No quantity is fitted to the target result, and no derivation reduces an equation to itself. The poisoning setup contains a simplifying assumption that all inserted training samples are attacker-controlled ('our experiments assume all samples inserted to form the new training set are poisoning samples'), but this is a stated threat-model simplification, not a way of defining the result into existence; the claimed drop in ASR with more inserted samples is read off retrained victim models and supported by measured CAE distances in Tables 5 and 6. Self-citations are not load-bearing: the cited detectors, datasets, and prior attacks are external artifacts, and no uniqueness theorem from the authors' prior work is invoked. The mechanism explanations are post hoc interpretations of measured curves, which may be correct or incorrect but are not circular.
Assumptions & free parameters
free parameters (4)
- Top-k inspection budget k =
500 (default), 200 (one experiment)
- Androzoo significant feature budget =
200 or 1000 features
- Perturbation limit =
0.25, 0.5, 0.75, 1.0 (Bodmas, scaled); 50, 100, 200, 500 (Androzoo)
- Number of inserted poisoning samples =
100 to 1000 (Bodmas); 500 to 4000 (Androzoo)
assumptions (5)
- domain assumption A victim uses the drift detector to select the k most drifted samples for manual inspection and blocks malware found among them.
- domain assumption The attacker knows 10% of the benignware and 10% of the malware in the initial training set and can train surrogate models that transfer to the victim.
- domain assumption Poisoning is clean-label: every inserted poisoning sample is derived from benignware and is correctly labeled benign during the victim's manual triage.
- domain assumption Only 17 of 2,381 Bodmas features can be modified without breaking file functionality, while all Androzoo features can be incremented to 1.
- ad hoc to paper In poisoning experiments, every sample inserted into the retraining set is an attacker-controlled poisoning sample.
Cite this review
Pith. "Pith review of Empirical Analysis of Evasion and Poisoning Against Malware Data Drift Detection." pith.science (2026). https://pith.science/paper/N4V2EAEQ
@misc{pith2026260803642,
author = {Pith},
title = {Pith review of: Empirical Analysis of Evasion and Poisoning Against Malware Data Drift Detection},
year = {2026},
howpublished = {\url{https://pith.science/paper/N4V2EAEQ}},
note = {Machine review of arXiv:2608.03642}
}
read the original abstract
As concept drift due to malware evolution presents challenges for malware classification, machine learning-based data drift detection tools are developed to mitigate this problem. These data drift detector tools are designed for a different purpose and built with different techniques compared to malware classifiers. Although evasion and poisoning attacks against machine learning-based malware classifiers can cause misclassification of malware samples, it is not clear how these attacks work against data drift detectors and malware classifiers in combination. This work investigates the effect of evasion and poisoning attacks on the data drift detector along with the malware classifier. We demonstrate how unique characteristics of data drift detectors cause attacks against malware classifiers to work differently against them.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
Androzoo: Collecting millions of android apps for the research community,
K. Allix, T. F. Bissyand´ e, J. Klein, and Y. Le Traon, “Androzoo: Collecting millions of android apps for the research community,” inProceedings of the 13th interna- tional conference on mining software repositories, 2016, pp. 468–471
2016
-
[2]
Ember: an open dataset for training static pe mal- ware machine learning models,
H. S. Anderson and P. Roth, “Ember: an open dataset for training static pe mal- ware machine learning models,”arXiv preprint arXiv:1804.04637, 2018
arXiv 2018
-
[3]
Drebin: Effective and explainable detection of android malware in your pocket
D. Arp, M. Spreitzenbarth, M. Hubner, H. Gascon, K. Rieck, and C. Siemens, “Drebin: Effective and explainable detection of android malware in your pocket.” inNdss, vol. 14, 2014, pp. 23–26
2014
-
[4]
Transcending transcend: Revisiting malware classification in the presence of concept drift,
F. Barbero, F. Pendlebury, F. Pierazzi, and L. Cavallaro, “Transcending transcend: Revisiting malware classification in the presence of concept drift,” in2022 IEEE Symposium on Security and Privacy (SP). IEEE, 2022, pp. 805–823
2022
-
[5]
Evad- ing malware classifiers via monte carlo mutant feature discovery,
J. Boutsikas, M. E. Eren, C. Varga, E. Raff, C. Matuszek, and C. Nicholas, “Evad- ing malware classifiers via monte carlo mutant feature discovery,”arXiv preprint arXiv:2106.07860, 2021
arXiv 2021
-
[6]
Continuous learning for android malware detection,
Y. Chen, Z. Ding, and D. Wagner, “Continuous learning for android malware detection,” in32nd USENIX Security Symposium (USENIX Security 23), 2023, pp. 1127–1144
2023
-
[7]
Adversarial exem- ples: Functionality-preserving optimization of adversarial windows malware,
L. Demetrio, B. Biggio, G. Lagorio, A. Armando, and F. Roli, “Adversarial exem- ples: Functionality-preserving optimization of adversarial windows malware,” in ICML 2021 Workshop on Adversarial Machine Learning, 2021
2021
-
[8]
Explaining vul- nerabilities of deep learning to adversarial malware binaries,
L. Demetrio, B. Biggio, G. Lagorio, F. Roli, and A. Armando, “Explaining vul- nerabilities of deep learning to adversarial malware binaries,”arXiv preprint arXiv:1901.03583, 2019
arXiv 1901
Show all 19 references
-
[9]
Transcend: Detecting concept drift in malware classification mod- els,
R. Jordaney, K. Sharad, S. K. Dash, Z. Wang, D. Papini, I. Nouretdinov, and L. Cavallaro, “Transcend: Detecting concept drift in malware classification mod- els,” in26th USENIX security symposium (USENIX security 17), 2017, pp. 625– 642
2017
-
[11]
Deceiving end-to-end deep learning malware detectors using adversarial examples,
——, “Deceiving end-to-end deep learning malware detectors using adversarial examples,”arXiv preprint arXiv:1802.04528, 2018
2018 arXiv
-
[12]
Malware makeover: Breaking ml-based static analysis by modifying executable bytes,
K. Lucas, M. Sharif, L. Bauer, M. K. Reiter, and S. Shintre, “Malware makeover: Breaking ml-based static analysis by modifying executable bytes,” inProceedings of the 2021 ACM Asia Conference on Computer and Communications Security, 2021, pp. 744–758
2021
-
[13]
Malware detection by eating a whole exe,
E. Raff, J. Barker, J. Sylvester, R. Brandon, B. Catanzaro, and C. K. Nicholas, “Malware detection by eating a whole exe,” inWorkshops at the thirty-second AAAI conference on artificial intelligence, 2018
2018
-
[14]
{Explanation-Guided}backdoor poisoning attacks against malware classifiers,
G. Severi, J. Meyer, S. Coull, and A. Oprea, “{Explanation-Guided}backdoor poisoning attacks against malware classifiers,” in30th USENIX security symposium (USENIX security 21), 2021, pp. 1487–1504
2021
-
[15]
Automatic gener- ation of adversarial examples for interpreting malware classifiers,
W. Song, X. Li, S. Afroz, D. Garg, D. Kuznetsov, and H. Yin, “Automatic gener- ation of adversarial examples for interpreting malware classifiers,”arXiv preprint arXiv:2003.03100, 2020
2003 arXiv
-
[16]
Exploring adversarial examples in malware detection,
O. Suciu, S. E. Coull, and J. Johns, “Exploring adversarial examples in malware detection,” in2019 IEEE Security and Privacy Workshops (SPW). IEEE, 2019, pp. 8–14
2019
-
[17]
When does machine learning{F AIL}? generalized transferability for evasion and poisoning attacks,
O. Suciu, R. Marginean, Y. Kaya, H. Daume III, and T. Dumitras, “When does machine learning{F AIL}? generalized transferability for evasion and poisoning attacks,” in27th USENIX Security Symposium (USENIX Security 18), 2018, pp. 1299–1316
2018
-
[18]
Jigsaw puzzle: Selective backdoor attack to subvert malware clas- sifiers,
L. Yang, Z. Chen, J. Cortellazzi, F. Pendlebury, K. Tu, F. Pierazzi, L. Cavallaro, and G. Wang, “Jigsaw puzzle: Selective backdoor attack to subvert malware clas- sifiers,” in2023 IEEE Symposium on Security and Privacy (SP). IEEE, 2023, pp. 719–736
2023
-
[19]
Bodmas: An open dataset for learning based temporal analysis of pe malware,
L. Yang, A. Ciptadi, I. Laziuk, A. Ahmadzadeh, and G. Wang, “Bodmas: An open dataset for learning based temporal analysis of pe malware,” in2021 IEEE Security and Privacy Workshops (SPW). IEEE, 2021, pp. 78–84
2021
-
[20]
{CADE}: Detecting and explaining concept drift samples for security applica- tions,
L. Yang, W. Guo, Q. Hao, A. Ciptadi, A. Ahmadzadeh, X. Xing, and G. Wang, “{CADE}: Detecting and explaining concept drift samples for security applica- tions,” in30th USENIX Security Symposium (USENIX Security 21), 2021, pp. 2327–2344. A Appendix Table 3: Varying Perturbation ...
2021
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.