REVIEW 4 major objections 5 minor 38 references
RoMA: Robust Malware Attribution via Byte-level Adversarial Training with Global Perturbations and Adversarial Consistency Regularization
T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read RoMA, a single-step adversarial training scheme, holds about 80% robust accuracy under PGD attacks on APT malware attribution while keeping clean accuracy near 91%.
desk verdict A credible first pass at robust APT malware attribution with a large within-mask improvement, but the headline 80% robustness is only demonstrated under the four-region attack surface used in training. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is a pool of $K=50$ global perturbation vectors maintained in the word-embedding space. For each malware, a selection head chooses one GP via a contrastively trained logits layer; the GP is added to embeddings at the four perturbation positions and mapped back to nearest byte values, followed by a single FGSM gradient step that updates both the bytes and the selected GP with momentum. Around this, two auxiliary losses act on the projection space: adversarial contrastive loss pulls clean and adversarial projections of the same APT group together while pushing different groups apart, and adversarial distribution loss, a KL divergence, aligns the clean and adversarial prediction distributions.
What would settle it
Run PGD-50 with the same perturbation budget but permit byte changes in a fifth region outside the allowed four, such as the code section under a semantics-preserving transformation. If RoMA's robust accuracy drops from about 80% toward the 1-2% range seen for undefended MalConv, the robustness claim is limited to the paper's perturbation set rather than to advanced attacks in general.
Extended reading notes
Core claim
The central claim is that a byte-level APT malware attribution model trained with RoMA can withstand strong white-box PGD attacks while retaining clean-set accuracy and training faster than multi-step alternatives. The authors report 80.13% robust accuracy at PGD-50, falling to 78.79% at PGD-70, with attack success rate around 12-13%, compared with 36.85% RA and 58.86% ASR for the second-best PGD-4-AT; clean accuracy is 91.00% and training time is 2378 minutes, roughly half of PGD-4-AT. Against C&W attacks all adversarially trained models stay above 84% RA. The claimed mechanism is that single-step FGSM becomes sufficient when the perturbation is preceded by a learned global perturbation pattern and the representation is regularized by contrastive and distribution consistency losses.
Load-bearing premise
The results stand on the assumption that an attacker's functionality-preserving byte modifications are confined to the four regions of a Windows executable that RoMA perturbs: the DOS header except its magic and PE pointer, a 1KB shift gap, slack space, and up to 100KB of appended padding.
Editorial extensions
If this is right
- Deployed attribution models can keep working under white-box PGD attacks: RoMA's RA stays near 80% at PGD-50 versus below 2% for standard MalConv.
- Multi-step PGD training is not necessary for this task: RoMA trains in about 2378 minutes versus 4965 for PGD-4-AT while being more robust.
- All three components matter: removing GP drops PGD-50 RA by 65.97 points, and removing all three drops RA by 69.80 points, so the robustness gain is attributed to the combination.
- C&W optimization attacks are a weak test after adversarial training; all adversarially trained methods exceed 84% RA, so PGD remains the discriminating evaluation.
Reading between the lines
- Editorial inference: the four-region perturbation set defines the threat surface; expanding to other functionality-preserving regions, such as resources or relocation data, would likely change absolute numbers even if the relative ranking persists.
- Editorial inference: the GP pool should be dataset-dependent; a model trained on AMG18 may need its GP pool re-initialized when applied to a different PE corpus, and that is a testable extension.
- Editorial inference: ACLoss and ADLoss are not tied to attribution and could be tested on binary malware detection or binary authorship tasks with the same single-step training recipe.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes RoMA, a single-step adversarial training method for byte-level APT malware attribution. RoMA combines a Global Perturbation (GP) pool, which stores learned perturbation patterns in embedding space and is selected per-sample via a contrastively trained Selection Head, with two consistency regularizers (Adversarial Contrastive Loss and Adversarial Distribution Loss) that encourage the model to treat clean and adversarial samples consistently. The authors introduce a new APT dataset AMG18 (6360 samples, 18 groups) and report experiments comparing RoMA against seven baselines, including non-adversarial MalConv/AvastNet, FGSM-AT, PGD-2-AT, PGD-4-AT, and image-domain adversarial training methods (FGSM-RS, NuAT). The main reported results are that RoMA achieves about 80% Robust Accuracy under PGD-50/60/70 while the next-best method (PGD-4-AT) reaches about 36%, and that RoMA trains in about half the time of PGD-4-AT, while also achieving the highest clean accuracy (91.00%).
Significance. If the claims hold, the paper would be the first systematic study of adversarial training for APT malware attribution at the byte level, and it would demonstrate a substantial robustness improvement over existing approaches. The GP mechanism, which explicitly learns shared perturbation patterns and uses them to generate stronger adversarial examples, is a novel idea, and the consistency regularizers are well-motivated for the attribution task. The public release of the trained model and dataset is a positive element for reproducibility. However, the significance is tempered by a missing alignment between the stated threat model and the evaluation attack surface, and by the absence of statistical reliability measures; these issues must be resolved before the central claims can be accepted.
major comments (4)
- [Section 3, Section 4.2, Section 5.2] The threat model in Section 3 states that the attacker can modify raw bytes through functionality-preserving transformations with no restriction on which bytes are modified, yet both the training procedure (Algorithm 1) and the reported evaluation appear to confine perturbations to four predefined PE regions: the DOS header (excluding MZ and the PE pointer), a 1KB shifting space before the first section, slack space between sections, and up to 100KB of trailing padding. The paper does not explicitly state in Section 5.2 whether the PGD and C&W attacks used for evaluation apply the same four-position mask. If they do, the claimed ~80% Robust Accuracy is measured only against attacks restricted to the exact regions RoMA was trained to defend, which is not equivalent to robustness under the broad threat model stated in Section 3. Other editable, functionality-preserving byte regions (e.g., unused section-header fields, optional-header padding, import/export name padding, resource data) are not tested, so no evidence is provided that robustness transfers to those regions. The authors should either narrow the threat model to the four-position set with an explicit justification that these positions constitute the realistic attack surface, or broaden the evaluation to additional editable regions and demonstrate that RoMA's robustness advantage persists.
- [Table 1, Section 5.2] All reported accuracy numbers (SA, RA, ASR) are single runs without error bars, multiple seeds, or significance tests. The claim of 'superior standard accuracy' rests on a 91.00% versus 90.06% gap over MalConv, which is likely within run-to-run variability for this type of model; similarly, the training-time comparison (2378 vs 4965 minutes) is a single observation. The authors should report the mean and standard deviation over at least three independent training runs for each method, and include a significance test (e.g., paired bootstrap or Wilcoxon test) for the key SA and RA comparisons.
- [Section 5.2] The hyperparameters K=50, λ1=λ2=0.3, τ=0.6, and ε=0.6 are described as 'selected through extensive tuning,' but no validation split or tuning protocol is given. If the test set was used to select these values, then the comparison against baselines is biased, as the baselines do not appear to have received comparable tuning. The authors should specify the validation split used for model selection, report the ranges over which hyperparameters were tuned, and state whether the baselines' hyperparameters were tuned on the same validation set.
- [Section 5.2, Table 1] The C&W attack evaluation is underspecified: the paper reports only that C&W is an optimization-based attack and gives a single row of RA/ASR numbers, with no details on the attack configuration (e.g., learning rate, number of iterations, binary-search steps, confidence parameter). Given that the contributions explicitly claim robustness against 'optimization-based attacks,' the paper should provide the full C&W attack settings and ideally evaluate over a range of attack strengths or budgets, rather than a single configuration that shows all adversarial-training methods performing similarly.
minor comments (5)
- [Sections 5.3–5.4] The text refers to 'Table A3' for the experiment results, but the displayed main-results table is numbered 'Table 1,' while the appendix also contains a dataset table numbered 'Table A3.' This duplicate reference should be resolved by renumbering the main table and updating all cross-references.
- [Abstract] The phrase 'more than twice as fast as the second-best method in terms of accuracy' is confusing; the intended comparison is training time, not accuracy. Please rephrase to say 'more than twice as fast in training time' or similar.
- [Section 5.2] The adaptations of FGSM-RS and NuAT from the image domain to malware attribution are not described. Provide implementation details, including how the perturbations are projected onto the four PE positions, to allow reproducibility and fair comparison.
- [Section 5.1] The provenance of the newly added samples in AMG18 is described only briefly as 'hash-label pairs from public threat repositories.' A more detailed data-availability statement, including exact source repositories and sample hashes, would strengthen the dataset's usability and credibility.
- [Section 6] The sentence 'The RoMA-trained model and visualized malware samples are publicly available' is odd; presumably 'visualized representations' is intended. Please clarify.
Circularity Check
No material circularity: RoMA's robustness and efficiency claims rest on experiments against external baselines, not on assumptions that encode the conclusions.
full rationale
RoMA's derivation chain is an empirical training/evaluation pipeline, not a formal derivation. The min-max objective (Eq. 1) and the GP/regularization losses (Eqs. 3-6) are proposed mechanisms evaluated against seven baselines on the new AMG18 dataset. The few self-citations (e.g., Sun et al. 2024 for APT adversariality) are background only and do not carry the robustness claim. The robustness numbers are measured with PGD/C&W attacks in Section 5.4 and are not obtained by fitting a parameter and re-predicting the same quantity. One scope caveat, not a circularity: Section 5.2 specifies only epsilon/iterations for PGD and does not state whether the evaluation attack is restricted to the same four byte regions used in training (Section 4.2, Appendix A). If the evaluation shares that mask, the reported ~80% robust accuracy is limited to those regions; that is an external-validity concern, not an identity between input and conclusion. No self-citation chain or definitional reduction was found.
Assumptions & free parameters
free parameters (6)
- K, number of global perturbation vectors =
50
- lambda_1, weight for Adversarial Contrastive Loss =
0.3
- lambda_2, weight for Adversarial Distribution Loss =
0.3
- tau, contrastive temperature =
0.6
- epsilon, maximum perturbation strength =
0.6
- Perturbation position sizes =
1KB shift, up to 100KB padding
assumptions (4)
- domain assumption The four perturbation regions (DOS header, shift, slack, padding) preserve malware functionality.
- ad hoc to paper Malware samples with similar perturbations share a common perturbation pattern in the embedding space.
- domain assumption The AMG18 APT group labels are correct.
- domain assumption The attacker has white-box access but cannot alter the trained model.
invented entities (1)
-
Global Perturbation (GP) pool
Cite this review
Pith. "Pith review of RoMA: Robust Malware Attribution via Byte-level Adversarial Training with Global Perturbations and Adversarial Consistency Regularization." pith.science (2026). https://pith.science/paper/XT3HRS2W
@misc{pith2026250207492,
author = {Pith},
title = {Pith review of: RoMA: Robust Malware Attribution via Byte-level Adversarial Training with Global Perturbations and Adversarial Consistency Regularization},
year = {2026},
howpublished = {\url{https://pith.science/paper/XT3HRS2W}},
note = {Machine review of arXiv:2502.07492}
}
read the original abstract
Attributing APT (Advanced Persistent Threat) malware to their respective groups is crucial for threat intelligence and cybersecurity. However, APT adversaries often conceal their identities, rendering attribution inherently adversarial. Existing machine learning-based attribution models, while effective, remain highly vulnerable to adversarial attacks. For example, the state-of-the-art byte-level model MalConv sees its accuracy drop from over 90% to below 2% under PGD (projected gradient descent) attacks. Existing gradient-based adversarial training techniques for malware detection or image processing were applied to malware attribution in this study, revealing that both robustness and training efficiency require significant improvement. To address this, we propose RoMA, a novel single-step adversarial training approach that integrates global perturbations to generate enhanced adversarial samples and employs adversarial consistency regularization to improve representation quality and resilience. A novel APT malware dataset named AMG18, with diverse samples and realistic class imbalances, is introduced for evaluation. Extensive experiments show that RoMA significantly outperforms seven competing methods in both adversarial robustness (e.g., achieving over 80% robust accuracy-more than twice that of the next-best method under PGD attacks) and training efficiency (e.g., more than twice as fast as the second-best method in terms of accuracy), while maintaining superior standard accuracy in non-adversarial scenarios.
Figures
Reference graph
Works this paper leans on
-
[1]
Structure and Interpretation of Computer Programs
Harold Abelson, Gerald Jay Sussman, and Julie Sussman. Structure and Interpretation of Computer Programs . MIT Press, Cambridge, Massachusetts, 1985
1985
-
[2]
Visual information extraction with Lixto
Robert Baumgartner, Georg Gottlob, and Sergio Flesca. Visual information extraction with Lixto . In Proceedings of the 27th International Conference on Very Large Databases , pages 119--128, Rome, Italy, September 2001. Morgan Kaufmann
2001
-
[3]
Brachman and James G
Ronald J. Brachman and James G. Schmolze. An overview of the KL-ONE knowledge representation system. Cognitive Science , 9(2):171--216, April--June 1985
1985
-
[4]
Hypertree decompositions and tractable queries
Georg Gottlob, Nicola Leone, and Francesco Scarcello. Hypertree decompositions and tractable queries. Journal of Computer and System Sciences , 64(3):579--627, May 2002
2002
-
[5]
Complexity results for nonmonotonic logics
Georg Gottlob. Complexity results for nonmonotonic logics. Journal of Logic and Computation , 2(3):397--425, June 1992
1992
-
[6]
Levesque
Hector J. Levesque. Foundations of a functional approach to knowledge representation. Artificial Intelligence , 23(2):155--212, July 1984
1984
-
[7]
Levesque
Hector J. Levesque. A logic of implicit and explicit belief. In Proceedings of the Fourth National Conference on Artificial Intelligence , pages 198--202, Austin, Texas, August 1984. American Association for Artificial Intelligence
1984
-
[8]
On the compilability and expressive power of propositional planning formalisms
Bernhard Nebel. On the compilability and expressive power of propositional planning formalisms. Journal of Artificial Intelligence Research , 12:271--315, 2000
2000
Show all 38 references
-
[9]
Malware Statistics & Trends Report
AV-TEST. Malware Statistics & Trends Report . Available:https://www.av-test.org/en/statistics/malware, 2023. Online [Accessed: 1-11-2024]
2023
-
[10]
Towards evaluating the robustness of neural networks
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp) , pages 39--57. Ieee, 2017
2017
-
[11]
Machine learning-enabled IoT security: Open issues and challenges under advanced persistent threats
Zhiyan Chen, Jinxin Liu, Yu Shen, Murat Simsek, Burak Kantarci, Hussein T Mouftah, and Petar Djukic. Machine learning-enabled IoT security: Open issues and challenges under advanced persistent threats . ACM Computing Surveys , 55(5):1--37, 2022
2022
-
[12]
Crowdstrike 2024 global threat report
CrowdStrike. Crowdstrike 2024 global threat report. Available:https://www.crowdstrike.com/global-threat-report/, 2024. Online [Accessed: 1-11-2024]
2024
-
[13]
``apt malware dataset"
cyber research. ``apt malware dataset". Available:https://github.com/cyber-research/APTMalware, 2019. Online [Accessed:1-1-2025]
2019
-
[14]
Adversarial exemples: A survey and experimental evaluation of practical attacks on machine learning for windows malware detection
Luca Demetrio, Scott E Coull, Battista Biggio, Giovanni Lagorio, Alessandro Armando, and Fabio Roli. Adversarial exemples: A survey and experimental evaluation of practical attacks on machine learning for windows malware detection. ACM Transactions on Privacy and Security (TOP...
2021
-
[15]
APTM alinsight: Identify and cognize apt malware based on system call information and ontology knowledge framework
Weijie Han, Jingfeng Xue, Yong Wang, Fuquan Zhang, and Xianwei Gao. APTM alinsight: Identify and cognize apt malware based on system call information and ontology knowledge framework. Information Sciences , 546:633--664, 2021
2021
-
[16]
Classifying malwares for identification of author groups
Jiwon Hong, Sanghyun Park, Sang-Wook Kim, Dongphil Kim, and Wonho Kim. Classifying malwares for identification of author groups. Concurrency and Computation: Practice and Experience , 30(3):e4197, 2018
2018
-
[17]
Adversarial malware binaries: Evading deep learning for malware detection in executables
Bojan Kolosnjaji, Ambra Demontis, Battista Biggio, Davide Maiorca, Giorgio Giacinto, Claudia Eckert, and Fabio Roli. Adversarial malware binaries: Evading deep learning for malware detection in executables. In 2018 26th European signal processing conference (EUSIPCO) , pages 5...
2018
-
[18]
Deep convolutional malware classifiers can learn from raw executables and labels only
Marek Kr c \'a l, Ond r ej S vec, Martin B \'a lek, and Otakar Ja s ek. Deep convolutional malware classifiers can learn from raw executables and labels only. In Proc. ICLRW , 2018
2018
-
[19]
Deep Convolutional Malware Classifiers Can Learn from Raw Executables and Labels Only
Marek Kr c \'a l, Ond r ej S vec, Martin B \'a lek, and Otakar Ja s ek. Deep Convolutional Malware Classifiers Can Learn from Raw Executables and Labels Only . 2018
2018
-
[20]
Deceiving end-to-end deep learning malware detectors using adversarial examples
Felix Kreuk, Assi Barak, Shir Aviv-Reuven, Moran Baruch, Benny Pinkas, and Joseph Keshet. Deceiving end-to-end deep learning malware detectors using adversarial examples. arXiv preprint arXiv:1802.04528 , 2018
2018 arXiv
-
[21]
Malware triage for early identification of advanced persistent threat activities
Giuseppe Laurenza, Riccardo Lazzeretti, and Luca Mazzotti. Malware triage for early identification of advanced persistent threat activities. Digital Threats: Research and Practice , 1(3):1--17, 2020
2020
-
[22]
Adversarial attacks against windows PE malware detection: A survey of the state-of-the-art
Xiang Ling, Lingfei Wu, Jiangyu Zhang, Zhenqing Qu, Wei Deng, Xiang Chen, Yaguan Qian, Chunming Wu, Shouling Ji, Tianyue Luo, et al. Adversarial attacks against windows PE malware detection: A survey of the state-of-the-art. Computers & Security , page 103134, 2023
2023
-
[23]
Functions-based CFG embedding for malware homology analysis
Jieran Liu, Yuan Shen, and Hanbing Yan. Functions-based CFG embedding for malware homology analysis. In 2019 26th International Conference on Telecommunications (ICT) , pages 220--226. IEEE, 2019
2019
-
[24]
Reiter, and Saurabh Shintre
Keane Lucas, Mahmood Sharif, Lujo Bauer, Michael K. Reiter, and Saurabh Shintre. Malware makeover: Breaking ml-based static analysis by modifying executable bytes. In Proceedings of the 2021 ACM Asia Conference on Computer and Communications Security , ASIA CCS '21, page 744–7...
2021
-
[25]
Reiter, and Mahmood Sharif
Keane Lucas, Samruddhi Pai, Weiran Lin, Lujo Bauer, Michael K. Reiter, and Mahmood Sharif. Adversarial training for Raw-Binary malware classifiers. In 32nd USENIX Security Symposium (USENIX Security 23) , pages 1163--1180, Anaheim, CA, August 2023. USENIX Association
2023
-
[26]
Towards deep learning models resistant to adversarial attacks
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 , 2017
2017 arXiv
-
[27]
``advisory: Turla group exploits iranian apt to expand coverage of victims"
NCSC and NSA. ``advisory: Turla group exploits iranian apt to expand coverage of victims". Available:https://www.ncsc.gov.uk/news/turla-group-exploits-iran-apt-to-expand-coverage-of-victims, 2019. Online [Accessed:1-11-2024]
2019
-
[28]
Intriguing Properties of Adversarial ML Attacks in the Problem Space
Fabio Pierazzi, Feargus Pendlebury, Jacopo Cortellazzi, and Lorenzo Cavallaro. Intriguing Properties of Adversarial ML Attacks in the Problem Space . In 2020 IEEE symposium on security and privacy (SP) , pages 1332--1349. IEEE, 2020
2020
-
[29]
Malware detection by eating a whole exe
Edward Raff, Jon Barker, Jared Sylvester, Robert Brandon, Bryan Catanzaro, and Charles K Nicholas. Malware detection by eating a whole exe. In Workshops at the thirty-second AAAI conference on artificial intelligence , 2018
2018
-
[30]
Classifying sequences of extreme length with constant memory applied to malware detection
Edward Raff, William Fleshman, Richard Zak, Hyrum S Anderson, Bobby Filar, and Mark McLean. Classifying sequences of extreme length with constant memory applied to malware detection. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 9386--9394, 2021
2021
-
[31]
Bin MLM : Binary Authorship Verification with Flow-aware Mixture-of-Shared Language Model
Qige Song, Yongzheng Zhang, Linshu Ouyang, and Yige Chen. Bin MLM : Binary Authorship Verification with Flow-aware Mixture-of-Shared Language Model . In 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) , pages 1023--1033. IEEE, 2022
2022
-
[32]
Towards efficient and effective adversarial training
Gaurang Sriramanan, Sravanti Addepalli, Arya Baburaj, et al. Towards efficient and effective adversarial training. Advances in Neural Information Processing Systems , 34:11821--11833, 2021
2021
-
[33]
Mgap3: Malware group attribution based on perceiverio and polytype pre-training
Yuxia Sun, Shiqi Chen, Song Lin, Aoxiang Sun, Saiqin Long, and Zhetao Li. Mgap3: Malware group attribution based on perceiverio and polytype pre-training. IEEE Transactions on Dependable and Secure Computing , pages 1--16, 2024
2024
-
[34]
The Cyberthreat Report
Trellix. The Cyberthreat Report . Available:https://www.trellix.com/en-us/advanced-research-center/threat-reports/jun-2023.html, 2024. Online [Accessed 1-11-2024]
2023
-
[35]
Visualizing data using t-sne
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research , 9(11), 2008
2008
-
[36]
VirusSign - Open Malware Database
VirusSign. VirusSign - Open Malware Database . virussign.com , 2024. (accessed July 27, 2024)
2024
-
[37]
Fast is better than free: Revisiting adversarial training
Eric Wong, Leslie Rice, and J Zico Kolter. Fast is better than free: Revisiting adversarial training. In International Conference on Learning Representations, ICLR , 2020
2020
-
[38]
write newline
" write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.