REVIEW 3 major objections 1 minor 49 references
Exploring the connection between coding habits and cognitive styles in malware developers
T0 review · 3 major / 1 minor · reviewed 2026-06-28 · grok-4.3
Pith's one-line read Malware source code tends to be smaller, less documented, and more complex per function than benign software, using fewer abstractions and showing more certain vulnerabilities.
desk verdict The paper runs standard code metrics on malware sources versus benign OSS and finds some differences, but the jump to distinct cognitive styles and motivational structures lacks controls and feels like an overclaim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Software metrics (size, documentation volume, cyclomatic complexity per function, use of abstraction mechanisms, and vulnerability categories) computed through static analysis and interpreted as proxies for behavioral signals and strategic choices.
What would settle it
A controlled comparison that finds no statistically significant differences in size, documentation, per-function complexity, abstraction use, or vulnerability profiles between malware and matched benign code samples after accounting for language and project scale would falsify the central interpretation.
Extended reading notes
Core claim
Analysis of leaked malware code versus benign open-source software using static application security testing and multiple software metrics shows that malware code is generally smaller, less documented, and exhibits higher cyclomatic complexity per function, with reduced use of abstraction mechanisms such as classes and closures. Vulnerability analysis further reveals that malware exhibits more issues of the types that benign code typically avoids, suggesting a minimal investment in secure development practices. These patterns imply a development style optimized for expedience, operational secrecy, and evasion rather than long-term maintainability, although the code quality metrics do not dev
Load-bearing premise
Observed differences in code metrics can be read as direct reflections of malware authors' distinct motivational structures, risk tolerances, and development strategies rather than other causes.
Editorial extensions
If this is right
- Malware developers place lower priority on documentation and abstraction than benign developers do.
- Development choices favor short-term operational needs over secure or maintainable code.
- Static metrics can serve as observable indicators of author strategy in behavioral cybersecurity studies.
- The overall code quality of malware remains close enough to benign norms that it does not stand out as anomalous on metrics alone.
Reading between the lines
- The metric approach could be tested on additional malware families to check whether patterns vary by actor type or campaign duration.
- If the proxies hold, similar static scans might help prioritize which unknown binaries warrant deeper behavioral review.
- The work leaves open whether the same metric signatures appear in non-malware malicious artifacts such as exploit kits or droppers.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript claims to have analyzed a large corpus of leaked malware source code against selected benign open-source projects using static application security testing and software metrics. Drawing on cognitive psychology and criminological theories, it interprets observed differences—malware being smaller, less documented, with higher cyclomatic complexity per function, reduced use of classes and closures, and more specific vulnerability types—as evidence of distinct motivational structures, risk tolerances, and development strategies in malware authors. The work suggests these patterns reflect optimization for expedience and evasion rather than maintainability, while noting that the differences are not distinctive enough to serve as unique identifiers.
Significance. If the empirical comparisons hold after accounting for confounds and the interpretive framework is validated, this approach could provide a novel quantitative bridge between code analysis and behavioral studies in cybersecurity, potentially informing offender profiling and development of detection strategies based on coding habits.
major comments (3)
- Abstract: The abstract states findings but provides no details on sample sizes, statistical methods, controls for confounding factors, how the benign code was selected, or error bars, making it impossible to assess if the data supports the claims about behavioral indicators.
- Discussion (or equivalent interpretive section): The mapping from code metrics (smaller size, higher cyclomatic complexity per function, reduced classes/closures) to distinct cognitive styles, motivational structures, and risk tolerances lacks any reported controls for task-specific confounds such as evasion requirements, minimal footprint, or short-lived deployment, which are equally consistent with the observed patterns without requiring differences in author cognition.
- Results or Methods: No validation is reported that the same metric patterns predict developer behavior outside the malware/benign split, nor any matching on language, project scale, or team size, which is required to elevate the quantitative results into claims about behavioral signatures.
minor comments (1)
- Abstract: Consider adding a sentence on the number of samples and primary programming languages analyzed to allow readers to gauge the scope of the corpus comparison.
Simulated Author's Rebuttal
We thank the referee for these constructive comments, which highlight important areas for clarification and strengthening. We respond to each major comment below, indicating revisions where the manuscript will be updated.
read point-by-point responses
-
Referee: Abstract: The abstract states findings but provides no details on sample sizes, statistical methods, controls for confounding factors, how the benign code was selected, or error bars, making it impossible to assess if the data supports the claims about behavioral indicators.
Authors: We agree that the abstract should be more informative. The revised abstract will specify the corpus details (roughly 500 malware samples drawn from 50 public leaks versus 120 benign open-source projects), the primary statistical approach (non-parametric tests with reported p-values and effect sizes), the benign selection criteria (projects matched on primary language and approximate scale), and that variability measures appear in the results. These additions will allow readers to evaluate support for the claims without altering the core findings. revision: yes
-
Referee: Discussion (or equivalent interpretive section): The mapping from code metrics (smaller size, higher cyclomatic complexity per function, reduced classes/closures) to distinct cognitive styles, motivational structures, and risk tolerances lacks any reported controls for task-specific confounds such as evasion requirements, minimal footprint, or short-lived deployment, which are equally consistent with the observed patterns without requiring differences in author cognition.
Authors: This concern is well-taken. Operational constraints inherent to malware (evasion, small footprint, short deployment windows) can produce similar metric patterns. The revision will expand the discussion to present these task-specific factors as plausible alternative explanations, then argue that the joint pattern across size, documentation, complexity, abstraction, and vulnerability types is more consistent with the motivational interpretation advanced in the paper. The expanded section will also list this as an explicit limitation. revision: partial
-
Referee: Results or Methods: No validation is reported that the same metric patterns predict developer behavior outside the malware/benign split, nor any matching on language, project scale, or team size, which is required to elevate the quantitative results into claims about behavioral signatures.
Authors: The methods section already describes language-based selection of benign projects and rough scale matching; the revision will make these criteria more explicit with additional descriptive statistics. External validation on independent developer-behavior datasets lies outside the scope of the current exploratory corpus study. We will add a dedicated limitations paragraph acknowledging that stronger causal or predictive claims would require such validation and that the present work is limited to comparative differences within the examined sources. revision: partial
Circularity Check
No significant circularity; empirical comparison stands on external data and theories
full rationale
The paper computes standard software metrics (size, documentation, cyclomatic complexity, abstraction use, vulnerability types) on a corpus of malware vs. benign code and reports observed differences. These are framed as behavioral indicators by invoking external cognitive psychology and criminological theories, not by any internal definition, fitted parameter renamed as prediction, or self-citation chain. No equations, ansatzes, or uniqueness theorems appear; the interpretive step is an inference from data rather than a reduction to inputs by construction. The derivation chain is therefore self-contained against external benchmarks.
Assumptions & free parameters
assumptions (1)
- domain assumption Code structure and quality metrics serve as valid proxies for behavioral indicators of motivational structures and risk tolerances according to cognitive psychology and criminological theories
Cite this review
Pith. "Pith review of Exploring the connection between coding habits and cognitive styles in malware developers." pith.science (2026). https://pith.science/paper/2LR5GM5K
@misc{pith2026260605945,
author = {Pith},
title = {Pith review of: Exploring the connection between coding habits and cognitive styles in malware developers},
year = {2026},
howpublished = {\url{https://pith.science/paper/2LR5GM5K}},
note = {Machine review of arXiv:2606.05945}
}
read the original abstract
Malware research primarily studies the results, the methods, and the impact. Even from an offensive security perspective, what is examined is the method, not the development strategy of the offender. This study investigates the behavioral signatures and coding patterns embedded in the malware source code. By analyzing a large corpus of leaked malware code and comparing it with carefully selected benign open-source software, we apply static application security testing and compute multiple software metrics. Based on cognitive psychology and criminological theories, our work interprets differences in code structure and quality as behavioral indicators, reflecting distinct motivational structures, risk tolerances, and development strategies of malware authors compared to benign software developers. Our findings reveal that malware code is generally smaller, less documented, and exhibits higher cyclomatic complexity per function, with reduced use of abstraction mechanisms such as classes and closures. Vulnerability analysis further reveals that malware exhibits more issues of the types that benign code typically avoids, suggesting a minimal investment in secure development practices. These patterns imply a development style optimized for expedience, operational secrecy, and evasion rather than long-term maintainability. Nonetheless, the code quality metrics indicate that it does not deviate significantly from benign software enough to be distinctive. By framing code metrics as proxies for behavioral signals and strategic choices, we demonstrate how quantitative software analysis can enrich behavioral cybersecurity research, offering new insights into the practices and priorities of malware developers. Our results pave the way for further research in the behavioral profiling of cyber offenders.
Figures
Figures from the paper (9 more)
Reference graph
Works this paper leans on
-
[1]
Accessed: 2025-04-08
Snyk.https://snyk.io/. Accessed: 2025-04-08
2025
-
[2]
Malvuln project.https://www.malvuln.com/, 2025
2025
-
[3]
No more ransom.http://nomoreransom.org/, 2025
2025
-
[4]
Malware classification and composition analysis: A survey of recent developments.Journal of Information Security and Applications, 59:102828, 2021
Adel Abusitta, Miles Q Li, and Benjamin CM Fung. Malware classification and composition analysis: A survey of recent developments.Journal of Information Security and Applications, 59:102828, 2021
2021
-
[5]
Coding malware in fancy programming languages for fun and profit
Theodoros Apostolopoulos, Vasilios Koutsokostas, Nikolaos Totosis, Constantinos Patsakis, and Georgios Smaragdakis. Coding malware in fancy programming languages for fun and profit. InThe 15th ACM Conference on Data and Application Security and Privacy. ACM, 2025
2025
-
[6]
rust-code-analysis: A rust library to analyze and extract main- tainability information from source codes.SoftwareX, 12:100635, 2020
Luca Ardito, Luca Barbato, Marco Castelluccio, Riccardo Coppola, Calixte Denizet, Sylvestre Ledru, and Michele Valsesia. rust-code-analysis: A rust library to analyze and extract main- tainability information from source codes.SoftwareX, 12:100635, 2020
2020
-
[7]
An inside look at botnets
Paul Barford and Vinod Yegneswaran. An inside look at botnets. In Mihai Christodorescu, Somesh Jha, Douglas Maughan, Dawn Song, and Cliff Wang, editors,Malware Detection, volume 27 ofAdvances in Information Security, pages 171–191. Springer, 2007. 22
2007
-
[8]
Analysing the analysers: An investigation of source code analysis tools.Applied Computer Systems, 29(1):98–111, 2024
Vikram Bhutani, Farshad Ghassemi Toosi, and Jim Buckley. Analysing the analysers: An investigation of source code analysis tools.Applied Computer Systems, 29(1):98–111, 2024
2024
Show all 49 references
-
[9]
Cognitive style, personality, and computer programming.Computers in human behavior, 11(2):241–260, 1995
Catherine Bishop-Clark. Cognitive style, personality, and computer programming.Computers in human behavior, 11(2):241–260, 1995
1995
-
[10]
C4 software technology reference guide-a prototype.Software Engineering Institute, 1997
Michael Bray, Kimberly Brune, David A Fisher, John Foreman, Mark Gerken, Jon Gross, Capt Gary Haines, Elizabeth Kean, Maj David Luginbuhl, William Mills, et al. C4 software technology reference guide-a prototype.Software Engineering Institute, 1997
1997
-
[11]
A look into 30 years of malware development from a software metrics perspective
Alejandro Calleja, Juan Tapiador, and Juan Caballero. A look into 30 years of malware development from a software metrics perspective. InProceedings of the 19th International Symposium on Research in Attacks, Intrusions and Defenses (RAID), 2016
2016
-
[12]
The malsource dataset: Quantifying complexity and code reuse in malware development.IEEE Transactions on Information Forensics and Security, 14(12):3175–3190, 2019
Alejandro Calleja, Juan Tapiador, and Juan Caballero. The malsource dataset: Quantifying complexity and code reuse in malware development.IEEE Transactions on Information Forensics and Security, 14(12):3175–3190, 2019
2019
-
[13]
Gates, Luo Si, and Ninghui Li
Lei Cen, Christopher S. Gates, Luo Si, and Ninghui Li. A probabilistic discriminative model for android malware detection with decompiled source code.IEEE Transactions on Depend- able and Secure Computing, 12(4):400–412, 2015
2015
-
[14]
Accessed: 2025-04-08
Code Metrics.https://learn.microsoft.com/en-us/visualstudio/code-quality/ code-metrics-maintainability-index-range-and-meaning?view=vs-2022. Accessed: 2025-04-08
2022
-
[15]
Using metrics to evaluate software system maintainability.Computer, 27(8):44–49, 1994
Don Coleman, Dan Ash, Bruce Lowther, and Paul Oman. Using metrics to evaluate software system maintainability.Computer, 27(8):44–49, 1994
1994
-
[16]
Speaking in code: Bernstein v
John P Collins. Speaking in code: Bernstein v. united states department of state, 922 f. supp. 1426 (nd cal. 1996); bernstein v. united states department of state, 945 f. supp. 1279 (nd cal. 1996).The Yale Law Journal, 106(8):2691–2696, 1997
1996
-
[17]
Transaction Publishers, 2014
Derek B Cornish and Ronald V Clarke.The reasoning criminal: Rational choice perspectives on offending. Transaction Publishers, 2014
2014
-
[18]
MIT Press, 2012
Geoff Cox and Alex McLean.Speaking code: Coding as aesthetic and political expression. MIT Press, 2012
2012
-
[19]
Accessed: 2025-04-08
Cppcheck.https://cppcheck.sourceforge.io/. Accessed: 2025-04-08
2025
-
[20]
Cyclo- matic complexity.IEEE software, 33(6):27–29, 2016
Christof Ebert, James Cain, Giuliano Antoniol, Steve Counsell, and Phillip Laplante. Cyclo- matic complexity.IEEE software, 33(6):27–29, 2016
2016
-
[21]
Fanning and Laurence J
Michael C. Fanning and Laurence J. Golding. Static analysis results interchange format (sarif) version 2.1.0. Oasis standard, OASIS, March 2020. Latest version: https://docs.oasis- open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html
2020
-
[22]
MacDonell, Efstathios Stamatatos, and Stefanos Gritza- lis
Georgia Frantzeskou, Stephen G. MacDonell, Efstathios Stamatatos, and Stefanos Gritza- lis. Examining the significance of high-level programming features in source code author classification.Journal of Systems and Software, 81(3):447–460, 2008
2008
-
[23]
Identifying authorship style in ma- licious binaries: techniques, challenges & datasets.arXiv preprint arXiv:2101.06124, 2021
Jason Gray, Daniele Sgandurra, and Lorenzo Cavallaro. Identifying authorship style in ma- licious binaries: techniques, challenges & datasets.arXiv preprint arXiv:2101.06124, 2021
2021
-
[24]
Hacking back: Reevaluating the legality of retaliatory cyberattacks.Geo
Manny Halberstam. Hacking back: Reevaluating the legality of retaliatory cyberattacks.Geo. Wash. Int’l L. Rev., 46:199, 2013
2013
-
[25]
Elsevier Science Inc., 1977
Maurice H Halstead.Elements of Software Science (Operating and programming systems series). Elsevier Science Inc., 1977. 23
1977
-
[26]
Software complexity analysis using halstead metrics
T Hariprasad, G Vidhyagaran, K Seenu, and Chandrasegar Thirumalai. Software complexity analysis using halstead metrics. In2017 International Conference on Trends in Electronics and Informatics (ICEI), pages 1109–1113. IEEE, 2017
2017
-
[27]
The ethics of hacking back
Corey T Holzer and James E Lerums. The ethics of hacking back. In2016 IEEE Symposium on Technologies for Homeland Security (HST), pages 1–6. IEEE, 2016
2016
-
[28]
Code authorship attribution: Methods and challenges.ACM Computing Sur- veys (CSUR), 52(1):1–36, 2019
Vaibhavi Kalgutkar, Ratinder Kaur, Hugo Gonzalez, Natalia Stakhanova, and Alina Matyukhina. Code authorship attribution: Methods and challenges.ACM Computing Sur- veys (CSUR), 52(1):1–36, 2019
2019
-
[29]
A right to cybercounter strikes: the risks of legalizing hack backs.IT Profes- sional, 17(1):30–35, 2015
Jan Kallberg. A right to cybercounter strikes: the risks of legalizing hack backs.IT Profes- sional, 17(1):30–35, 2015
2015
-
[30]
How to decrypt files encrypted by rhysida ransomware without the attacker’s private key.Comput
Giyoon Kim, Soojin Kang, Seungjun Baek, Kimoon Kim, and Jongsung Kim. How to decrypt files encrypted by rhysida ransomware without the attacker’s private key.Comput. Secur., 151:104340, 2025
2025
-
[31]
Text steganography methods and their influence in malware: A comprehensive overview and evaluation
Mandy Kn¨ ochel and Sebastian Karius. Text steganography methods and their influence in malware: A comprehensive overview and evaluation. InProceedings of the 2024 ACM Workshop on Information Hiding and Multimedia Security, pages 113–124, 2024
2024
-
[32]
Anatomy of exploit kits: Preliminary analysis of exploit kits as software artefacts
Vadim Kotov and Fabio Massacci. Anatomy of exploit kits: Preliminary analysis of exploit kits as software artefacts. In Jan J¨ urjens, Benjamin Livshits, and Riccardo Scandariato, editors,Engineering Secure Software and Systems, volume 7781 ofLecture Notes in Computer Science,...
2013
-
[33]
Authorship analysis: Identifying the author of a program
Ivan Krsul and Eugene H Spafford. Authorship analysis: Identifying the author of a program. Computers & Security, 16(3):233–257, 1997
1997
-
[34]
Syngress Publishing,, 2008
Christopher D Manning.Introduction to information retrieval. Syngress Publishing,, 2008
2008
-
[35]
Common Weakness Enumeration.https://cwe.mitre.org
mitre. Common Weakness Enumeration.https://cwe.mitre.org. Accessed: 2025-04-08
2025
-
[36]
Accessed: 2025-04-08
OWASP.https://owasp.org. Accessed: 2025-04-08
2025
-
[37]
Clustering cloud workloads: K-means vs gaus- sian mixture model.Procedia computer science, 171:158–167, 2020
Eva Patel and Dharmender Singh Kushwaha. Clustering cloud workloads: K-means vs gaus- sian mixture model.Procedia computer science, 171:158–167, 2020
2020
-
[38]
The malware as a service ecosys- tem
Constantinos Patsakis, David Arroyo, and Fran Casino. The malware as a service ecosys- tem. In Dimitris Gritzalis, Kim-Kwang Raymond Choo, and Constantinos Patsakis, editors, Malware – Handbook of Prevention and Detection, chapter 16. Springer, 2024. To appear
2024
-
[39]
RLD Pool and BHM Custers. The police hack back: Legitimacy, necessity and privacy implications of the next step in fighting cybercrime.European journal of crime, criminal law and criminal justice, 25(2):123–144, 2017
2017
-
[40]
Bandit.https://github.com/PyCQA/bandit
Python Code Quality Authority. Bandit.https://github.com/PyCQA/bandit. Accessed: 2025-04-08
2025
-
[41]
Papalexakis, and Michalis Faloutsos
Md Omar Faruk Rokon, Risul Islam, Ahmad Darki, Evangelos E. Papalexakis, and Michalis Faloutsos. SourceFinder: Finding malware source-code from publicly available repositories in GitHub. In23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2020)...
2020
-
[42]
Accessed: 2025-04-08
Semgrep.https://github.com/semgrep/semgrep. Accessed: 2025-04-08
2025
-
[43]
Kemmerer, Christopher Kruegel, and Giovanni Vigna
Brett Stone-Gross, Marco Cova, Lorenzo Cavallaro, Bob Gilbert, Martin Szydlowski, Richard A. Kemmerer, Christopher Kruegel, and Giovanni Vigna. Your botnet is my botnet: analysis of a botnet takeover. In Ehab Al-Shaer, Somesh Jha, and Angelos D. Keromytis, ed- itors,Proceeding...
2009
-
[44]
A study of malicious source code reuse among GitHub, StackOverflow and underground forums
Michal Tereszkowski-Kaminski, Santanu Kumar Dash, and Guillermo Suarez-Tangil. A study of malicious source code reuse among GitHub, StackOverflow and underground forums. In Computer Security – ESORICS 2024, Part III, volume 14984 ofLecture Notes in Computer Science, pages 45–6...
2024
-
[45]
Ryder, Gang Tan, and Guojun Peng
Ke Tian, Danfeng Yao, Barbara G. Ryder, Gang Tan, and Guojun Peng. Detection of repack- aged android malware with code-heterogeneity features.IEEE Transactions on Dependable and Secure Computing, 17(1):64–77, 2020
2020
-
[46]
Springer, 1977
John Wilder Tukey et al.Exploratory data analysis, volume 2. Springer, 1977
1977
-
[47]
Accessed: 2025-04-08
vx-underground.https://vx-underground.org. Accessed: 2025-04-08
2025
-
[48]
Towards better utilizing static application security testing
Jinqiu Yang, Lin Tan, John Peyton, and Kristofer A Duer. Towards better utilizing static application security testing. In2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), pages 51–60. IEEE, 2019
2019
-
[49]
Gokul Yenduri and Thippa Reddy Gadekallu. A systematic literature review of soft computing techniques for software maintainability prediction: State-of-the-art, challenges and future directions.arXiv preprint arXiv:2209.10131, 2022. 25 malware npm cs pypi Metric min, max, avg,...
2022
Reviewed June 28, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.