REVIEW 3 major objections 6 minor 1 cited by
A Comparison of Selected Image Transformation Techniques for Malware Classification
T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Eight malware-to-image conversions perform about equally, so the classifier matters more than the conversion.
desk verdict A useful but statistically thin benchmark; the main claim is plausible but the paper overstates it. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing device is a standardized comparison: every conversion takes the same first 50,176 bytes of each of 17,000 samples, produces a 224×224 image, and is evaluated under the same 80:10:10 train-validation-test split with the same Optuna hyperparameter search (random seed 42). This uniformity isolates the conversion method as the only variable, and the box-and-whisker plots over the eleven model/feature combinations per conversion are what support the claim of near-equivalence. Within this apparatus, the conversion techniques themselves—grayscale, Byteclass, Hilbert, Entropy, HIT, Spiral, Cartesian bigrams, and Polar bigrams—are the objects being compared.
What would settle it
Re-run the 92 experiments under ten different random seeds and compare the per-configuration accuracy ranges; if for any classifier grayscale and the other top conversions are not separated beyond the run-to-run spread, the paper's core claim is supported, whereas stable, non-overlapping gaps would show that conversion choice does matter.
Extended reading notes
Core claim
On the authors' own terms, the discovery is that eight substantially different executable-to-image conversion strategies produce overlapping accuracy ranges across eleven model/feature configurations, and the simplest one, grayscale, is the best or near-best in nearly every setting. The strongest single result is XGBoost with HOG features on grayscale images at 0.7512 accuracy, followed by Hilbert at 0.7424, Byteclass at 0.7306, and HIT at 0.7200, while the best baseline using byte histograms reaches only 0.6906. All image-based conversions except Spiral beat the histogram baseline, which the authors take as evidence that the image-based pipeline carries the discriminative signal. Because the conversions differ so much in how they lay out bytes, the authors argue the comparable results imply that the image analysis techniques themselves, rather than the conversion detail, are what drive the success.
Load-bearing premise
The load-bearing premise is that the accuracy numbers, each measured once on a single random 80/10/10 split with one seed, are stable enough that the differences between conversion techniques are real and not just run-to-run noise.
Editorial extensions
If this is right
- A simple grayscale conversion can serve as a default for image-based malware classification, since it matched or beat all other tested conversions in accuracy.
- Image-based representations outperform byte-histogram baselines across nearly all conversions, so the spatial arrangement of bytes carries information that histograms discard.
- Transfer-learning models are not all equal on this task: VGG16 consistently outperformed InceptionV3 and DenseNet121 across every conversion technique.
- Because conversion choice is secondary, the wide variety of ad hoc malware-to-image schemes in the literature can be treated as mostly interchangeable, and effort is better spent on classifiers and features.
- The histogram-based Spiral images underperform, showing that not every image conversion is equally useful and that basing an image on histogram features forfeits the advantage.
Reading between the lines
- One testable extension the authors do not run: randomly permute the byte-to-pixel mapping so local neighborhoods are destroyed, and retrain VGG16; if accuracy collapses, the conversion layout still contributes signal, and if accuracy holds, the image-ness itself is secondary in a way the paper does not isolate.
- Because Spiral is the only conversion built from histogram features and the only one that underperforms the histogram baseline, a per-family error analysis could reveal whether the spiral ordering hides or destroys the discriminative byte-value patterns.
- A practical corollary the authors leave implicit: ensemble classifiers over multiple conversions (for example, concatenating HOG features from grayscale and Hilbert images) may add little if the representations are redundant; testing this would quantify how much unique information each conversion carries.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper compares eight malware-to-image conversion techniques (Grayscale, Byteclass, Hilbert, Entropy, HIT, Spiral, Cartesian bigrams, and Polar bigrams) across seven classifiers (KNN, MLP, SVM, XGBoost, VGG16, InceptionV3, DenseNet121), with HOG and Haralick features for the four non-CNN models. The experiments use 17,000 samples from RawMalTF (17 families, 1000 per family) and an 80/10/10 single split. The main finding is that several conversion techniques yield similar classification accuracy, with Grayscale being the best overall, and that image-based approaches generally outperform byte-histogram baselines. The paper concludes that the specific image conversion strategy is secondary to the inherent strengths of image analysis techniques.
Significance. If the central claim were robust, this would be a practically useful result: it would support defaulting to simple grayscale conversion and focusing research effort on classifier improvements rather than bespoke conversion schemes. The paper is honest and internally consistent: the balanced-class dataset makes accuracy equal recall in every table, which the authors correctly note, and the dataset construction is clearly described. The comparison is broad, covering 92 configurations, and the qualitative ordering of techniques is visible in the reported tables. However, the evidentiary value is currently limited by the single-split, single-run design and the Spiral construction leak, which prevent the main similarity claim from being distinguished from run-to-run noise.
major comments (3)
- [§4.1, §5.2, Fig. 14] The central claim that conversion techniques 'perform similarly' is not supported by the reported single-run results. Each configuration is evaluated once on a fixed 80/10/10 split (random seed 42), with no confidence intervals, repeated splits, or significance tests. On the 1,700-sample test set, the binomial standard error of a 0.75 accuracy is approximately 0.0105, so the XGBoost/HOG gap between Grayscale (0.7512) and Hilbert (0.7424) is 0.0088—less than one standard error—and the Grayscale-to-HIT gap of 0.0312 is only about three standard errors. Without repeated trials or interval estimates, the observed similarity could equally be an artifact of underpowered measurement. The paper should report mean accuracies over multiple seeds or folds, or at least provide confidence intervals, before claiming that conversion strategy is a secondary factor.
- [§3.4.7, §5.2] The Spiral image construction is not a clean comparison arm because the byte-ordering is fitted on all dataset samples before the split. Section 3.4.7 states that Random Forest Gini importances are computed 'for all dataset samples,' which means test-set information influences the image representation used for evaluation. This leakage invalidates the Spiral accuracy numbers as an unbiased estimate, and because Spiral is the worst-performing conversion in Figures 11–14, the related claim that baseline histogram features underperform all image-based conversions except Spiral is also suspect. The authors should recompute the Gini ordering within each training fold only (or use nested cross-validation) and report whether the Spiral results change.
- [§6, Conclusion] The concluding assertion that 'the simplest (i.e., Grayscale) yielded the best results' is stronger than the evidence permits. In the best-performing model class (XGBoost with HOG features), Grayscale is not statistically separated from Hilbert or Byteclass given the single-run measurement noise discussed above. The authors should temper this claim or support it with inferential statistics showing that the top conversions are genuinely ordered.
minor comments (6)
- [§3.2.5, InceptionV3 paragraph] The phrase 'successful applied' should read 'successfully applied.'
- [§3.4.1] The sentence 'Each value in this two-dimensional array represent the luminosity of an image pixel' should be 'Each value ... represents the luminosity ...'
- [§3.4.4] The entropy construction is underspecified: the text says the entropy value at position i uses a block B=(b_i,...,b_{i+n}) with n=min{255,N-i}, but then states that the result is an array of entropy values x_0,...,x_255. Clarify how the number of window positions relates to the 224×224 canvas and how the block length n is used in the sum.
- [§3.4.7] The phrase 'we consider each byte position as a feature' should be 'each byte value as a feature' to avoid confusion with offsets in the executable file.
- [§4.1, §4.3] The train:test:validation order is given inconsistently (once as 'train:test:validation' and once as 'train:validation:test'). Also, the statement that accuracy equals recall because the dataset is balanced could explicitly note that the test set is balanced by construction (1000 samples per family).
- [Appendix B] The confusion matrices in Figures B.1–B.8 are very dense and difficult to read at the printed size; consider enlarging or providing a table of per-family recall values alongside them.
Circularity Check
Main comparison is self-contained; the only circular/leaky step is the Spiral arm, whose Gini-based byte ordering is fit on all dataset samples before the split.
-
fitted input called prediction
[Section 3.4.7 (Spiral), with the split defined in Section 4.1]
"For all dataset samples, we consider each byte position as a feature, and use Random Forest to compute the Gini value of each feature. Ranking from highest to lowest Gini importance, we obtain an order of importance for each byte."
The Spiral image construction is parameterized by a byte ordering selected via Random Forest Gini importances computed over 'all dataset samples' before the train/validation/test split described in Section 4.1. Because the family labels of the test samples are available during the fitting of this ordering, the Spiral images themselves are built using test-set information. When a classifier is then evaluated on those images, the Spiral arm is not a clean test of an image conversion technique; it is a representation fitted with test labels, with the fitted ordering then treated as a fixed property of the conversion method. This is a fitted input presented as part of the image transformation rather than as a prediction.
full rationale
The paper is an empirical comparison, not a derivation, and the central claim that several image conversion techniques perform similarly rests on accuracy tables and figures across 92 single-split configurations. No equation is defined in terms of another, and no fitted parameter is renamed as the main predicted quantity. The self-citations to prior MalImg, HIT4Mal, and spiral-visualization work are used as background and inspiration, not as a load-bearing uniqueness argument or as the source of the compared techniques. The single-run, single-split design is a statistical robustness concern (binomial standard errors on the 1,700-sample test set are close to several reported between-conversion gaps), but that is not circularity. The one genuinely self-referential step is the Spiral arm of Section 3.4.7, where the byte ordering is computed from all samples, leaking test labels into the image construction; this invalidates Spiral as a clean comparison arm. Because Spiral performs worst and the remaining seven conversions are constructed from raw bytes without any fitted parameters, the circularity is localized and does not drive the paper's main conclusion.
Assumptions & free parameters
free parameters (3)
- Per-method hyperparameters (Optuna-selected) =
Not reported in detail; sampled from ranges in Table 3
- CNN grid-search hyperparameters (learning rate, momentum) =
Grid: learning rate {0.0001, 0.001, 0.01}, momentum {0.9, 0.99}
- Spiral byte ordering from Random Forest Gini importances =
A 256-position permutation derived from the dataset
assumptions (4)
- domain assumption The first 50,176 bytes of a sample are sufficiently representative for malware family classification
- domain assumption The 17-family labels in RawMalTF are correct and meaningful
- domain assumption ImageNet-pretrained weights transfer to malware images
- standard math The encoding formulas from [42] are reproduced correctly
Cite this review
Pith. "Pith review of A Comparison of Selected Image Transformation Techniques for Malware Classification." pith.science (2026). https://pith.science/paper/KG7BJN5U
@misc{pith2026250910838,
author = {Pith},
title = {Pith review of: A Comparison of Selected Image Transformation Techniques for Malware Classification},
year = {2026},
howpublished = {\url{https://pith.science/paper/KG7BJN5U}},
note = {Machine review of arXiv:2509.10838}
}
read the original abstract
Recently, a considerable amount of malware research has focused on the use of powerful image-based machine learning techniques, which generally yield impressive results. However, before image-based techniques can be applied to malware, the samples must be converted to images, and there is no generally-accepted approach for doing so. The malware-to-image conversion strategies found in the literature often appear to be ad hoc, with little or no effort made to take into account properties of executable files. In this paper, we experiment with eight distinct malware-to-image conversion techniques, and for each, we test a variety of learning models. We find that several of these image conversion techniques perform similarly across a range of learning models, in spite of the image conversion processes being quite different. These results suggest that the effectiveness of image-based malware classification techniques may depend more on the inherent strengths of image analysis techniques, as opposed to the precise details of the image conversion strategy.
Figures
Figures from the paper (12 more)
Forward citations
Cited by 1 Pith paper
-
CAM-Guided Saliency Cutout and Image-Based Malware Classification
Saliency-guided cutout does not improve grayscale malware classification in controlled ResNet18 experiments, though low-saliency cutout slightly helps CIFAR-100, indicating domain-dependent value.
Reference graph
Works this paper leans on
-
[1]
Trojan-psw.msil.agensla.https://threats.kaspersky.com/en/threat/ Trojan-PSW.MSIL.Agensla/, 2025
work page 2025
-
[2]
Optuna: A next-generation hyperparameter optimization frame- work
Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next-generation hyperparameter optimization frame- work. InProceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’19, pages 2623–2631, 2019
work page 2019
-
[3]
Backdoor.win32.androm.https://threats.kaspersky.com/en/threat/ Backdoor.Win32.Androm/, 2025
work page 2025
-
[4]
Springer, 2006
John Aycock.Computer Viruses and Malware. Springer, 2006
2006
-
[5]
RawMal-TF: Raw malware dataset labeled by type and family.https://arxiv.org/abs/2506.23909, 2025
David B´ alik, Martin Jure˘ cek, and Mark Stamp. RawMal-TF: Raw malware dataset labeled by type and family.https://arxiv.org/abs/2506.23909, 2025
arXiv 2025
-
[6]
Binod Bhattarai, Ronast Subedi, Rebati Raman Gaire, Eduard Vazquez, and Danail Stoyanov. Histogram of oriented gradients meet deep learning: A novel multi-task deep network for 2D surgical image semantic segmentation. Medical Image Analysis, 85:102747, 2023
work page 2023
-
[7]
Transfer learning for image-based malware classification
Niket Bhodia, Pratikkumar Prajapati, Fabio Di Troia, and Mark Stamp. Transfer learning for image-based malware classification. In Paolo Mori, Steven Furnell, and Olivier Camp, editors,Proceedings of the 5th Interna- tional Conference on Information Systems Security and Privacy, ICISSP, pages 719–726, 2019
work page 2019
-
[8]
Haralick texture features.https://murphylab
Michael Boland. Haralick texture features.https://murphylab. web.cmu.edu/publications/boland/boland_node26.html#sec: cho_methods_haralick, 1999
work page 1999
Show all 45 references
-
[9]
A comprehensive survey on support vector machine classifi- cation: Applications, challenges and trends.Neurocomputing, 408:189–215, 2020
Jair Cervantes, Farid Garcia-Lamont, Lisbeth Rodr´ ıguez-Mazahua, and As- drubal Lopez. A comprehensive survey on support vector machine classifi- cation: Applications, challenges and trends.Neurocomputing, 408:189–215, 2020. 20
2020
-
[10]
XGBoost: A scalable tree boosting sys- tem
Tianqi Chen and Carlos Guestrin. XGBoost: A scalable tree boosting sys- tem. InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, pages 785–794, 2016
2016
-
[11]
Backdoor.win32.convagent.abq.https://threats.kaspersky.com/en/ threat/Backdoor.Win32.Convagent.abq/, 2025
2025
-
[12]
Hacktool.win32.crypt.au.https://threats.kaspersky.com/en/threat/ HackTool.Win32.Crypt.au/, 2025
2025
-
[13]
Backdoor.win32.crysan.gen.https://threats.kaspersky.com/en/ threat/VHO:Backdoor.Win32.Crysan.gen/, 2025
2025
-
[14]
K-nearest neighbour classi- fiers — A tutorial.ACM Computing Surveys, 54(6):1–25, 2021
P´ adraig Cunningham and Sarah Jane Delany. K-nearest neighbour classi- fiers — A tutorial.ACM Computing Surveys, 54(6):1–25, 2021
2021
-
[15]
Backdoor.msil.dcrat.aae.https://threats.kaspersky.com/en/threat/ Backdoor.MSIL.DCRat.aae/, 2025
2025
-
[16]
Haralick, K
Robert M. Haralick, K. Shanmugam, and Its’Hak Dinstein. Textural fea- tures for image classification.IEEE Transactions on Systems, Man, and Cybernetics, SMC-3(6):610–621, 1973
1973
-
[17]
hilbertcurve 2.0.5.https://pypi.org/project/hilbertcurve/, 2025
2025
-
[18]
Wein- berger
Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Wein- berger. Densely connected convolutional networks. In2017 IEEE Con- ference on Computer Vision and Pattern Recognition, CVPR, pages 2261– 2269, 2017
2017
-
[19]
ImageNet.https://image-net.org/, 2021
2021
-
[20]
Trojan.win32.injuke.gen.https://threats.kaspersky.com/en/threat/ HEUR:Trojan.Win32.Injuke.gen/, 2025
2025
-
[21]
Rendering along the Hilbert curve
Alexander Keller, Carsten W¨ achter, and Nikolaus Binder. Rendering along the Hilbert curve. In Zdravko Botev, Alexander Keller, Christiane Lemieux, and Bruno Tuffin, editors,Advances in Modeling and Simulation, pages 319–
-
[22]
Image-based malware classification using qr and aztec codes
Atharva Khadilkar and Mark Stamp. Image-based malware classification using qr and aztec codes. In Mark Stamp and M. Jure˘ cek, editors,Machine Learning, Deep Learning, and AI for Cybersecurity, pages 3–35. Springer, 2025
2025
-
[23]
Trojan.win32.makoob.gen.https://threats.kaspersky.com/en/ threat/HEUR:Trojan.Win32.Makoob.gen/, 2025
2025
-
[24]
Matplotlib: Visualization with Python.https://matplotlib.org/, 2025
2025
-
[25]
Backdoor.win32.mokes.https://threats.kaspersky.com/en/threat/ Backdoor.Win32.Mokes/, 2025
2025
-
[26]
Nataraj, S
L. Nataraj, S. Karthikeyan, G. Jacob, and B. S. Manjunath. Malware im- ages: Visualization and automatic classification. InProceedings of the 8th International Symposium on Visualization for Cyber Security, VizSec ’11, 2011. 21
2011
-
[27]
Generative adversarial networks and image-based malware classification.Journal of Computer Virology and Hacking Techniques, 19(4):579–595, 2023
Huy Nguyen, Fabio Di Troia, Genya Ishigaki, and Mark Stamp. Generative adversarial networks and image-based malware classification.Journal of Computer Virology and Hacking Techniques, 19(4):579–595, 2023
2023
-
[28]
Trojan-spy.win32.noon.https://threats.kaspersky.com/en/threat/ Trojan-Spy.Win32.Noon/, 2025
2025
-
[29]
An introduction to convolutional neural networks.https://arxiv.org/abs/1511.08458, 2015
Keiron O’Shea and Ryan Nash. An introduction to convolutional neural networks.https://arxiv.org/abs/1511.08458, 2015
2015 arXiv
-
[30]
Multilayer perceptron and neural networks.WSEAS Transactions on Circuits and Systems, 8, 2009
Marius-Constantin Popescu, Valentina Balas, Liliana Perescu-Popescu, and Nikos Mastorakis. Multilayer perceptron and neural networks.WSEAS Transactions on Circuits and Systems, 8, 2009
2009
-
[31]
An empirical analysis of image- based learning techniques for malware classification
Pratikkumar Prajapati and Mark Stamp. An empirical analysis of image- based learning techniques for malware classification. In Mark Stamp, Mamoun Alazab, and Andrii Shalaginov, editors,Malware Analysis Using Artificial Intelligence and Deep Learning, pages 411–435. Springer, 2021
2021
-
[32]
Backdoor.win32.remcos.aaaa.https://threats.kaspersky.com/en/ threat/Backdoor.Win32.Remcos.aaaa/, 2025
2025
-
[33]
Darknet traffic clas- sification and adversarial attacks using machine learning.Computers & Security, 127:103098, 2023
Nhien Rust-Nguyen, Shruti Sharma, and Mark Stamp. Darknet traffic clas- sification and adversarial attacks using machine learning.Computers & Security, 127:103098, 2023
2023
-
[34]
Trojan-downloader.win32.seraph.gen.https://threats.kaspersky.com/ en/threat/HEUR:Trojan-Downloader.Win32.Seraph.gen/, 2025
2025
-
[35]
Very deep convolutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. InInternational Conference on Learning Representations, ICLR, 2015
2015
-
[36]
Trojan-spy.win32.snakelogger.gen.https://threats.kaspersky.com/en/ threat/HEUR:Trojan-Spy.Win32.SnakeLogger.gen/, 2025
2025
-
[37]
Trojan-psw.win32.stealerc.gen.https://threats.kaspersky.com/en/ threat/HEUR:Trojan-PSW.Win32.Stealerc.gen/, 2025
2025
-
[38]
Trojan.win32.strab.gen.https://threats.kaspersky.com/en/threat/ HEUR:Trojan.Win32.Strab.gen/, 2025
2025
-
[39]
Going deeper with convolutions
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Ra- binovich. Going deeper with convolutions. In2015 IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pages 1–9, 2015
2015
-
[40]
Trojan-downloader.win32.taskun.gen.https://threats.kaspersky.com/ en/threat/HEUR:Trojan-Downloader.Win32.Taskun.gen/, 2025
2025
-
[41]
Simple binary data visualization.https://martin.varela
Mart´ ın Varela. Simple binary data visualization.https://martin.varela. fi/2017/09/09/simple-binary-data-visualization/, 2017
2017
-
[42]
Nguyen, Tu N
Duc-Ly Vu, Trong-Kha Nguyen, Tam V. Nguyen, Tu N. Nguyen, Fabio Massacci, and Phu H. Phung. HIT4Mal: Hybrid image transformation 22 for malware classification.Transactions on Emerging Telecommunications Technologies, 31(11):e3789, 2020
2020
-
[43]
Guoqing Xiao, Jingning Li, Yuedan Chen, and Kenli Li. MalFCS: An ef- fective malware classification framework with automated feature extraction based on deep convolutional neural networks.Journal of Parallel and Dis- tributed Computing, 141:49–58, 2020
2020
-
[44]
Deep learning versus gist descriptors for image-based malware classi- fication
Sravani Yajamanam, Vikash Raja Samuel Selvin, Fabio Di Troia, and Mark Stamp. Deep learning versus gist descriptors for image-based malware classi- fication. In Paolo Mori, Steven Furnell, and Olivier Camp, editors,Proceed- ings of the 4th International Conference on Informati...
2018
-
[45]
Appendix A In this appendix, we provide detailed results for each experiment we have con- ducted
Trojan.win32.zenpak.gen.https://threats.kaspersky.com/en/threat/ HEUR:Trojan.Win32.Zenpak.gen/, 2025. Appendix A In this appendix, we provide detailed results for each experiment we have con- ducted. Table A.1 contains the results of our baseline experiments. Tables A.2 throug...
2025
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.