REVIEW 2 major objections 5 minor 57 references
Towards usable automated detection of CPU architecture and endianness for arbitrary binary files and object code sequences
T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A classifier trained on whole binary files identifies CPU architecture with 99.2% accuracy; code-section-only training still exceeds 98%.
desk verdict The code-section reproduction and the open dataset are the real contributions; the 99.2% complete-binary headline is not controlled for ELF header leakage. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is a 293-dimensional feature vector: 256 byte-frequency values, 4 endianness marker signatures, 31 function prologue and epilogue byte signatures, and 2 new powerpcspe signatures. The byte frequencies give each architecture a characteristic distribution, the endianness markers capture byte-order conventions, and the prologue and epilogue signatures capture coding idioms that differ from CPU to CPU. The paper's novel move is to compute these features over the whole executable file rather than the code section only, and to train the classifiers the same way; the Random Forest that does this reaches the paper's headline 99.2% accuracy.
What would settle it
Zero out or remove the architecture field from the headers of the complete-binary test set and repeat the 99.2% experiment; if accuracy falls sharply, the classifier was using header metadata rather than code bytes.
Extended reading notes
Core claim
The paper's central claim is that byte-level statistics of a complete executable file are enough to identify both the instruction set architecture and the endianness, and that the classifier works even better when trained on whole files instead of extracted code sections. In a 23-architecture corpus of 67,285 samples, a Random Forest trained and tested on complete binaries reached 99.2% accuracy, the best result in the paper. When training and testing were restricted to code sections, every classifier still exceeded 98% accuracy, which the paper presents as an independent confirmation of previous findings. The paper also contributes two byte-signatures for the powerpcspe architecture that raise the F1 score for PowerPC classes by about 2 percentage points.
Load-bearing premise
The 99.2% result assumes that testing on complete executable files, whose headers openly record the target architecture, tells us how the classifier would perform on the headerless and fragmented object code the paper is ultimately about.
Editorial extensions
If this is right
- Architecture identification on complete executable files can reach 99.2% accuracy with a Random Forest, which is high enough for automated triage of firmware and malware corpora.
- Code-section-only classifiers match prior results above 98%, so the byte-frequency-plus-signature approach generalizes beyond the original datasets.
- Fragments as short as 128 bytes are classified with roughly 90% accuracy by several classifiers, extending the method toward shellcode and memory-dump use cases.
- The two new powerpcspe signatures can be reused by disassembler and firmware-analysis tools to distinguish PowerPC from PowerPC SPE code.
Reading between the lines
- If the 99.2% complete-file accuracy comes largely from reading the architecture field in the file header, the method may not transfer to headerless shellcode; a strip-the-header test would settle this.
- The same feature pipeline could be extended to estimate word size and byte order as separate outputs, letting analysts label a binary before a disassembler is chosen.
- The released dataset and API make it possible to crowd-source corrections, which could turn accuracy gains from retraining on real-world misclassified samples.
- If the complete-binary training trick works for the executable format tested here, it likely carries over to other container formats such as PE or Mach-O, provided code sections are extracted consistently.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a toolset, dataset, and web-service API for automated identification of CPU architecture and endianness from binary files and object code. The authors reconstruct and cross-validate two prior methods (Clemens 2015; De Nicolao et al. 2018) using 67,285 ELF code-section samples spanning 23 architectures and multiple classifiers, reporting over 98% accuracy. They also propose training classifiers on complete ELF binaries rather than only code sections, reporting 99.2% accuracy for a Random Forest classifier in Azure. The paper claims this supports usable detection for "arbitrary binary files and object code sequences" and releases all artifacts as open source.
Significance. The strongest contribution is reproducibility infrastructure: the paper appears to be the first to publicly release the dataset, extraction toolset, and models for this problem, and the code-section evaluation provides an independent confirmation of prior results on a larger and more balanced corpus. If the code-section claims hold, they usefully validate byte-frequency and signature features across dataset construction choices. The complete-binary result would be a practical convenience, but its current experimental basis does not support the paper's headline generalization; the missing header-ablation control is a fixable experimental gap rather than an internal inconsistency.
major comments (2)
- [Section 4.3, Table 9] The 99.2% complete-binary claim lacks a control for ELF header leakage. The 256 byte-frequency features are computed over the entire file, and the ELF header directly contains e_machine, EI_CLASS, and EI_DATA, which encode the ground-truth architecture, word size, and endianness. No experiment strips, masks, zeroes, or otherwise ablates these header bytes, and no headerless or non-ELF corpus is tested. The reported accuracy therefore cannot be attributed to analysis of object code rather than to reading container metadata. The authors should add an ablation (e.g., zeroed or stripped headers, or code-section-only evaluation under the same training/test setup) and restate the complete-binary conclusion accordingly.
- [Sections 3.1.1 and 4.3.1] The title and contribution (c) claim applicability to "arbitrary binary files and object code sequences," but all complete-binary experiments use only Debian-built ELF files. The authors explicitly concede in Section 4.3.1 that file-format structural differences could influence the complete-binary method and that non-ELF evaluation is only future work. Without an out-of-distribution test (e.g., non-Debian ELF, PE/Mach-O, firmware images, or extracted headerless fragments), the generalization claim is not supported. The claims should either be scoped to ELF binaries or substantiated with additional experiments.
minor comments (5)
- [Section 3.1, Table 2] The mips class is listed as upsampled from 418 to 2997 samples, but the resampling procedure is not described; if samples are duplicated, 10-fold cross-validation may be optimistically biased, so the method should be stated explicitly.
- [Table 5] The column headers for the accuracy results are duplicated and misaligned ("Accuracy BFD+endian" appears twice along with the Clemens comparison); please relabel the columns to distinguish this paper's BFD, BFD+endian, and prior-work results.
- [Section 4.4] The text says the powerpc/powerpcspe confusion matrices "can be seen in Table 8," but Table 8 reports Azure performance; the matrices are actually in Table 11, so the cross-reference should be corrected.
- [Section 4.2, Table 7] The row "Average (De Nicolao et al. [15])" appears to quote that paper's published metrics rather than metrics recomputed on the authors' own 500-binaries-per-architecture test set; the paper should state explicitly whether this is a direct reproduction or a literature comparison.
- [References] References [46] and [47] cite the same SoK paper; this duplicate should be removed or one of the entries should be corrected.
Circularity Check
Complete-binary 99.2% result is partially circular: the byte-frequency features are computed over full ELF files, whose headers encode the ground-truth architecture label; no header-stripped control is provided.
-
self definitional
[Section 4.3 (Table 9), with the feature definition in Section 3.2 and the limitation in Section 4.3.1]
"We also explored how well the classifiers perform when given the task to classify a complete binary (i.e., containing headers, and code and data sections). ... we trained those classifiers with a training set consisting of 1000 complete binaries for each of those 23 architectures. Finally, we tested them against a test set of 1000 full binaries for each architecture. ... all the classifiers we tested achieved over 97% accuracy, while the Random Forest implemented in Azure platform performed the best with 99.2% accuracy."
The feature pipeline used in the complete-binary experiment is the same 256-feature byte-frequency distribution (Section 3.2), computed over the whole file. A complete ELF binary contains its own header, and that header's e_machine, EI_CLASS, and EI_DATA fields directly encode the ground-truth architecture, wordsize, and endianness used as training labels (Table 2 labels are read from the ELF file info). The classifier can therefore attain 99.2% by counting header bytes rather than by analyzing op-code content; the prediction target is contained in the input features by construction. The paper reports no control with headers stripped, zeroed, or replaced, and Section 4.3.1 itself concedes that 'file format header variations' could influence the method.
full rationale
The code-section experiments (Section 4.1) are self-contained: features are extracted from code bytes only, while labels come from ELF headers that are not part of the input, and the results agree with prior work (Clemens 2015; De Nicolao et al. 2018). Those claims are not circular. The circularity is confined to the paper's novel complete-binary contribution (abstract, contribution (c), Section 4.3, Table 9). Because the byte-frequency feature vector spans the entire file, the ELF header—which directly carries the architecture label—is part of the classifier input; the 99.2% accuracy is therefore explainable as header reading, not code analysis. The paper's own Section 4.3.1 acknowledges that header/format variations could influence the complete-binary method and defers non-ELF evaluation to future work, while Section 3.1.1 limits all experiments to ELF. This is a label-leakage/trivial-feature circularity for that one central claim, not a derivation forced by self-citation: the paper's self-citations (Costin et al.) are contextual and not load-bearing. Overall score 6 reflects partial circularity: one load-bearing prediction reduces to its input metadata, while the replicated code-section results remain independent.
Assumptions & free parameters
free parameters (4)
- Logistic regression regularization strength C (scikit-learn) =
1000
- Logistic regression regularization strength C (Keras) =
0.0000001
- Minimum code-section size threshold =
4000 bytes
- mips class upsampling =
418 to 2997 samples
assumptions (5)
- domain assumption ELF headers correctly report the true CPU architecture
- domain assumption Code sections extracted from ELF contain valid opcode sequences for the declared architecture
- domain assumption Debian repository ELF binaries are representative of the arbitrary binaries targeted in the introduction
- domain assumption Byte-frequency distribution plus hand-crafted signatures is sufficient for accurate ISA classification
- domain assumption Complete ELF binaries with headers are an appropriate test input for the motivating use cases
Cite this review
Pith. "Pith review of Towards usable automated detection of CPU architecture and endianness for arbitrary binary files and object code sequences." pith.science (2026). https://pith.science/paper/QQABI7P3
@misc{pith2026190805459,
author = {Pith},
title = {Pith review of: Towards usable automated detection of CPU architecture and endianness for arbitrary binary files and object code sequences},
year = {2026},
howpublished = {\url{https://pith.science/paper/QQABI7P3}},
note = {Machine review of arXiv:1908.05459}
}
read the original abstract
Static and dynamic binary analysis techniques are actively used to reverse engineer software's behavior and to detect its vulnerabilities, even when only the binary code is available for analysis. To avoid analysis errors due to misreading op-codes for a wrong CPU architecture, these analysis tools must precisely identify the Instruction Set Architecture (ISA) of the object code under analysis. The variety of CPU architectures that modern security and reverse engineering tools must support is ever increasing due to massive proliferation of IoT devices and the diversity of firmware and malware targeting those devices. Recent studies concluded that falsely identifying the binary code's ISA caused alone about 10\% of failures of IoT firmware analysis. The state of the art approaches to detect ISA for arbitrary object code look promising - their results demonstrate effectiveness and high-performance. However, they lack the support of publicly available datasets and toolsets, which makes the evaluation, comparison, and improvement of those techniques, datasets, and machine learning models quite challenging (if not impossible). This paper bridges multiple gaps in the field of automated and precise identification of architecture and endianness of binary files and object code. We develop from scratch the toolset and datasets that are lacking in this research space. As such, we contribute a comprehensive collection of open data, open source, and open API web-services. We also attempt experiment reconstruction and cross-validation of effectiveness, efficiency, and results of the state of the art methods. When training and testing classifiers using solely code-sections from executable binary files, all our classifiers performed equally well achieving over 98\% accuracy. The results are consistent and comparable with the current state of the art, hence supports the general validity of the algorithms
Figures
Reference graph
Works this paper leans on
-
[1]
[n. d.]. binwalk – Firmware Analysis Tool. https://github.com/binwalk/binwalk
-
[2]
[n. d.]. UNIX System V: understanding ELF object files and debugging tools . Towards usable automated detection of CPU architecture and endianness for arbitrary binary files and object code sequences arXiv.org, pre-print, 15.8.2019 Table 9: Performance on complete binaries for classifiers trained with complete binaries. Random Forest and Random Jungle are...
work page 2019
-
[3]
Nicole L Beebe, Laurence A Maddox, Lishu Liu, and Minghe Sun. 2013. Sceadan: using concatenated n-gram vectors for improved file and data type classification. IEEE Transactions on Information Forensics and Security 8, 9 (2013), 1519–1530
work page 2013
-
[4]
Andrés Blanco and Matias Eissler. 2012. One firmware to monitor ’em all
work page 2012
-
[5]
David Brumley, Ivan Jager, Thanassis Avgerinos, and Edward J Schwartz. 2011. BAP: A binary analysis platform. In International Conference on Computer Aided Verification. Springer, 463–469
work page 2011
-
[6]
Sang Kil Cha, Thanassis Avgerinos, Alexandre Rebert, and David Brumley. 2012. Unleashing mayhem on binary code. In 2012 IEEE Symposium on Security and Privacy. IEEE, 380–394
work page 2012
-
[7]
Vitaly Chipounov and George Candea. 2010. Reverse engineering of binary device drivers with RevNIC. In Proceedings of the 5th European conference on Computer systems. ACM, 167–180
work page 2010
-
[8]
Vitaly Chipounov, Volodymyr Kuznetsov, and George Candea. 2011. S2E: A platform for in-vivo multi-path analysis of software systems. In ACM SIGARCH Computer Architecture News, Vol. 39. ACM, 265–278
work page 2011
Show all 57 references
-
[9]
François Chollet et al. 2015. Keras. https://keras.io
2015
-
[10]
John Clemens. 2015. Automatic classification of object code using machine learning. Digital Investigation 14 (2015), S156–S162
2015
-
[11]
Andrei Costin, Jonas Zaddach, Aurélien Francillon, Davide Balzarotti, and Sophia Antipolis. 2014. A Large-Scale Analysis of the Security of Embedded Firmwares. In USENIX Security
2014
-
[12]
Andrei Costin, Apostolis Zarras, and Aurélien Francillon. 2016. Automated dynamic firmware analysis at scale: a case study on embedded web interfaces. In Proceedings of the 11th ACM on Asia Conference on Computer and Communications Security. ACM
2016
-
[13]
Andrei Costin, Apostolis Zarras, and Aurélien Francillon. 2017. Towards auto- mated classification of firmware images and identification of embedded devices. In IFIP International Conference on ICT Systems Security and Privacy Protection . Springer, 233–247
2017
-
[14]
Drew Davidson, Benjamin Moench, Thomas Ristenpart, and Somesh Jha. 2013. FIE on Firmware: Finding Vulnerabilities in Embedded Systems Using Symbolic Execution. In USENIX Security
2013
-
[15]
Pietro De Nicolao, Marcello Pogliani, Mario Polino, Michele Carminati, Davide Quarta, and Stefano Zanero. 2018. ELISA: ELiciting ISA of Raw Binaries for Fine-Grained Code and Data Separation. In International Conference on Detection of Intrusions and Malware, and Vulnerability...
2018
-
[16]
Guillaume Delugré. 2010. Closer to metal: reverse-engineering the Broadcom NetExtremeâĂŹs firmware. Hack.Lu 10 (2010)
2010
-
[17]
Loïc Duflot, Yves-Alexis Perez, and Benjamin Morin. 2011. What if you canâĂŹt trust your network card?. In International Workshop on Recent Advances in Intru- sion Detection
2011
-
[18]
Joseph A Durlak and Emily P DuPre. 2008. Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American journal of community psychology 41, 3-4 (2008), 327–350
2008
-
[19]
Chris Eagle. 2011. The IDA pro book . No Starch Press
2011
-
[20]
Sebastian Eschweiler, Khaled Yakdan, and Elmar Gerhards-Padilla. 2016. discovRE: Efficient Cross-Architecture Identification of Bugs in Binary Code.. In NDSS
2016
-
[21]
Qian Feng, Rundong Zhou, Chengcheng Xu, Yao Cheng, Brian Testa, and Heng Yin. 2016. Scalable graph-based bug search for firmware images. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security . ACM, 480–491
2016
-
[22]
Simran Fitzgerald, George Mathews, Colin Morris, and Oles Zhulyn. 2012. Using NLP techniques for file fragment classification. Digital Investigation 9 (2012), S44–S49
2012
-
[23]
James C Foster. 2005. Sockets, Shellcode, Porting, and Coding: Reverse Engineering Exploits and Tool Coding for Security Professionals . Elsevier
2005
-
[24]
Louis Granboulan. [n. d.]. cpu_rec – Recognize CPU instructions in an arbitrary binary file. https://github.com/airbus-seclab/cpu_rec
-
[25]
Louis Granboulan. 2017. cpu_rec.py. SSTIC (2017). https://airbus-seclab.github. io/cpurec/SSTIC2017-Article-cpu_rec-granboulan.pdf
2017
-
[26]
Nathalie Japkowicz. 2003. Class imbalances: are we focusing on the right issue. In Workshop on Learning from Imbalanced Data Sets II , Vol. 1723. 63
2003
-
[27]
Asim Kadav and Michael M Swift. 2012. Understanding modern device drivers. ACM SIGARCH Computer Architecture News 40, 1 (2012), 87–98
2012
-
[28]
Sami Kairajärvi. 2019. Automatic identification of architecture and endianness using binary file contents . Master’s thesis. University of Jyväskylä, Jyväskylä, Finland. http://urn.fi/URN:NBN:fi:jyu-201904182217
2019
-
[29]
Hans-Peter Kriegel, Erich Schubert, and Arthur Zimek. 2017. The (black) art of runtime evaluation: Are we comparing algorithms or implementations? Knowl- edge and Information Systems 52, 2 (2017), 341–378
2017
-
[30]
Qiming Li, A Ong, P Suganthan, and V Thing. 2011. A novel support vector machine approach to high entropy data fragment classification. In Proceedings of the South African Information Security Multi-Conf (SAISMC) . 236–247
2011
-
[31]
Wei-Jen Li, Ke Wang, Salvatore J Stolfo, and Benjamin Herzog. 2005. Fileprints: Identifying file types by n-gram analysis. In Proceedings from the Sixth Annual IEEE SMC Information Assurance Workshop. IEEE, 64–71
2005
-
[32]
Yanlin Li, Jonathan M McCune, and Adrian Perrig. 2011. VIPER: verifying the integrity of PERipherals’ firmware. In Proceedings of the 18th ACM conference on Computer and communications security . ACM, 3–16
2011
-
[33]
Kaiping Liu, Hee Beng Kuan Tan, and Xu Chen. 2013. Binary code analysis. Computer 46, 8 (2013). arXiv.org, pre-print, 15.8.2019 S. Kairajärvi, A. Costin, T. Hämäläinen
2013
-
[34]
Jayaraman Manni, Ashar Aziz, Fengmin Gong, Upendran Loganathan, and Muhammad Amin. 2014. Network-based binary file extraction and analysis for malware detection. US Patent 8,832,829
2014
-
[35]
Mason McDaniel and Mohammad Hossain Heydari. 2003. Content based file type detection algorithms. In 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the . IEEE, 10–pp
2003
-
[36]
Charlie Miller. 2011. Battery firmware hacking. Black Hat USA (2011), 3–4
2011
-
[37]
Marius Muench, Jan Stijohann, Frank Kargl, Aurélien Francillon, and Davide Balzarotti. 2018. What you corrupt is not what you crash: Challenges in fuzzing embedded devices. In Proceedings of the Network and Distributed System Security Symposium
2018
-
[38]
Karsten Nohl and Jakob Lell. 2014. BadUSB-On accessories that turn evil. Black Hat USA (2014)
2014
-
[39]
Sergi "pancake" Alvarez and core contributors. [n. d.]. radare2 – unix-like reverse engineering framework and commandline tools. https://www.radare.org/
-
[40]
Pedregosa, G
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cour- napeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine ...
2011
-
[41]
Philip Penrose, Richard Macfarlane, and William J Buchanan. 2013. Approaches to the classification of high entropy file fragments. Digital Investigation 10, 4 (2013), 372–384
2013
-
[42]
Jannik Pewny, Behrad Garmany, Robert Gawlik, Christian Rossow, and Thorsten Holz. 2015. Cross-architecture bug search in binary executables. In IEEE Sympo- sium on Security and Privacy
2015
-
[43]
Michalis Polychronakis, Kostas G Anagnostakis, and Evangelos P Markatos. 2010. Comprehensive shellcode detection using runtime heuristics. In Proceedings of the 26th Annual Computer Security Applications Conference . ACM, 287–296
2010
-
[44]
Nguyen Anh Quynh. 2014. Capstone: Next-gen disassembly framework. Black Hat USA (2014)
2014
-
[45]
Yan Shoshitaishvili, Ruoyu Wang, Christophe Hauser, Christopher Kruegel, and Giovanni Vigna. 2015. Firmalice-Automatic Detection of Authentication Bypass Vulnerabilities in Binary Firmware.. In NDSS
2015
-
[46]
Yan Shoshitaishvili, Ruoyu Wang, Christopher Salls, Nick Stephens, Mario Polino, Andrew Dutcher, John Grosen, Siji Feng, Christophe Hauser, Christopher Kruegel, et al. 2016. Sok:(state of) the art of war: Offensive techniques in binary analysis. In 2016 IEEE Symposium on Secur...
2016
-
[47]
Yan Shoshitaishvili, Ruoyu Wang, Christopher Salls, Nick Stephens, Mario Polino, Audrey Dutcher, John Grosen, Siji Feng, Christophe Hauser, Christopher Kruegel, and Giovanni Vigna. 2016. SoK: (State of) The Art of War: Offensive Techniques in Binary Analysis. In IEEE Symposium...
2016
-
[48]
Richard L Sites, Anton Chernoff, Matthew B Kirk, Maurice P Marks, and Scott G Robinson. 1993. Binary translation. Digital Technical Journal 4 (1993), 137–137
1993
-
[49]
Dawn Song, David Brumley, Heng Yin, Juan Caballero, Ivan Jager, Min Gyung Kang, Zhenkai Liang, James Newsome, Pongsin Poosankam, and Prateek Saxena
-
[50]
Luigi Sportiello and Stefano Zanero. 2012. Context-based file block classification. In IFIP International Conference on Digital Forensics . Springer, 67–82
2012
-
[51]
Iain Sutherland, George E Kalb, Andrew Blyth, and Gaius Mulley. 2006. An em- pirical examination of the reverse engineering process for binary files.Computers & Security 25, 3 (2006)
2006
-
[52]
Dave Jing Tian, Adam Bates, and Kevin Butler. 2015. Defending against malicious USB firmware with GoodUSB. InProceedings of the 31st Annual Computer Security Applications Conference. ACM, 261–270
2015
-
[53]
Eric Van Den Berg and Ramkumar Chinchani. 2009. Detecting exploit code in network flows. US Patent App. 11/260,914
2009
-
[54]
Fish Wang and Yan Shoshitaishvili. 2017. Angr-the next generation of binary analysis. In 2017 IEEE Cybersecurity Development (SecDev) . IEEE, 8–9
2017
-
[55]
Tielei Wang, Tao Wei, Zhiqiang Lin, and Wei Zou. 2009. IntScope: Automati- cally Detecting Integer Overflow Vulnerability in X86 Binary Using Symbolic Execution.. In NDSS
2009
-
[56]
a r c h i t e c t u r e
H Xie, Azizi Abdullah, and Rossilawati Sulaiman. 2013. Byte frequency analysis descriptor with spatial information for file fragment classification. In Proceeding of the International Conference on Artificial Intelligence in Computer Science and ICT. 6 APPENDICES Listing 1: Ma...
2013
-
[2008]
In International Conference on Information Systems Security
BitBlaze: A new approach to computer security via binary analysis. In International Conference on Information Systems Security . Springer, 1–25
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.