REVIEW 4 major objections 5 minor 38 references
PackHero: A Scalable Graph-based Approach for Efficient Packer Identification
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read PackHero identifies packers from static call graphs, reaching 93.7% macro-F1 with only 10 samples per packer.
desk verdict A genuinely useful, well-run packer identifier with code and a public dataset, but the paper overclaims cross-version transfer: its own RGD table shows 0% identification for three entire packer versions. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the 'unpacking stub call graph' produced by Algorithm 1: take the directed call graph, convert it to undirected, keep the connected component containing each entry point, and if the graph is empty keep entry functions plus external library calls. This heuristic isolates the only part of a packed binary that is statically visible and is supposed to be packer-specific. The learned similarity comes from a Graph Matching Network (GMN), a graph neural network that produces a 256-dimensional embedding per graph by propagating information between paired graphs; training minimizes a margin loss on cosine similarity. Hierarchical clustering per packer then groups embeddings, each cluster gets a medoid and a threshold (mean intra-cluster cosine minus standard deviation), and inference first compares only against medoids, so the number of GMN inference calls stays near the cluster count plus stored samples rather than the full database size.
What would settle it
Take a packer version never seen during configuration, pack a set of programs with it, and run PackHero configured only on an older version; if identification recall stays at 0% across many such version pairs, the transferability assumption fails. The paper already shows three such failures (PELock v1.06, kkrunchy v0.23alpha, Themida v3.04), so a systematic study across version pairs would settle how general the claim is.
Extended reading notes
Core claim
The central claim is that packer identity is readable from the shape of a small static call-graph subgraph, without executing the program or matching byte signatures. PackHero extracts the connected component reachable from the program entry point (keeping the entry function and external-library calls when no edges exist), labels graphs as similar when they come from the same packer, and trains a Graph Matching Network to embed graph pairs so that same-packer pairs are close and different-packer pairs are far. At inference, it compares the input graph's embedding with cluster medoids, then with graphs in selected clusters, and assigns the packer whose clusters give the highest normalized count of near-threshold matches. The paper reports macro-average F1 of 93.7% and accuracy 98.7% at 10 samples per packer, rising to 98.3% and 99.8% at 100 samples, with perfect recall on Themida versus 92% for DIE and 31% for PackGenome.
Load-bearing premise
Programs packed with the same packer share a visible web of function calls near the program's entry point that is more similar inside a packer than across packers, and this similarity survives when the packer's version or settings change.
Editorial extensions
If this is right
- With as few as 10 wild samples per packer, a static-only identifier can match signature-based tools, removing the need to manually write signatures for each new packer.
- Virtualization-based packers such as Themida/Winlicense, whose runtime evasions break dynamic signature extraction, can be identified statically with recall 1.0.
- Adding a new packer costs only the samples for that packer: without fine-tuning, integration uses m·l samples instead of (n+m)·l, so cost stays constant as the number of recognized packers grows.
- Clustering keeps inference time bounded: with 200 packers and 100 samples each, the estimated identification time drops from about 35 seconds without clustering to about 21 milliseconds with it.
- A single configured version generalizes across 16 of 19 packer versions in the RGD dataset, suggesting one version can seed recognition of related versions.
Reading between the lines
- The paper's RGD transfer table is a warning: PELock v1.06, kkrunchy v0.23alpha, and Themida v3.04 are identified in 0/5, 0/2, and 0/5 configurations, so the headline numbers, which come from same-version lab-10 samples, likely overstate performance on unseen variants; version drift is a known failure mode, not a solved one.
- Because identity hinges on a small entry-point-connected subgraph, an adversary who inserts decoy functions or redirects calls inside that subgraph could plausibly push a sample below cluster thresholds and force an 'unknown' label; the paper acknowledges this possibility but does not test it.
- The same GMN-plus-clustering recipe could in principle be applied to other binary graph representations or to packer detection (packed versus unpacked), but the paper explicitly leaves detection out of scope.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes PackHero, a static packer identifier that (i) extracts a heuristic 'unpacking stub' call graph via Algorithm 1, (ii) embeds call graphs with a Graph Matching Network (GMN) trained on same-packer/different-packer pairs, and (iii) uses hierarchical clustering with medoids and cluster-specific thresholds to match an input graph against a database of labeled graphs. The method is evaluated on the lab-10 subset of the public Aghakhani et al. dataset and on the RGD dataset from PackGenome. The authors report a macro-average F1-score of 93.7% with 10 training samples per packer and 98.3% with 100 samples, and they claim that PackHero matches or outperforms signature-based tools, particularly on Themida/Winlicense, where they report 100% recall.
Significance. If the central claims held as stated, PackHero would be a valuable contribution: it offers static, few-shot packer identification, uses public datasets, compares against several baselines, and the authors release their source code. The strongest claim, however, is that PackHero identifies packers generally, including unseen versions and configurations. The paper's own RGD transfer results (Table 4) do not support that broad claim: PELock v1.06 (0/5), kkrunchy v0.23alpha (0/2), and Themida v3.04 (0/5) have zero identified configurations. The 100% recall on Themida/Winlicense is a same-version estimate from lab-10, not a cross-version result. The work is promising, but the generalization claim needs substantial revision and additional evidence before the paper can be accepted.
major comments (4)
- [§4.2, Table 4] The transfer experiment is reported as a binary per-configuration outcome, where a configuration counts as identified only if all of its samples are correctly classified, and the text then summarizes this as 'generalizes across 16 out of 19 different versions.' This all-or-nothing measure masks complete failures: PELock v1.06 (0/5), kkrunchy v0.23alpha (0/2), and Themida v3.04 (0/5) have no identified configurations at all. Because the abstract and the RQ1 answer claim 100% recall on Themida/Winlicense while that recall is computed only on the single lab-10 version, the headline numbers are same-version estimates. Please report per-configuration recall and precision rather than only identified/not, analyze the failed versions, and re-frame the abstract and conclusions to distinguish same-version identification from cross-version generalization.
- [§4.2 and §4.1 (Hyperparameter Tuning)] The minimum-samples experiment is a single run with no variance reporting, as the authors state in §4.2: 'the experiment was done with a single run due to the computational cost.' In addition, the GMN hyperparameters are chosen by a grid search that 'maximiz[es] intra-packet similarity' on the same lab-10 corpus, with no described held-out validation split. This makes the reported macro-F1, precision, and recall point estimates whose sensitivity to random initialization and hyperparameter choice is unknown. Repeated runs with fixed seeds or a cross-validation-style protocol are needed to support the quantitative claims.
- [§3.3, Algorithm 1] The core assumption of Algorithm 1 is that the connected component containing the entry point (or a fallback second component) is the unpacking stub and is sufficient for identification. The paper reports that stubs have on average ≈3 functions, but does not validate the assumption beyond the aggregate result. The RGD failures suggest the heuristic is version-dependent; for instance, Themida v3.04 and PELock v1.06 produce 0% identification. Please provide an analysis of the failed configurations—e.g., stub size, topology, or feature distributions—to characterize when the heuristic breaks, rather than attributing failures only to 'differences in the unpacking stub.'
- [§4.4, Fig. 5 and subsequent paragraph] The claim that PackHero without fine-tuning performs well is supported only after removing two of the nine packers (Obsidium and Petite) from the average, and the remaining per-packer results are not shown. This selective exclusion, together with the stated assumption that samples to be integrated are 'always correctly labeled,' weakens the RQ3 conclusion. Please report per-packer results for the leave-one-out integration experiment and justify excluding Obsidium and Petite using objective criteria (e.g., heterogeneity metrics) rather than post hoc performance.
minor comments (5)
- [§3.1, threshold formula] The text says 'the average cosine distance between pairs of graphs belonging to the cluster minus the standard deviation,' but the formula computes average cosine similarity minus σ; please clarify whether distance or similarity is intended.
- [Table 2] In the row for 'outdegree,' the description reads 'The in-degree of the function in the call graph'; this is a typo and should read 'out-degree.'
- [§4.2] The description of building the collections by 'gradually eliminate 10 programs' is ambiguous; the plots show sizes 10 to 100, but the text should state explicitly that the collections are of size 10, 20, ..., 100 samples per packer.
- [§2.1, reference [19]] The sentence attributes the CG/SVM approach to 'Hao et al.,' but the cited reference [19] is authored by Liu, Guo, Cui, Shen, and Ping (2-SPIFF); please correct the attribution.
- [§3.1, GMN details] Only the embedding size (256) is given for the GMN; to support reproducibility beyond the released code, please specify the number of layers, hidden dimensions, learning rate, and training epochs used.
Circularity Check
No significant circularity: PackHero's central results come from supervised training on separate splits plus an external RGD transfer test, not from fitting the target labels into the model.
full rationale
PackHero's claimed derivation chain is not circular. The pipeline is: (1) extract a heuristic 'unpacking stub' call graph with Algorithm 1; (2) train a Graph Matching Network on graph pairs labeled by packer; (3) cluster per packer and compute cluster-specific thresholds from training statistics; (4) classify by cosine similarity to stored medoids; and (5) evaluate on held-out lab-10 samples and the external PackGenome RGD corpus. None of these steps defines the target quantity in terms of itself: packer labels are external ground truth, the test samples are disjoint from the configuration samples, and the RGD experiment tests versions and configurations not seen in training. The minor caveats are real but non-circular: hyperparameters are tuned on the same corpus, cluster thresholds are derived from training clusters, and the integration experiment assumes correctly labeled new samples. These affect external validity, not self-referentiality. Section 5 honestly acknowledges heuristic, disassembler, and adversarial limitations, including the fact that PackHero currently does not address detection. The paper's self-citations, e.g., reference [10], are background context and are not load-bearing for the central identification claim. The RGD transfer table shows 0% identification for some versions, which weakens the generalization claim, but this is an empirical support problem rather than a derivation that reduces to its own inputs. Therefore, no circular step is present and the paper merits a low circularity score.
Assumptions & free parameters
free parameters (5)
- GMN trainable weights =
learned during training
- Embedding size =
256
- Margin gamma in GMN loss =
not reported
- Cluster threshold offset =
average intra-cluster cosine minus one standard deviation
- Clustering linkage and number of clusters =
single linkage, silhouette-selected k
assumptions (6)
- domain assumption Programs packed with the same packer have more similar call graphs than programs packed with different packers.
- ad hoc to paper The connected component containing the entry point (or a fallback second component) is the unpacking stub and is sufficient for identification.
- domain assumption radare2 disassembly and function identification are accurate enough on packed binaries.
- domain assumption The lab-10 dataset labels and repacking reflect real-world packed programs.
- domain assumption Samples from the same packer version and configuration as training are representative of the packer in general.
- domain assumption The GMN trained on a few samples generalizes to new samples and new packers.
Cite this review
Pith. "Pith review of PackHero: A Scalable Graph-based Approach for Efficient Packer Identification." pith.science (2026). https://pith.science/paper/L7IDDCTK
@misc{pith2026250600659,
author = {Pith},
title = {Pith review of: PackHero: A Scalable Graph-based Approach for Efficient Packer Identification},
year = {2026},
howpublished = {\url{https://pith.science/paper/L7IDDCTK}},
note = {Machine review of arXiv:2506.00659}
}
read the original abstract
Anti-analysis techniques, particularly packing, challenge malware analysts, making packer identification fundamental. Existing packer identifiers have significant limitations: signature-based methods lack flexibility and struggle against dynamic evasion, while Machine Learning approaches require extensive training data, limiting scalability and adaptability. Consequently, achieving accurate and adaptable packer identification remains an open problem. This paper presents PackHero, a scalable and efficient methodology for identifying packers using a novel static approach. PackHero employs a Graph Matching Network and clustering to match and group Call Graphs from programs packed with known packers. We evaluate our approach on a public dataset of malware and benign samples packed with various packers, demonstrating its effectiveness and scalability across varying sample sizes. PackHero achieves a macro-average F1-score of 93.7% with just 10 samples per packer, improving to 98.3% with 100 samples. Notably, PackHero requires fewer samples to achieve stable performance compared to other Machine Learning-based tools. Overall, PackHero matches the performance of State-of-the-art signature-based tools, outperforming them in handling Virtualization-based packers such as Themida/Winlicense, with a recall of 100%.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[17]
IEEE Access7, 51620–51629 (2019)
Li, X., Shan, Z., Liu, F., Chen, Y., Hou, Y.: A Consistently-Executing Graph- Based Approach for Malware Packer Identification. IEEE Access7, 51620–51629 (2019). https://doi.org/10.1109/ACCESS.2019.2910268
arXiv 2019
-
[1]
In: Proceedings of Symposium on Network and Distributed System Security (NDSS) (Feb 2020)
Aghakhani, H., Gritti, F., Mecca, F., Lindorfer, M., Ortolani, S., Balzarotti, D., Vigna, G., Kruegel, C.: When Malware is Packin’ Heat; Limits of Machine Learning Classifiers Based on Static Analysis Features. In: Proceedings of Symposium on Network and Distributed System Security (NDSS) (Feb 2020)
work page 2020
-
[2]
International Journal of Advanced Computer Science and Applications5(1) (2014)
Al-Anezi, D.M.M.K.: Generic packing detection using several complexity analy- sis for accurate malware detection. International Journal of Advanced Computer Science and Applications5(1) (2014). https://doi.org/10.14569/IJACSA.2014. 050102
-
[3]
https://virustotal.github.io/yara/ (2024), accessed: 2024-04-15
Alvarez, V.M.: Yara. https://virustotal.github.io/yara/ (2024), accessed: 2024-04-15
work page 2024
-
[4]
arXiv preprint arXiv:1804.04637 (2018)
Anderson, H.S., Roth, P.: Ember: an open dataset for training static pe malware machine learning models. arXiv preprint arXiv:1804.04637 (2018)
arXiv 2018
-
[5]
ASPack Software: ASPack Software - Application for compression, packing and protection of software.http://www.aspack.com/ (2024), accessed: 2024-04-15
work page 2024
-
[6]
IEEE Transactions on Software Engineering16(4), 483–487 (1990)
Callahan, D., Carle, A., Hall, M., Kennedy, K.: Constructing the procedure call multigraph. IEEE Transactions on Software Engineering16(4), 483–487 (1990). https://doi.org/10.1109/32.54302
- [7]
Show all 38 references
-
[8]
In: Proceedings of the 18th Virus Bulletin International Conference
Ebringer, T., Sun, L., Boztas, S.: A fast randomness test that preserves local detail. In: Proceedings of the 18th Virus Bulletin International Conference. pp. 34–42. Virus Bulletin Ltd (2008) 20 M. Di Gennaro et al
2008
-
[9]
ACM Comput
Egele, M., Scholte, T., Kirda, E., Kruegel, C.: A survey on automated dynamic malware-analysis techniques and tools. ACM Comput. Surv.44(2) (mar 2008). https://doi.org/10.1145/2089125.2089126
2008
-
[10]
Computers & Se- curity 113, 102550 (Feb 2022).https://doi.org/10.1016/j.cose.2021.102550
Galloro, N., Polino, M., Carminati, M., Continella, A., Zanero, S.: A Systematical and longitudinal study of evasive behaviors in windows malware. Computers & Se- curity 113, 102550 (Feb 2022).https://doi.org/10.1016/j.cose.2021.102550
2022
-
[11]
Synthesis Lectures on Artificial Intelligence and Machine Learning 14(3), 1–159 (2020), publisher: Morgan and Claypool
Hamilton, W.L.: Graph Representation Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning 14(3), 1–159 (2020), publisher: Morgan and Claypool
2020
-
[12]
IEEE Security & Privacy5(02), 40–45 (mar 2007)
Hamrock, J., Lyda, R.: Using entropy analysis to find encrypted and packed mal- ware. IEEE Security & Privacy5(02), 40–45 (mar 2007). https://doi.org/10. 1109/MSP.2007.48
2007
-
[13]
https://github.com/horsicq/Detect-It-Easy (2024), accessed: 2024-04-15
Horsicq: Detect it easy. https://github.com/horsicq/Detect-It-Easy (2024), accessed: 2024-04-15
2024
-
[14]
In: International Con- ference on Detection of intrusions and malware, and vulnerability assessment
Jacob, G., Comparetti, P., Neugschwandtner, M., Kruegel, C., Vigna, G.: A static, packer-agnostic filter to detect similar malware samples. In: International Con- ference on Detection of intrusions and malware, and vulnerability assessment. vol. 7591 (01 2010).https://doi.org/...
2010 doi
-
[15]
In: 2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC)
Kim, Y., Paik, J.Y., Choi, S., Cho, E.S.: Efficient svm based packer identification with binary diffing measures. In: 2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC). vol. 1, pp. 795–800 (2019).https://doi. org/10.1109/COMPSAC.2019.00117
2019
-
[16]
In: Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security
Li,S., Ming, J.,Qiu, P., Chen, Q., Liu,L., Bao, H.,Wang,Q., Jia, C.:PackGenome: Automatically Generating Robust YARA Rules for Accurate Malware Packer De- tection. In: Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security. pp. 3078–3092. CCS ’23...
2023
-
[18]
In: Chaudhuri, K., Salakhut- dinov, R
Li, Y., Gu, C., Dullien, T., Vinyals, O., Kohli, P.: Graph Matching Networks for Learning the Similarity of Graph Structured Objects. In: Chaudhuri, K., Salakhut- dinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning. Proceedings of Machine Lear...
2019
-
[19]
Applied Intelligence 51(12), 9038–9053 (2021).https://doi.org/10.1007/s10489-021-02347-w
Liu, H., Guo, C., Cui, Y., Shen, G., Ping, Y.: 2-SPIFF: a 2-stage packer identifica- tion method based on function call graph and file attributes. Applied Intelligence 51(12), 9038–9053 (2021).https://doi.org/10.1007/s10489-021-02347-w
2021 doi
-
[20]
Expert Syst
Liu, Z., Wang, R., Japkowicz, N., Gomes, H.M., Peng, B., Zhang, W.: Segdroid: An android malware detection method based on sensitive function call graph learning. Expert Syst. Appl.235(C) (Jan 2024),https://doi.org/10.1016/j.eswa.2023. 121125
2024 doi
-
[21]
SIGPLAN Not.40(6), 190–200 (jun 2005)
Luk, C.K., Cohn, R., Muth, R., Patil, H., Klauser, A., Lowney, G., Wallace, S., Reddi, V.J., Hazelwood, K.: Pin: building customized program analysis tools with dynamic instrumentation. SIGPLAN Not.40(6), 190–200 (jun 2005). https:// doi.org/10.1145/1064978.1065034
2005
-
[22]
Proceedings 2020 Network and Distributed System Security Symposium (2020)
Mantovani, A., Aonzo, S., Ugarte-Pedrero, X., Merlo, A., Balzarotti, D.: Prevalence and impact of low-entropy packing schemes in the malware ecosystem. Proceedings 2020 Network and Distributed System Security Symposium (2020)
2020
-
[23]
Morgan Kauf- mann, San Francisco, CA (1997) PackHero 21
Muchnick, S.S.: Advanced Compiler Design and Implementation. Morgan Kauf- mann, San Francisco, CA (1997) PackHero 21
1997
-
[24]
55(5)(dec2022)
Muralidharan, T., Cohen, A., Gerson, N., Nissim, N.: File packing from the mal- ware perspective: Techniques, analysis approaches, and directions for enhance- ments.ACMComput.Surv. 55(5)(dec2022). https://doi.org/10.1145/3530810
-
[25]
Oreans Technologies: Winlicense.https://www.oreans.com/WinLicense.php, ac- cessed: 2024-07-08
2024
-
[26]
In: 2021 IEEE Symposium on Security and Privacy (SP)
Pang, C., Yu, R., Chen, Y., Koskinen, E., Portokalidis, G., Mao, B., Xu, J.: Sok: All you ever wanted to know about x86/x64 binary disassembly but were afraid to ask. In: 2021 IEEE Symposium on Security and Privacy (SP). pp. 833–851 (2021)
2021
-
[27]
https://www.aldeid.com/wiki/PEiD (2024), accessed: 2024-04-15
PEiD: Peid. https://www.aldeid.com/wiki/PEiD (2024), accessed: 2024-04-15
2024
-
[28]
Pattern Recognition Letters29(14), 1941–1946 (2008)
Perdisci, R., Lanzi, A., Lee, W.: Classification of packed executables for accurate computer virus detection. Pattern Recognition Letters29(14), 1941–1946 (2008). https://doi.org/10.1016/j.patrec.2008.06.016
2008 doi
-
[29]
radare2: radare2: Unix-like reverse engineering framework and command-line tools (2024), https://github.com/radareorg/radare2, accessed: 2024-04-15
2024
-
[30]
In: Pro- ceedings of the 13th ACM Workshop on Artificial Intelligence and Security
Raff, E., Zak, R., Lopez Munoz, G., Fleming, W., Anderson, H.S., Filar, B., Nicholas, C., Holt, J.: Automatic yara rule generation using biclustering. In: Pro- ceedings of the 13th ACM Workshop on Artificial Intelligence and Security. p. 71–82. AISec’20, Association for Comput...
2020
-
[31]
In: 2017 47th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)
Rahbarinia, B., Balduzzi, M., Perdisci, R.: Exploring the Long Tail of (Malicious) Software Downloads. In: 2017 47th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). pp. 391–402 (2017).https://doi. org/10.1109/DSN.2017.19
2017 doi
-
[32]
In: Proceedings of the 3rd USENIX Conference on Offensive Technologies
Rolles, R.: Unpacking virtualization obfuscators. In: Proceedings of the 3rd USENIX Conference on Offensive Technologies. p. 1. WOOT’09, USENIX Associ- ation, USA (2009)
2009
-
[33]
In: Steinfeld, R., Hawkes, P
Sun, L., Versteeg, S., Boztaş, S., Yann, T.: Pattern recognition techniques for the classification of malware packers. In: Steinfeld, R., Hawkes, P. (eds.) Information Security and Privacy. pp. 370–390. Springer Berlin Heidelberg, Berlin, Heidelberg (2010)
2010
-
[34]
In: 2015 IEEE Symposium on Security and Privacy
Ugarte-Pedrero, X., Balzarotti, D., Santos, I., Bringas, P.G.: SoK: Deep Packer Inspection: A Longitudinal Study of the Complexity of Run-Time Packers. In: 2015 IEEE Symposium on Security and Privacy. pp. 659–673 (2015).https:// doi.org/10.1109/SP.2015.46
2015 doi
-
[35]
Ugarte-Pedrero, X., Graziano, M., Balzarotti, D.: A close look at a daily dataset of malware samples22(1) (Jan 2019)
2019
-
[36]
https://upx.github.io/ (2024), accessed: 2024-04-15
UPX: UPX – the ultimate packer for executables. https://upx.github.io/ (2024), accessed: 2024-04-15
2024
-
[37]
In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security
Yadegari, B., Debray, S.: Symbolic execution of obfuscated code. In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. p. 732–744. CCS ’15, Association for Computing Machinery, New York, NY, USA (2015). https://doi.org/10.1145/2810103.2813663
2015
-
[38]
Cambridge University Press, 2 edn
Zaki, M.J., Meira Jr, W.: Data mining and machine learning: fundamental concepts and algorithms. Cambridge University Press, 2 edn. (2020)
2020
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.