Pith. sign in

REVIEW 4 major objections 3 minor 66 references

Statistical Analysis of Executability and Program Equivalence in Decompilation for IoT Vulnerability Detection

T0 review · 4 major / 3 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A new nine-dimensional score measures how faithfully decompiled IoT code preserves structure, behavior, and semantics rather than just whether test cases pass.

desk verdict Useful metric framework, but the headline d=0.96 is partly definitional because the behavioral score is zeroed on recompilation failure, and the paper's own tables contradict that rule. read the letter →

arxiv 2608.06960 v1 pith:2UFWK3CY submitted 2026-08-07 cs.SE cs.CR

Minami Yoda , Jialong Li , Yasuyuki Tahara , Yuichi Sei , Yutaka Matsuno This is my paper · ORCID
classification cs.SEcs.CR
keywords DecompilationLargeLanguageModelsProgramEquivalenceIoTSecurityStatisticalEvaluationBinaryAnalysisStructuralSimilarityBehavioral
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Decompilation of IoT firmware is meant to reconstruct code that behaves like the original binary, but current evaluation metrics concentrate on whether test cases pass and can miss code that looks plausible while its internal control flow or error handling has changed. The paper proposes a nine-dimensional quality score spanning structural similarity (function signatures, control-flow graph, data structures), behavioral similarity (logic operations, API/system calls, error handling), and semantic similarity (identifiers, variable names, type use). On 19,223 decompiled outputs from 318 OpenWrt programs, it finds that outputs that recompile score higher overall (Cohen's $d=0.92$), with behavioral similarity the strongest single predictor ($d=0.96$) and structural similarity also substantial ($d=0.69$). It isolates 700 cases that failed to recompile despite behavioral similarity of 0.8 or higher, and those cases have lower structural similarity than the recompilation-success group, which the paper takes as evidence that structural fidelity is an independent axis that behavioral checks cannot replace. The result matters because vulnerability analysts need to know whether decompiled code they are reading can be trusted in the details where IoT vulnerabilities actually hide.

What carries the argument

The load-bearing object is the weighted nine-dimensional score. The three category aggregates are $S_{\mathrm{struct}} = 0.30S_{\mathrm{sig}} + 0.50S_{\mathrm{cfg}} + 0.20S_{\mathrm{data}}$, $S_{\mathrm{behav}} = 0.40S_{\mathrm{logic}} + 0.30S_{\mathrm{api}} + 0.30S_{\mathrm{error}}$, and $S_{\mathrm{sem}} = 0.50S_{\mathrm{ident}} + 0.30S_{\mathrm{var}} + 0.20S_{\mathrm{type}}$, combined into $S_{\mathrm{total}} = 0.35S_{\mathrm{struct}} + 0.40S_{\mathrm{behav}} + 0.25S_{\mathrm{sem}}$. The structural axis compares function signatures, data-structure restoration, and, with the highest weight, a control-flow-graph (CFG) similarity computed from normalized differences in node count, edge count, and cyclomatic complexity. The behavioral axis accumulates severity-weighted counts of logic, API/system-call, and error-handling defects, with API correctness formed from order-preserving sequence matching plus a Jaccard set-overlap coefficient. The semantic axis covers magic-number avoidance, variable-name agreement, and type-system usage. Because the dataset is so large that ordinary $p$-values are trivially significant, the analysis uses Cohen's $d$ effect size as the criterion for whether a metric distinguishes the recompilation-success group from the failure group.

What would settle it

Recompute $S_{\mathrm{behav}}$ for the 700 high-behavior failure cases under the paper's stated rule that the API/system-call sub-metric is 0 on recompilation failure; the maximum possible behavioral score would then be 0.7, so the reported mean of 0.814 directly shows whether the rule was applied or whether the effect sizes are partly definitional artifacts.

Watch

Extended reading notes

Core claim

The paper's central claim is that decompilation quality for vulnerability analysis is not a single axis: recompilability and behavioral equivalence are necessary but not sufficient, and the internal structure of restored code has to be scored explicitly. It defines a nine-dimensional metric and applies it to 19,223 decompilation outputs from 318 OpenWrt programs produced by one rule-based method and four LLM-based methods. Recompilation-successful outputs beat failed ones on overall score with Cohen's $d=0.92$; behavioral similarity separates the groups most strongly ($d=0.96$), structural similarity follows ($d=0.69$), and semantic similarity is comparatively weak ($d=0.33$). The decisive evidence is the high-behavior failure group: 700 outputs whose behavioral similarity (0.814) exceeded the success group's (0.787) yet failed to recompile, while their structural similarity (0.549) fell below the success group's (0.603). The paper reads this as showing that API-level and test-level agreement can look correct while defects in branch structure or error-handling paths are what actually break recompilation, and that only a structural axis can detect those.

Load-bearing premise

The central comparison assumes that the behavioral-similarity score is computed independently of the recompilation outcome, even though the paper's own rule (Section 3.3.2) sets the API/system-call sub-metric to zero on recompilation failure and folds that zero into the behavioral score.

Editorial extensions

If this is right

  • Vulnerability analysts can use the nine-dimensional score, or its structural sub-scale, to rank decompiled firmware by how faithful the internal structure is before manual review.
  • Behavioral similarity alone is not enough: the 700-case high-behavior failure group shows that a decompiled function can match API calls and logic yet still fail to compile because of branch and error-handling structure.
  • General-purpose LLMs (ChatGPT-5.1, Claude Haiku 4.5, DeepSeek-R1) score about 0.1 higher than Ghidra and LLM4Decompile, suggesting that current LLM-based decompilation is the more promising route for faithful restoration.
  • Prompt strategy and program category have negligible effect on quality ($\eta^2=0.006$ and $\eta^2=0.023$), so improvements should target model choice and metric-driven training rather than prompt engineering.
  • Higher compiler optimization levels lower recompilation success (15.3% at -O0 to 12.6% at -O3), so optimization-induced structure loss is a concrete obstacle for practical firmware analysis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to turn the two axes into a classifier: measuring precision and recall of a behavioral-threshold/structural-threshold rule (for example, behavior at least 0.8 and structure at least 0.6) for predicting recompilation would quantify how much the reported group differences help in practice; the paper reports group means but not classification performance.
  • Because the API/system-call sub-metric alone has $d=1.91$ while the other sub-metrics are small or inverse, most of the behavioral signal may be carried by API-call fidelity; a follow-up could test whether a one-dimensional API-call metric predicts recompilation almost as well as the full nine-dimensional score.
  • The weights assigned to the three categories and nine sub-metrics are chosen by the authors and not derived from data; re-weighting with equal weights or with weights fitted to predict recompilation could change the reported effect sizes, and the paper itself flags this as open.
  • If the metric is intended for vulnerability detection, the next check would be correlating low structural scores in decompiled firmware with known CVE-class defects such as missing NULL checks and leak-on-error paths, which the paper motivates but does not measure.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper proposes a nine-dimensional quality metric for decompiled C code, organized into structural, behavioral, and semantic similarity, and uses it to compare recompilation-success versus recompilation-failure groups across 318 OpenWrt programs and five decompilation methods. The authors report large effect sizes for overall score (d=0.92) and behavioral similarity (d=0.96), analyze sub-metrics, compiler options, prompt strategies, model differences, and existing metrics, and identify a 'high-behavior failure' group to argue that structural similarity captures defects missed by behavioral evaluation.

Significance. A validated multi-dimensional decompilation quality metric would be valuable for IoT vulnerability analysis and for evaluating LLM-generated code, so the paper addresses an important problem. The study has notable strengths: a large dataset (nominally 19k+ generated programs), automated metric computation, comparison with CodeBLEU and CodeScore, and the high-behavior-failure analysis. However, the primary statistical claim is currently not interpretable because one component of the behavioral metric is assigned a score of 0 when recompilation fails, which makes the outcome partly definitional; moreover, the paper's own reported numbers contradict this rule. The metric framework and dataset may be useful after correction, but the present evidence does not establish the headline predictive claim.

major comments (4)
  1. [Section 3.3.2, Eq. (8), Tables 4 and 6] The rule in Section 3.3.2, 'If recompilation fails, the score is 0,' applied to S_api, combined with Eq. (8), implies that for every failed recompilation S_behav <= 0.4*1.0 + 0.3*0 + 0.3*1.0 = 0.7. Yet Table 4 reports a failure-group behavioral similarity of 0.728, and Table 6 reports the high-behavior failure group's behavioral similarity as 0.814. These values are impossible unless the zero rule was not actually applied. The text and the numbers cannot both be true, so the reported d=0.96 cannot be interpreted as evidence that behavioral similarity predicts recompilation success.
  2. [Section 4.2.4, Table 5] The sub-metric analysis shows API detection with d=1.91, while logic operations and error handling have inverse effect sizes (d=-0.49 and d=-0.35). Because S_api is zeroed on recompilation failure, the large API effect is definitional rather than evidential, and the remaining behavioral sub-metrics actually oppose the headline direction. The claim that 'behavioral similarity is an important predictor' is therefore largely an artifact of including S_api in S_behav. A corrected analysis should remove the zero rule or report S_behav excluding S_api.
  3. [Sections 4.2.1, 4.2.2, Table 3, Figure 3, Table 6] The sample counts are mutually inconsistent. The abstract says 19,625 results; Section 4.2.1 says 21,624 generated programs, with 2,299 generation errors and 402 unmeasurable cases, leaving 19,223; Section 4.2.2 reports 2,736 successes out of 19,625; Table 3's per-model counts sum to 19,625; Figure 3 and Table 6 use 19,223 (2,702 success and 16,521 failure); and Section 4.2.7's F-statistic denominator implies 19,625. These inconsistencies affect every reported effect size and must be reconciled, and the statistical analyses must be re-run on a single, clearly stated sample.
  4. [Section 4.2.5, Table 6] The 'high-behavior failure' analysis is used to argue that structural similarity is complementary to behavioral similarity, but the group's mean behavioral similarity of 0.814 is impossible under the scoring rule stated in Section 3.3.2. The complementary-relationship claim cannot be evaluated until the behavioral scores are recomputed under a consistent rule. In addition, the thresholds of 0.8 (behavioral) and 0.6 (structural) appear to be introduced post hoc and should be justified independently of the observed data.
minor comments (3)
  1. [Section 4.2.1, Figure 2] The claim that the score distribution is 'approximately normal' is not supported by a normality test or diagnostic plots; given the bounded 0-1 scale and the reported mean/median near 0.6, this should be substantiated or rephrased.
  2. [Section 3.3.2] The phrase 'If recompilation fails, the score is 0' is ambiguous about whether it zeroes the entire API/system-call score or only one component. Please specify exactly which term in the computation is affected and how this interacts with the SequenceMatcher and Jaccard components.
  3. [Tables 4 and 6] Reporting standard deviations or confidence intervals alongside the group means would help readers assess the overlap between the success and failure distributions, especially for the sub-metric comparisons with small or inverse effect sizes.

Circularity Check

1 steps flagged · score 7.0 of 10

The headline d=0.96 for behavioral similarity is partly definitional: S_api is zeroed on recompilation failure and then included in S_behav, so the metric encodes the outcome it claims to predict; Table 6's high-behavior failure mean 0.814 is arithmetically impossible under the stated rule.

  1. self definitional [Section 3.3.2, Eq. (8), Tables 5 and 6]
    "If recompilation fails, the score is 0, imposing a substantial penalty. The weight of this metric is 30%. ... Sbehav = 0.40×Slogic + 0.30×Sapi + 0.30×Serror ... API detection, a sub-metric of behavioral similarity, showed the largest effect size at d = 1.91, revealing that accurate restoration of external library calls is decisively important for recompilation success."

    Recompilation success/failure is the grouping variable in Section 4.2.3, and S_api is set to 0 whenever recompilation fails. Because S_behav includes S_api with weight 0.30, failure mechanically lowers S_behav by construction. Thus the observed success-group advantage in behavioral similarity (d=0.96) and the dominant sub-metric API detection (d=1.91) are partly artifacts of the scoring rule rather than independent evidence that behavioral similarity predicts recompilability. The definitional coupling is exposed by Table 6: a high-behavior failure group with mean S_behav=0.814 cannot exist if S_api=0 for all failures, since the maximum then is 0.40×1 + 0.30×0 + 0.30×1 = 0.70.

full rationale

The paper's central claim that behavioral similarity strongly predicts recompilation success is partly circular. Equation (8) defines S_behav as a weighted sum that includes S_api, and Section 3.3.2 explicitly zeroes S_api when recompilation fails. Since recompilation success is the outcome variable, the success group is guaranteed a higher S_behav, all else equal. The sub-metric API detection, with d=1.91, is exactly the component forced to zero on failure, so its reported 'decisive importance' is at least in part a scoring artifact. Moreover, Table 6 reports a high-behavior failure group with mean behavioral similarity 0.814, which is arithmetically impossible under the stated zero-if-failure rule (maximum 0.70). This internal inconsistency means the reported success-failure separation is not a clean validation of an independently defined metric. Structural similarity (d=0.69) and semantic similarity (d=0.33) do not have this direct definitional coupling, and the paper does not rely on any load-bearing self-citation chain. The circularity is therefore partial rather than total, but it affects the most emphasized predictive result, warranting a score of 7.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on hand-chosen weights, arbitrary thresholds, the assumption that recompilation success is a valid quality label, and the unverified notion that the metric components do not encode the outcome. No independent evidence is provided for these choices.

free parameters (6)
  • Weights of three major categories = S_struct 0.35, S_behav 0.40, S_sem 0.25
    Hand-chosen; Section 4.3 admits optimal weights remain unclear.
  • Sub-metric weights within structural similarity = S_sig 0.30, S_cfg 0.50, S_data 0.20
    Chosen by design, with CFG given the highest weight because it is considered most important.
  • Sub-metric weights within behavioral similarity = S_logic 0.40, S_api 0.30, S_error 0.30
    Hand-chosen, with logic operations weighted highest despite later showing an inverse effect size.
  • Sub-metric weights within semantic similarity = S_ident 0.50, S_var 0.30, S_type 0.20
    Hand-chosen with no stated justification beyond design intent.
  • Deduction rates, floors, and severity weights in scoring formulas = 0.15/0.10 per error; floors 0.2 and 0.3; severity weights 1.0/0.5
    Ad hoc constants in Eqs. (6) and (7) and Table 1, chosen to reflect presumed importance without a fitting procedure.
  • S_api penalty on recompilation failure = 0
    Section 3.3.2 sets S_api to 0 if recompilation fails, embedding the outcome into the predictor and making the success/failure comparison partly definitional.
assumptions (4)
  • domain assumption Recompilation success is a meaningful proxy for decompilation quality.
    Used as the grouping variable throughout Section 4, but no external ground truth is given for whether recompilable code is actually more faithful to the original binary.
  • ad hoc to paper The nine hand-weighted metrics measure structural, behavioral, and semantic similarity as intended.
    Section 4.3 acknowledges weights are based on prior work and preliminary experiments, with optimality unverified.
  • domain assumption CFG node and edge counts from the binary and from the decompiled C are comparable with the tools used.
    Section 3.2.2 does not specify how the CFG is extracted from each representation or how tool differences are reconciled.
  • domain assumption The 19k+ decompilation results are independent samples for t-tests.
    Multiple outputs come from the same program with different models, prompts, and compiler options, creating clustering that is not accounted for in the Section 4.2 analyses.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Statistical Analysis of Executability and Program Equivalence in Decompilation for IoT Vulnerability Detection." pith.science (2026). https://pith.science/paper/2UFWK3CY

@misc{pith2026260806960,
  author       = {Pith},
  title        = {Pith review of: Statistical Analysis of Executability and Program Equivalence in Decompilation for IoT Vulnerability Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2UFWK3CY}},
  note         = {Machine review of arXiv:2608.06960}
}
read the original abstract

Internet of Things (IoT) devices handle sensitive privacy-related information such as user audio, video, and authentication data, making it essential to detect vulnerabilities in their firmware. Decompilation, a key detection technique, has recently attracted attention because Large Language Models (LLMs) enable high readability and high recompilation success rates. However, because LLM outputs depend on probabilistic token prediction, they tend to prioritize syntactic correctness and may generate plausible-looking code that is semantically different from the original binary. Vulnerabilities often arise in details that are easily lost in this process, such as error-handling flows and boundary checks. Existing evaluation metrics focus mainly on passing test cases and cannot sufficiently identify code whose internal structure has been altered despite appearing behaviorally valid, so a metric that quantifies the internal structure of decompiled code from multiple perspectives is needed. We propose a nine-dimensional quality evaluation metric consisting of three categories: structural, behavioral, and semantic similarity. Targeting 318 programs from OpenWrt, an open-source router platform underlying many commercial routers, we generated 19,625 decompilation results using five methods (one rule-based and four LLM-based) and analyzed them statistically. The recompilation-success group achieved significantly higher overall scores than the failure group (Cohen's d=0.92); behavioral similarity showed d=0.96 and structural similarity d=0.69, demonstrating that these metrics are important predictors of decompilation quality. This study provides a statistical evaluation foundation for quantifying implementation defects in IoT devices and a framework that generalizes to quality evaluation of black-box generative models.

Figures

Figures reproduced from arXiv: 2608.06960 by the authors.

Figure 1
Figure 1. Overview of the evaluation system 4 Experiments 4.1 Experimental Overview We decompile programs from IoT firmware and evaluate the generation results of each model based on the proposed metrics. The target programs were taken from OpenWrt, an open-source router program. OpenWrt is a Linux-based firmware for routers (OpenWrt Project, 2004). It is adopted as the basis of many commercial routers, and its vulnerabilitie… view at source ↗
Figure 2
Figure 2. Distribution of overall scores 4.2 Experimental Results 4.2.1 Score Distribution When decompilation was performed on a total of 21,624 C programs, generation errors occurred during decompilation for 2,299 programs, and 402 programs had at least one of the nine metrics unmeasurable. Accordingly, in this experiment we analyze the 19,223 programs for which all nine metrics are measurable [PITH_FULL_IMAGE:figures/full_… view at source ↗
Figure 3
Figure 3. Score comparison between recompilation success and failure groups [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Radar chart of sub-metrics [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: Distribution of behavioral vs. structural similarity [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 49 canonical work pages

  1. [1]

    White Paper on Information and Communications in Japan, 2025 Edition , year =

    work page 2025

  2. [2]

    Ghidra Software Reverse Engineering Framework , year =

    work page

  3. [3]

    Network and Distributed System Security Symposium , year =

    Peiwei Hu and Ruigang Liang and Kai Chen , title =. Network and Distributed System Security Symposium , year =

    work page

  4. [4]

    2025 , url =

    Muqi Zou and Arslan Khan and Ruoyu Wu and Han Gao and Antonio Bianchi and Dave Jing Tian , title =. 2025 , url =

    work page 2025

  5. [5]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , year =

    Hanzhuo Tan and Qi Luo and Jing Li and Yuqun Zhang , title =. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , year =

    work page 2024

  6. [6]

    Detecting Vulnerability on IoT Device Firmware: A Survey , year=

    Feng, Xiaotao and Zhu, Xiaogang and Han, Qing-Long and Zhou, Wei and Wen, Sheng and Xiang, Yang , journal=. Detecting Vulnerability on IoT Device Firmware: A Survey , year=

    work page

  7. [7]

    2024 , isbn =

    Xiao, Haoyu and Zhang, Yuan and Shen, Minghang and Lin, Chaoyang and Zhang, Can and Liu, Shengli and Yang, Min , title =. 2024 , isbn =

    work page 2024

  8. [8]

    Proceedings of the 2024 International Conference on Artificial Intelligence of Things and Computing , pages =

    Wei, Zihan and Wei, Qiang and Geng, Yangyang and Yang, Yahui , title =. Proceedings of the 2024 International Conference on Artificial Intelligence of Things and Computing , pages =. 2025 , isbn =

    work page 2024

Show all 66 references
  1. [9]

    Proceedings of the 36th Annual Computer Security Applications Conference , pages =

    Kim, Mingeun and Kim, Dongkwan and Kim, Eunsoo and Kim, Suryeon and Jang, Yeongjin and Kim, Yongdae , title =. Proceedings of the 36th Annual Computer Security Applications Conference , pages =. 2020 , isbn =

    2020

  2. [10]

    Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security , pages =

    Kang, Wooseok and Son, Byoungho and Heo, Kihong , title =. Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security , pages =. 2022 , isbn =

    2022

  3. [11]

    2024 , howpublished =

    2024

  4. [12]

    Proceedings of 2017 IEEE Cybersecurity Development , pages =

    Fish Wang and Yan Shoshitaishvili , title =. Proceedings of 2017 IEEE Cybersecurity Development , pages =. 2017 , url =

    2017

  5. [13]

    Proceedings of the 20th ACM Asia Conference on Computer and Communications Security , pages =

    Sirlanci, Melih and Yagemann, Carter and Lin, Zhiqiang , title =. Proceedings of the 20th ACM Asia Conference on Computer and Communications Security , pages =. 2025 , isbn =

    2025

  6. [14]

    Proceedings of the 2024 IEEE/ACM International Symposium on Code Generation and Optimization , pages =

    Jordi Armengol-Estap. Proceedings of the 2024 IEEE/ACM International Symposium on Code Generation and Optimization , pages =. 2024 , url =

    2024

  7. [15]

    Proceedings of the 2025 International Symposium on Code Generation and Optimization , pages =

    Chris Cummins and Volker Seeker and Dejan Grubisic and Baptiste Rozi. Proceedings of the 2025 International Symposium on Code Generation and Optimization , pages =. 2025 , url =

    2025

  8. [16]

    2025 , url =

    Hanzhuo Tan and Xiaolong Tian and Hanrui Qi and Jiaming Liu and Zuchen Gao and Siyi Wang and Qi Luo and Jing Li and Yuqun Zhang , title =. 2025 , url =

    2025

  9. [17]

    2025 , url =

    Zuchen Gao and others , title =. 2025 , url =

    2025

  10. [18]

    2025 , url =

    others , title =. 2025 , url =

    2025

  11. [19]

    ACM Transactions on Software Engineering and Methodology (TOSEM) , year =

    others , title =. ACM Transactions on Software Engineering and Methodology (TOSEM) , year =

  12. [20]

    Proceedings of the ACM on Software Engineering , volume =

    Wai Kin Wong and Daoyuan Wu and Huaijin Wang and Zongjie Li and Zhibo Liu and Shuai Wang and Qiyi Tang and Sen Nie and Shi Wu , title =. Proceedings of the ACM on Software Engineering , volume =. 2025 , url =

    2025

  13. [21]

    2020 , url =

    Shuo Ren and Daya Guo and Shuai Lu and Long Zhou and Shujie Liu and Duyu Tang and Neel Sundaresan and Ming Zhou and Ambrosio Blanco and Shuai Ma , title =. 2020 , url =

    2020

  14. [22]

    Journal of Systems and Software , numpages =

    Evtikhiev, Mikhail and Bogomolov, Egor and Sokolov, Yaroslav and Bryksin, Timofey , title =. Journal of Systems and Software , numpages =. 2023 , issue_date =

    2023

  15. [23]

    ExeBench: an ML-scale dataset of executable C functions , year =

    Armengol-Estap\'. ExeBench: an ML-scale dataset of executable C functions , year =. Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming , pages =

  16. [24]

    ACM Transactions on Software Engineering and Methodology , articleno =

    Dong, Yihong and Ding, Jiazheng and Jiang, Xue and Li, Ge and Li, Zhuo and Jin, Zhi , title =. ACM Transactions on Software Engineering and Methodology , articleno =. 2025 , issue_date =

    2025

  17. [25]

    John , title =

    Cifuentes, Cristina and Gough, K. John , title =. Software: Practice and Experience , volume =. 1995 , publisher =

    1995

  18. [26]

    A survey on software smells , journal =

    Tushar Sharma and Diomidis Spinellis , keywords =. A survey on software smells , journal =. 2018 , issn =

    2018

  19. [27]

    Empirical Softw

    Ragkhitwetsagul, Chaiyong and Krinke, Jens and Clark, David , title =. Empirical Softw. Engg. , month = aug, pages =. 2018 , issue_date =

    2018

  20. [28]

    Benchmarks and Metrics for Evaluations of Code Generation: A Critical Review , year =

    Paul, Debalina Ghosh and Zhu, Hong and Bayley, Ian , booktitle =. Benchmarks and Metrics for Evaluations of Code Generation: A Critical Review , year =

  21. [29]

    1988 , address =

    Cohen, Jacob , title =. 1988 , address =

    1988

  22. [30]

    Psychological Bulletin , volume =

    Jacob Cohen , title =. Psychological Bulletin , volume =. 1992 , url =

    1992

  23. [31]

    Image Vision Comput

    Riesen, Kaspar and Bunke, Horst , title =. Image Vision Comput. , pages =. 2009 , issue_date =

    2009

  24. [32]

    , location =

    Allen, Frances E. , location =. Control flow analysis , year =. SIGPLAN Not. , pages =

  25. [33]

    , title =

    Allen, Frances E. , title =. Proceedings of a Symposium on Compiler Optimization , pages =. 1970 , isbn =

    1970

  26. [34]

    Using Effect Size—or Why the P Value Is Not Enough , volume =

    Sullivan, Gail and Feinn, Richard , year =. Using Effect Size—or Why the P Value Is Not Enough , volume =. Journal of graduate medical education , url =

  27. [35]

    Predicting Vulnerable Software Components via Text Mining , year=

    Scandariato, Riccardo and Walden, James and Hovsepyan, Aram and Joosen, Wouter , journal=. Predicting Vulnerable Software Components via Text Mining , year=

  28. [36]

    Common Weakness Scoring System (CWSS) , howpublished =

  29. [37]

    and Metzener, David E

    Ratcliff, John W. and Metzener, David E. , title =. Dr. Dobb's Journal , volume =. 1988 , url =

    1988

  30. [38]

    The Distribution of the Flora in the Alpine Zone , urldate =

    Paul Jaccard , journal =. The Distribution of the Flora in the Alpine Zone , urldate =

  31. [39]

    , title =

    McCabe, Thomas J. , title =. IEEE Transactions on Software Engineering , year =

  32. [40]

    Frances E. Allen. Control flow analysis. In Proceedings of a Symposium on Compiler Optimization, page 1–19, New York, NY, USA, 1970. Association for Computing Machinery. ISBN 9781450373869. URL https://doi.org/10.1145/800028.808479

    1970

  33. [41]

    Jordi Armengol-Estap\' e , Jackson Woodruff, Alexander Brauckmann, Jos\' e Wesley de Souza Magalh\ a es, and Michael F. P. O'Boyle. Exebench: an ml-scale dataset of executable c functions. In Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming, pa...

    2022

  34. [42]

    Jordi Armengol-Estap \'e , Jackson Woodruff, Chris Cummins, and Michael F. P. O'Boyle. SLaDe : A portable small language model decompiler for optimized assembly. In Proceedings of the 2024 IEEE/ACM International Symposium on Code Generation and Optimization, pages 67--80, 2024...

    2024

  35. [43]

    John Gough

    Cristina Cifuentes and K. John Gough. Decompilation of binary programs. Software: Practice and Experience, 25 0 (7): 0 811--829, 1995. URL https://doi.org/10.1002/spe.4380250703

    1995 doi

  36. [44]

    LLM Compiler : Foundation language models for compiler optimization

    Chris Cummins, Volker Seeker, Dejan Grubisic, Baptiste Rozi \`e re, Jonas Gehring, Gabriel Synnaeve, and Hugh Leather. LLM Compiler : Foundation language models for compiler optimization. In Proceedings of the 2025 International Symposium on Code Generation and Optimization, p...

    2025

  37. [45]

    Codescore: Evaluating code generation by learning code execution

    Yihong Dong, Jiazheng Ding, Xue Jiang, Ge Li, Zhuo Li, and Zhi Jin. Codescore: Evaluating code generation by learning code execution. ACM Transactions on Software Engineering and Methodology, 34 0 (3), 2025. ISSN 1049-331X. URL https://doi.org/10.1145/3695991

    2025 doi

  38. [46]

    Out of the bleu: How should we assess quality of the code generation models? Journal of Systems and Software, 203 0 (C), 2023

    Mikhail Evtikhiev, Egor Bogomolov, Yaroslav Sokolov, and Timofey Bryksin. Out of the bleu: How should we assess quality of the code generation models? Journal of Systems and Software, 203 0 (C), 2023. ISSN 0164-1212. URL https://doi.org/10.1016/j.jss.2023.111741

    2023

  39. [47]

    Detecting vulnerability on iot device firmware: A survey

    Xiaotao Feng, Xiaogang Zhu, Qing-Long Han, Wei Zhou, Sheng Wen, and Yang Xiang. Detecting vulnerability on iot device firmware: A survey. IEEE/CAA Journal of Automatica Sinica, 10 0 (1): 0 25--41, 2023. URL https://doi.org/10.1109/JAS.2022.105860

    2023

  40. [48]

    DecompileBench : A comprehensive benchmark for evaluating decompilers in real-world scenarios, 2025

    Zuchen Gao et al. DecompileBench : A comprehensive benchmark for evaluating decompilers in real-world scenarios, 2025. URL https://arxiv.org/abs/2505.11340. Last accessed January 6, 2026

    2025 arXiv

  41. [49]

    Hex-Rays SA . IDA Pro . https://hex-rays.com/ida-pro/, 2024. Last accessed January 6, 2026

    2024

  42. [50]

    DeGPT : Optimizing decompiler output with LLM

    Peiwei Hu, Ruigang Liang, and Kai Chen. DeGPT : Optimizing decompiler output with LLM . In Network and Distributed System Security Symposium, 2024. URL https://www.ndss-symposium.org/ndss-paper/degpt-optimizing-decompiler-output-with-llm/. Last accessed January 6, 2026

    2024

  43. [51]

    The distribution of the flora in the alpine zone

    Paul Jaccard. The distribution of the flora in the alpine zone. The New Phytologist, 11 0 (2): 0 37--50, 1912. ISSN 0028646X, 14698137. URL http://www.jstor.org/stable/2427226

    1912

  44. [52]

    Firmae: Towards large-scale emulation of iot firmware for dynamic analysis

    Mingeun Kim, Dongkwan Kim, Eunsoo Kim, Suryeon Kim, Yeongjin Jang, and Yongdae Kim. Firmae: Towards large-scale emulation of iot firmware for dynamic analysis. In Proceedings of the 36th Annual Computer Security Applications Conference, page 733–745, New York, NY, USA, 2020. A...

    2020

  45. [53]

    Thomas J. McCabe. A complexity measure. IEEE Transactions on Software Engineering, SE-2: 0 308--320, 1976. URL https://doi.org/10.1109/TSE.1976.233837

    1976

  46. [54]

    White paper on information and communications in japan, 2025 edition

    Ministry of Internal Affairs and Communications . White paper on information and communications in japan, 2025 edition. https://www.soumu.go.jp/johotsusintokei/whitepaper/ja/r07/html/datashu.html, 2025. Last accessed January 6, 2026

    2025

  47. [55]

    Ghidra software reverse engineering framework

    National Security Agency . Ghidra software reverse engineering framework. https://ghidra-sre.org/, 2019. Last accessed January 6, 2026

    2019

  48. [56]

    Open W rt W ireless F reedom

    OpenWrt Project . Open W rt W ireless F reedom. https://openwrt.org/, 2004. Last accessed January 6, 2026

    2004

  49. [57]

    Benchmarks and metrics for evaluations of code generation: A critical review, 2024

    Debalina Ghosh Paul, Hong Zhu, and Ian Bayley. Benchmarks and metrics for evaluations of code generation: A critical review, 2024. URL https://doi.org/10.1109/AITest62860.2024.00019

    2024

  50. [59]

    CodeBLEU : a method for automatic evaluation of code synthesis, 2020

    Shuo Ren, Daya Guo, Shuai Lu, Long Zhou, Shujie Liu, Duyu Tang, Neel Sundaresan, Ming Zhou, Ambrosio Blanco, and Shuai Ma. CodeBLEU : a method for automatic evaluation of code synthesis, 2020. URL https://arxiv.org/abs/2009.10297. Last accessed January 6, 2026

    2020 arXiv

  51. [60]

    Predicting vulnerable software components via text mining

    Riccardo Scandariato, James Walden, Aram Hovsepyan, and Wouter Joosen. Predicting vulnerable software components via text mining. IEEE Transactions on Software Engineering, 40 0 (10): 0 993--1006, 2014. URL https://doi.org/10.1109/TSE.2014.2340398

    2014

  52. [61]

    An empirical study of c decompilers: Performance metrics and error taxonomy

    Melih Sirlanci, Carter Yagemann, and Zhiqiang Lin. An empirical study of c decompilers: Performance metrics and error taxonomy. In Proceedings of the 20th ACM Asia Conference on Computer and Communications Security, page 1707–1723, New York, NY, USA, 2025. Association for Comp...

    2025

  53. [62]

    Using effect size—or why the p value is not enough

    Gail Sullivan and Richard Feinn. Using effect size—or why the p value is not enough. Journal of graduate medical education, 4: 0 279--82, 2012. URL https://doi.org/10.4300/JGME-D-12-00156.1

    2012 doi

  54. [63]

    LLM4Decompile : Decompiling binary code with large language models

    Hanzhuo Tan, Qi Luo, Jing Li, and Yuqun Zhang. LLM4Decompile : Decompiling binary code with large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024. URL https://doi.org/10.18653/v1/2024.emnlp-main.203

    2024 doi

  55. [64]

    Common weakness scoring system (cwss)

    The MITRE Corporation . Common weakness scoring system (cwss). https://cwe.mitre.org/cwss/, 2014. Last accessed January 6, 2026

    2014

  56. [65]

    Angr -- the next generation of binary analysis

    Fish Wang and Yan Shoshitaishvili. Angr -- the next generation of binary analysis. In Proceedings of 2017 IEEE Cybersecurity Development, pages 8--9, 2017. URL https://doi.org/10.1109/SecDev.2017.14

    2017 doi

  57. [66]

    A survey on iot security: Vulnerability detection and protection

    Zihan Wei, Qiang Wei, Yangyang Geng, and Yahui Yang. A survey on iot security: Vulnerability detection and protection. In Proceedings of the 2024 International Conference on Artificial Intelligence of Things and Computing, page 1–8, New York, NY, USA, 2025. Association for Com...

    2024

  58. [67]

    D-LiFT : Improving LLM -based decompiler backend via code quality-driven fine-tuning, 2025

    Muqi Zou, Arslan Khan, Ruoyu Wu, Han Gao, Antonio Bianchi, and Dave Jing Tian. D-LiFT : Improving LLM -based decompiler backend via code quality-driven fine-tuning, 2025. URL https://arxiv.org/abs/2506.10125. Last accessed January 6, 2026

    2025 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.