Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

VulBinLLM: LLM-powered Vulnerability Detection for Stripped Binaries

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read Vul-BinLLM classifies CWEs in stripped binaries with over 96% accuracy on the Juliet suite by having an LLM add vulnerability comments during decompilation and a memory queue extend the context window.

desk verdict Plausible pipeline design, but the main result is confounded because the first GPT-4o stage injects vulnerability-hinting comments that the second stage reads, and Table 3's comparison is not on matched data. read the letter →

arxiv 2505.22010 v1 pith:QTM5JPHW submitted 2025-05-28 cs.CR

classification cs.CR
keywords binaryvulnerabilitydetectionstrippedbinarieslargelanguagemodelsdecompilationoptimizationCWEclassificationmemorymanagementchain-of-thoughtpromptingJuliettestsuite
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a large language model can classify vulnerability types (CWEs) in stripped binaries — executables with symbols and debug information removed — by combining two LLM passes. The first pass rewrites decompiled code, renaming variables, simplifying structure, and appending comments that point at suspicious patterns; the second pass decides which CWE, if any, is present, using a memory store and a function queue to cover code larger than the model's context window. On compiled Juliet test cases, the system reports accuracies above 96% on CWE-78, CWE-134, CWE-190, and CWE-606, beating LATTE, the prior LLM-based binary taint-analysis baseline, and claims roughly a 10% accuracy gain. If these results hold, vulnerability triage of closed-source and legacy binaries could become feasible without source access or model fine-tuning.

What carries the argument

The load-bearing mechanism is a two-stage LLM orchestration. A decompilation-optimization stage takes raw decompiler output and, through separate action agents for variables, structures, and comments, produces code with meaningful names and explicit annotations such as 'Vulnerability: Potential pointer arithmetic issue.' A detection stage then feeds that annotated code to GPT-4o alongside CWE descriptions, using few-shot chain-of-thought prompting, in-context learning, a memory management agent backed by archival storage, and a function queue ('VulBinQ') that schedules analysis of every function so the whole binary is covered even when it exceeds the context window. The stated purpose of the comments is to supply contextual information without altering code structure or functionality, so the detection stage is meant to classify based on the code as enriched.

What would settle it

Run the classification stage on the same stripped Juliet binaries decompiled without the vulnerability comments (for example, with only renamed variables and no annotations), and compare accuracy. If accuracy remains high, the comments are not necessary; if it collapses, the reported scores measure the first stage's hint-writing rather than binary-level vulnerability detection. As a further check, present the comments alone to the classifier and ask for a CWE label; if the label is recoverable from the annotations alone, the second stage is not the locus of detection.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a two-stage LLM pipeline can map stripped binary code to CWE labels at high accuracy, and that the first stage — decompilation optimization — is what makes the second stage work. The paper reports that Vul-BinLLM matches or beats LATTE across the four evaluated CWEs, with zero false negatives in its reported runs, and attributes the improvement to vulnerability-focused comments added during decompilation plus an archival memory that lets analysis proceed without a fixed context-window limit. The design goal is not to reconstruct the original source code faithfully but to make vulnerable features prominent enough for an LLM to reason about them.

Load-bearing premise

The load-bearing premise is that the comments appended during decompilation highlight suspicious patterns without giving away the vulnerability class, so the second stage's high accuracy reflects reasoning about the code rather than reading hints written by the first stage; the paper does not test what happens when those comments are removed.

Editorial extensions

If this is right

  • If the central claim is right, CWE classification of stripped binaries becomes a no-fine-tuning LLM task: decompile, annotate, classify, with no per-project training data.
  • The reported roughly 10% accuracy gain over LATTE implies that making vulnerable patterns prominent during decompilation contributes more to detection than prompting alone, shifting attention to decompilation quality in future tools.
  • The memory-and-queue design implies that binary size no longer has to fit the model's context window, so the same pipeline can be applied to binaries with many functions by analyzing them one at a time with shared context.
  • The paper's claim that Vul-BinLLM needs no fine-tuning and is flexible across programming languages implies that porting the approach to other compilers or languages would mainly require changing the decompilation front end.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the cleanest way to learn what is doing the classification is to run the detection stage on decompiled code without the LLM-added comments; the absence of such an ablation leaves open how much of the accuracy comes from reasoning versus hint-following.
  • Beyond the paper: the comment-generation stage could be reused independently as a vulnerability-highlighting tool for human reverse engineers, since it already produces natural-language pointers to suspicious code.
  • Beyond the paper: the memory-store design suggests a concrete scaling test — run the pipeline on stripped binaries with hundreds of functions and watch whether accuracy degrades at the point where older per-function analyses must be evicted from the store.
  • Beyond the paper: the same two-stage pattern could be applied to architectures beyond the tested compiled C/C++ by swapping the decompiler, because the pipeline's language-level reasoning operates on decompiled output rather than raw machine code.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Vul-BinLLM, a two-stage LLM-based framework for CWE-level vulnerability detection in stripped binaries. The first stage post-processes RetDec/Ghidra decompilation with GPT-4o to rename variables, simplify structures, and append vulnerability- and functionality-related comments. The second stage uses GPT-4o with in-context learning, chain-of-thought prompting, a SQL-backed memory store, and a function-analysis queue to classify decompiled functions against MITRE CWE descriptions. The evaluation is conducted on compiled Juliet C/C++ tests and reports per-CWE accuracy, precision, and F1 for CWE-78, CWE-134, CWE-190, and CWE-606, claiming roughly 10% higher accuracy than the LATTE baseline.

Significance. The problem is important and the proposed two-stage design is a concrete, plausible way to combine neural decompilation and LLM reasoning, with a memory/queue mechanism that addresses context-window limits. The paper also provides a comparative evaluation on a standard synthetic suite and is explicit about the limitation that only Juliet is used. If validated, the framework would be a practical addition to binary reverse-engineering workflows. However, the current evidence is not sufficient to support the state-of-the-art claim: the central evaluation table contains internal inconsistencies, and no ablation isolates the contribution of the vulnerability comments that the first stage injects, so the detector's high accuracy may simply reflect reading the first stage's hints.

major comments (4)
  1. [Table 3] The reported Vul-BinLLM TP of 1218 for CWE-606 cannot be correct if, as the column header states, CWE-606 has only 240 bad test cases; a true-positive count is bounded by the number of positive samples, and 1218 exceeds 240. The reported accuracy for CWE-606 (74.54%) also does not match the table's own TP/TN/FP/FN values, which yield (1218+4125)/(1218+4125+416+0) ≈ 92.8%. In addition, the Vul-BinLLM columns for the four CWEs use total counts of 5534, 5357, 5378, and 5759, whereas the LATTE columns use 1920, 2400, 5720, and 480, so the two methods are not being compared on the same number of files. The test protocol and all cell definitions must be clarified and the table corrected before any accuracy comparison can be interpreted.
  2. [Sections 3.2.1 and 4.1, Figures 3 and 5] The decompilation stage is itself an LLM that identifies weaknesses and appends comments such as 'Vulnerability: Potential pointer arithmetic issue if column_1 and row_2 are not properly allocated,' and the detection stage is then asked yes/no whether the code has OS Command Injection while focusing on the dataflow. Because the first stage already names or strongly hints at the weakness, the second-stage detector has access to the first stage's vulnerability assessment, and the reported per-CWE accuracies could be an artifact of following these hints rather than performing binary analysis. The paper needs an ablation in which the vulnerability-specific comments are removed while renaming and structural simplification are kept, to show that the detection stage does not depend on leaked labels; without such an ablation the central feasibility claim is untested.
  3. [Sections 4.2 and 4.3] The research questions are not answered by the reported experiments. RQ2 asks whether making vulnerability features prominent improves detection, but the evaluation compares only the full Vul-BinLLM pipeline with LATTE; there is no baseline that applies the same GPT-4o detector to raw RetDec/Ghidra output or to decompilation optimized only for readability. Similarly, the memory-management agent and VulBinQ are core contributions, yet no experiment removes or truncates the memory/queue to quantify their effect. These ablations are needed to attribute the claimed accuracy gains to the proposed components.
  4. [Sections 4.3 and 5] The comparison to LATTE is not controlled. The footnote to Table 3 says LATTE is run with 'additional information alongside the code,' the Vul-BinLLM totals differ from LATTE's totals as noted above, and no error bars or repeated-run variance are reported for any GPT-4o result. The paper appropriately limits itself to the synthetic Juliet suite and states that real-world binaries are future work, but the abstract's 'state-of-the-art performance' claim is therefore only about one synthetic benchmark and is not yet established relative to other binary vulnerability detectors.
minor comments (5)
  1. [Section 4.3] The text states that 'Table 2 summarizes the number of test cases used in our analysis,' but Table 2 lists LLM models and their performance on two vulnerability examples, not test-case counts; the cross-reference is incorrect.
  2. [Section 4.1] The implementation description says the framework is built on 'Ghidra and [9]' without naming [9] in the text; it should read 'Ghidra and RetDec [9]'.
  3. [Figure 3] The original source snippet shows an off-by-one loop bound ('k <= C1'), but the Vul-BinLLM output shows 'shared_dim < row_2'; the relationship between the two bounds should be explained so readers can see whether the vulnerability is preserved.
  4. [Section 2 heading] The heading 'Reserve Engineering for Vulnerability Analysis' should read 'Reverse Engineering for Vulnerability Analysis'.
  5. [Table 3] The footnote markers are formatted with '+:' and the parenthetical explanations are ambiguous; each cell should be defined explicitly, for instance whether the counts are per-file or per-function prompts.

Circularity Check

2 steps flagged · score 6.0 of 10

The claimed vulnerability detection is partly self-feeding: the decompiler stage is told to 'identify weaknesses' and append vulnerability comments (Figure 3 shows 'Vulnerability: Potential pointer arithmetic issue'), and the second-stage classifier then labels CWEs from that same annotated code, with no ablation removing the injected comments.

  1. self definitional [Section 3.2.1 (Vul-BinLLM - Vulnerability Prominence) and Section 4.1 (Implementation)]
    "This optimization process involves identifying and emphasizing code patterns that are commonly associated with vulnerabilities... By highlighting these vulnerability-related features, the recovered source code becomes a more informative and targeted input for LLMs, guiding their analysis towards potential security flaws. ... Through the API of GPT-4.0, Vul-BinLLM identifies weaknesses in the binary and augments the output by (1) appending comments about potential vulnerabilities and functionality..."

    The independent variable whose effect the paper claims to measure, 'decompilation optimization', is implemented as the first LLM identifying weaknesses and appending vulnerability comments. The dependent variable, the later vulnerability/CWE classification, is then produced from that same annotated text. The claimed gain from 'making vulnerability features more prominent' is therefore built into the construction: the optimization step already performs the vulnerability identification that the detection step is asked to reproduce.

  2. fitted input called prediction [Figure 3 caption and Section 4.3 / Figure 5 prompt]
    "Vul-BinLLM further augments the output by adding vulnerability-specific annotations, such as warnings about potential pointer arithmetic issues that could lead to buffer overflows or memory access vulnerabilities. ... based on the following code, does it have the OS Command Injection? Focus on the dataflow."

    The first GPT-4o stage injects the comment 'Vulnerability: Potential pointer arithmetic issue...' into the code, and the second GPT-4o stage is then asked to answer a CWE yes/no question about that same annotated code. Because no ablation removes the vulnerability comments, the high per-CWE accuracies in Table 3 (e.g., 99.74% for CWE-134 and 99.34% for CWE-190) can be achieved by the detector following the first stage's injected weakness hints rather than by independent analysis of the stripped binary. The predicted label is thus, in part, the first stage's own judgment fed back as input.

full rationale

Vul-BinLLM's central feasibility claim is that decompilation optimization plus long-term memory enables state-of-the-art stripped-binary vulnerability detection. The evidence for that claim is weakened by a structural feedback loop inside the pipeline. Section 4.1 says the decompiler 'identifies weaknesses in the binary and augments the output' with comments about potential vulnerabilities, and Figure 3 shows that this augmentation literally inserts the text 'Vulnerability: Potential pointer arithmetic issue...' into the decompiled code. The detection stage then classifies CWEs from that already-annotated code (Figure 5), and the paper provides no ablation that strips out only the added vulnerability comments. Consequently, the second stage's high accuracy in Table 3 is not an independent measurement of vulnerability detection from binaries: it can be explained by the detector reading the first stage's own weakness judgment. This is a partial circularity rather than a total one, because the memory queue, function queue, renaming, and structural simplification are separate components with at least some independent content, and Juliet is an external benchmark. There is no load-bearing self-citation or imported uniqueness theorem. The score reflects that one or more of the claimed predictions reduce, by construction, to the model-generated annotation fed into them.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central claim depends on three unverified premises: decompiled code plus injected comments preserves the binary's semantics; GPT-4o can reason about CWE presence from annotated code without fine-tuning; and the memory queue provides complete and faithful program context. The comment-injection premise doubles as a potential answer leak, making it the most consequential assumption.

assumptions (3)
  • domain assumption RetDec/Ghidra decompilation preserves the functional behavior of the original binary such that LLM-injected comments do not alter semantics.
    The pipeline modifies decompiled code by adding comments and renaming variables, claiming functionality is unchanged; if decompilation or comment injection corrupts semantics, detection results are invalid. Invoked in Section 3.2.1 and Figure 3.
  • domain assumption GPT-4o can reliably identify vulnerability-relevant patterns in decompiled code when provided with CWE descriptions and few-shot examples.
    The entire method rests on GPT-4o's zero/few-shot reasoning; the paper provides no fine-tuning or external validation. Invoked throughout Section 3.2.1 and Section 4.1.
  • ad hoc to paper The memory management agent and function queue provide complete coverage and avoid context loss.
    The system design assumes that summarizing prior functions in a SQL archival store is sufficient for cross-function vulnerability reasoning; no experiments verify this. Invoked in Section 3.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VulBinLLM: LLM-powered Vulnerability Detection for Stripped Binaries." pith.science (2026). https://pith.science/paper/QTM5JPHW

@misc{pith2026250522010,
  author       = {Pith},
  title        = {Pith review of: VulBinLLM: LLM-powered Vulnerability Detection for Stripped Binaries},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QTM5JPHW}},
  note         = {Machine review of arXiv:2505.22010}
}
read the original abstract

Recognizing vulnerabilities in stripped binary files presents a significant challenge in software security. Although some progress has been made in generating human-readable information from decompiled binary files with Large Language Models (LLMs), effectively and scalably detecting vulnerabilities within these binary files is still an open problem. This paper explores the novel application of LLMs to detect vulnerabilities within these binary files. We demonstrate the feasibility of identifying vulnerable programs through a combined approach of decompilation optimization to make the vulnerabilities more prominent and long-term memory for a larger context window, achieving state-of-the-art performance in binary vulnerability analysis. Our findings highlight the potential for LLMs to overcome the limitations of traditional analysis methods and advance the field of binary vulnerability detection, paving the way for more secure software systems. In this paper, we present Vul-BinLLM , an LLM-based framework for binary vulnerability detection that mirrors traditional binary analysis workflows with fine-grained optimizations in decompilation and vulnerability reasoning with an extended context. In the decompilation phase, Vul-BinLLM adds vulnerability and weakness comments without altering the code structure or functionality, providing more contextual information for vulnerability reasoning later. Then for vulnerability reasoning, Vul-BinLLM combines in-context learning and chain-of-thought prompting along with a memory management agent to enhance accuracy. Our evaluations encompass the commonly used synthetic dataset Juliet to evaluate the potential feasibility for analysis and vulnerability detection in C/C++ binaries. Our evaluations show that Vul-BinLLM is highly effective in detecting vulnerabilities on the compiled Juliet dataset.

Figures

Figures reproduced from arXiv: 2505.22010 by the authors.

Figure 1
Figure 1. Workflow of Vul-BinLLM : (1) Binary files are decompiled into source code, where an LLM-assisted decompiler enriches the code with contextual information for vulnerability detection. (2) The decompiled source code is then analyzed by Vul-BinLLM for vulnerability analysis which has an archival storage to store analysis, The analyzer then provides a comprehensive vulnerability detection result (3) VulBinQ: It features… view at source ↗
Figure 2
Figure 2. An example of a buffer overflow vulnerability: the [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An example of Matrix Multiplication decompilation output across different stages: the original source code, Ghidra, Ghidra enhanced with GPT-4o, and Vul-BinLLM . The Ghidra decompilation provides a low-level representation with generic variable names and lacks context, making the functionality and security aspects harder to interpret. Ghidra + GPT-4o improves readability with meaningful variable names and clarifying… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Vul-BinLLM -Decompiler overview. Vul-BinLLM -decompiler includes an Optimization Decision Agent and three Action Agents (Vul-variable, Vul-struct, Vul-comment). After getting raw decompilation output from reverse engineering tool, Vul-BinLLM -decompiler will perform an…
Figure 5
Figure 5. Figure 5: An example of binary classification with CWE-78. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Veritas: Grounding LLM Agents for Reliable Vulnerability Reasoning over Stripped Binaries

    cs.SE 2026-05 unverdicted novelty 7.0 of 10

    Veritas detects out-of-bounds vulnerabilities in stripped binaries at 90% recall by grounding LLM reasoning in static witness-backed flows and runtime validation.

Reference graph

Works this paper leans on

125 extracted references · 35 canonical work pages · cited by 1 Pith paper

  1. [1]

    https://ai.meta.com/blog/code-llama-large- language-model-coding/

    2023.CodeLlaMA by Meta AI. https://ai.meta.com/blog/code-llama-large- language-model-coding/

  2. [2]

    https://openai.com/research/gpt-4

    2023.GPT-4 by OpenAI. https://openai.com/research/gpt-4

  3. [3]

    https://huggingface.co/datasets/bstee615/bigvul

    2024.Big-Vul Dataset. https://huggingface.co/datasets/bstee615/bigvul

  4. [4]

    https://github.com/epicosy/devign

    2024.Devign Dataset. https://github.com/epicosy/devign

  5. [5]

    https://ghidra-sre.org/

    2024.Ghidra. https://ghidra-sre.org/

  6. [6]

    https://hex-rays.com/ida-pro

    2024.IDA Pro. https://hex-rays.com/ida-pro

  7. [7]

    https://samate.nist.gov/SARD/ testsuite.php

    2024.Juliet Test Suite for C/C++ and Java. https://samate.nist.gov/SARD/ testsuite.php

  8. [8]

    https://github.com/microsoft/MS- MVD

    2024.Microsoft Vulnerability Dataset (MVD). https://github.com/microsoft/MS- MVD

Show all 125 references
  1. [9]

    https://github.com/avast/retdec

    2024.RetDec. https://github.com/avast/retdec

  2. [10]

    https://github.com/VulDetProject/ReVeal

    2024.REVEAL Dataset. https://github.com/VulDetProject/ReVeal

  3. [11]

    https://samate.nist.gov/ SARD/

    2024.Software Assurance Reference Dataset (SARD). https://samate.nist.gov/ SARD/

  4. [12]

    https://github.com/tuhh-softsec/vul4j

    2024.Vul4J Dataset. https://github.com/tuhh-softsec/vul4j

  5. [13]

    Iftakhar Ahmad and Lannan Luo. 2023. Unsupervised Binary Code Translation with Application to Code Clone Detection and Vulnerability Discovery. In Findings of the Association for Computational Linguistics: EMNLP 2023. 14581– 14592

  6. [14]

    Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang

  7. [15]

    2024.Codestral

    Mistral AI. 2024.Codestral. https://mistral.ai/news/codestral/

  8. [16]

    2024.Codestral Mamba

    Mistral AI. 2024.Codestral Mamba. https://mistral.ai/news/codestral-mamba/

  9. [17]

    2024.Mixture of Experts Models

    Mistral AI. 2024.Mixture of Experts Models. https://mistral.ai/news/mixtral-of- experts/

  10. [18]

    Ioannis Angelakopoulos, Gianluca Stringhini, and Manuel Egele. 2023. {FirmSolo}: Enabling dynamic analysis of binary Linux-based{IoT} kernel modules. In32nd USENIX Security Symposium (USENIX Security 23). 5021–5038

  11. [19]

    2024.Claude

    anthropic. 2024.Claude. https://www.anthropic.com/claude

  12. [20]

    Tom B Brown. 2020. Language models are few-shot learners.arXiv preprint arXiv:2005.14165(2020)

  13. [21]

    Saikat Chakraborty, Rahul Krishna, Yangruibo Ding, and Baishakhi Ray. 2021. Deep learning based vulnerability detection: Are we there yet?IEEE Transactions on Software Engineering48, 9 (2021), 3280–3296

  14. [22]

    Ligeng Chen, Zhongling He, and Bing Mao. 2020. Cati: Context-assisted type inference from stripped binaries. In2020 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). IEEE, 88–98

  15. [23]

    Qibin Chen, Jeremy Lacomis, Edward J Schwartz, Claire Le Goues, Graham Neubig, and Bogdan Vasilescu. 2022. Augmenting decompiler output with learned variable names and types. In31st USENIX Security Symposium (USENIX Security 22). 4327–4343

  16. [24]

    Sanchuan Chen, Zhiqiang Lin, and Yinqian Zhang. 2021. {SelectiveTaint}: Efficient Data Flow Tracking With Static Binary Rewriting. In30th USENIX Security Symposium (USENIX Security 21). 1665–1682. 10

  17. [25]

    Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. 2023. Teaching large language models to self-debug.arXiv preprint arXiv:2304.05128(2023)

  18. [26]

    Victor Cochard, Damian Pfammatter, Chi Thang Duong, and Mathias Humbert

  19. [27]

    Tri Dao and Albert Gu. 2024. Transformers are SSMs: Generalized models and efficient algorithms through structured state space duality.arXiv preprint arXiv:2405.21060(2024)

  20. [28]

    Yaniv David, Uri Alon, and Eran Yahav. 2020. Neural reverse engineering of stripped binaries using augmented control flow graphs.Proceedings of the ACM on Programming Languages4, OOPSLA (2020), 1–28

  21. [29]

    Yinlin Deng, Chunqiu Steven Xia, Haoran Peng, Chenyuan Yang, and Lingming Zhang. 2023. Large language models are zero-shot fuzzers: Fuzzing deep- learning libraries via large language models. InProceedings of the 32nd ACM SIGSOFT international symposium on software testing and...

  22. [30]

    Jacob Devlin. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805(2018)

  23. [31]

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, et al. 2022. A survey on in-context learning.arXiv preprint arXiv:2301.00234(2022)

  24. [32]

    Xueying Du, Geng Zheng, Kaixin Wang, Jiayi Feng, Wentai Deng, Mingwei Liu, Bihuan Chen, Xin Peng, Tao Ma, and Yiling Lou. 2024. Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAG.arXiv preprint arXiv:2406.11147(2024)

  25. [33]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models.arXiv preprint arXiv:2407.21783(2024)

  26. [34]

    Jiahao Fan, Yi Li, Shaohua Wang, and Tien N Nguyen. 2020. AC/C++ code vulnerability dataset with code changes and CVE summaries. InProceedings of the 17th International Conference on Mining Software Repositories. 508–512

  27. [35]

    Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, et al . 2020. Codebert: A pre-trained model for programming and natural languages.arXiv preprint arXiv:2002.08155(2020)

  28. [36]

    Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Wen-tau Yih, Luke Zettlemoyer, and Mike Lewis. 2022. Incoder: A generative model for code infilling and synthesis.arXiv preprint arXiv:2204.05999 (2022)

  29. [37]

    Michael Fu and Chakkrit Tantithamthavorn. 2022. Linevul: A transformer- based line-level vulnerability prediction. InProceedings of the 19th International Conference on Mining Software Repositories. 608–620

  30. [38]

    Jian Gao, Xin Yang, Ying Fu, Yu Jiang, and Jiaguang Sun. 2018. Vulseeker: A semantic learning based vulnerability seeker for cross-platform binary. InPro- ceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering. 896–899

  31. [39]

    Mingyang Geng, Shangwen Wang, Dezun Dong, Haotian Wang, Ge Li, Zhi Jin, Xiaoguang Mao, and Xiangke Liao. 2023. An empirical study on using large language models for multi-intent comment generation.arXiv preprint arXiv:2304.11384(2023)

  32. [40]

    Redha Gouicem, Dennis Sprokholt, Jasper Ruehl, Rodrigo CO Rocha, Tom Spink, Soham Chakraborty, and Pramod Bhatotia. 2022. Risotto: a dynamic binary translator for weak memory model architectures. InProceedings of the 28th ACM International Conference on Architectural Support f...

  33. [41]

    Albert Gu and Tri Dao. 2023. Mamba: Linear-time sequence modeling with selective state spaces.arXiv preprint arXiv:2312.00752(2023)

  34. [42]

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al . 2024. DeepSeek-Coder: When the Large Language Model Meets Programming–The Rise of Code Intelligence. arXiv preprint arXiv:2401.14196(2024)

  35. [43]

    Hazim Hanif and Sergio Maffeis. 2022. Vulberta: Simplified source code pre- training for vulnerability detection. In2022 International joint conference on neural networks (IJCNN). IEEE, 1–8

  36. [44]

    Haojie He, Xingwei Lin, Ziang Weng, Ruijie Zhao, Shuitao Gan, Libo Chen, Yuede Ji, Jiashui Wang, and Zhi Xue. 2024. Code is not natural language: Unlock the power of semantics-oriented graph representation for binary code similarity detection. In33rd USENIX Security Symposium ...

  37. [45]

    Jingxuan He, Pesho Ivanov, Petar Tsankov, Veselin Raychev, and Martin Vechev

  38. [46]

    Jingxuan He and Martin Vechev. 2023. Large language models for code: Security hardening and adversarial testing. InProceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security. 1865–1879

  39. [47]

    S Hochreiter. 1997. Long Short-term Memory.Neural Computation MIT-Press (1997)

  40. [48]

    Peiwei Hu, Ruigang Liang, and Kai Chen. 2024. DeGPT: Optimizing Decompiler Output with LLM. InProceedings 2024 Network and Distributed System Security Symposium (2024). https://api. semanticscholar. org/CorpusID, Vol. 267622140

  41. [49]

    Nan Jiang, Kevin Liu, Thibaud Lutellier, and Lin Tan. 2023. Impact of code language models on automated program repair. In2023 IEEE/ACM 45th Interna- tional Conference on Software Engineering (ICSE). IEEE, 1430–1442

  42. [50]

    Sungmin Kang, Juyeon Yoon, and Shin Yoo. 2023. Large language models are few-shot testers: Exploring llm-based general bug reproduction. In2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 2312–2323

  43. [51]

    Dongkwan Kim, Eunsoo Kim, Sang Kil Cha, Sooel Son, and Yongdae Kim

  44. [52]

    Soomin Kim, Hyungseok Kim, and Sang Kil Cha. 2023. Funprobe: Probing functions from binary code through probabilistic analysis. InProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 1419–1430

  45. [53]

    JongHyup Lee, Thanassis Avgerinos, and David Brumley. 2011. TIE: Principled reverse engineering of types in binary programs. (2011)

  46. [54]

    Litao Li, Steven HH Ding, Yuan Tian, Benjamin CM Fung, Philippe Charland, Weihan Ou, Leo Song, and Congwei Chen. 2023. VulANalyzeR: Explainable binary vulnerability detection with multi-task learning and attentional graph convolution.ACM Transactions on Privacy and Security26,...

  47. [55]

    Revisiting binary code similarity analysis using interpretable feature engineering and lessons learned.IEEE Transactions on Software Engineering49, 4 (2022), 1661–1682

  48. [56]

    Zongjie Li, Chaozheng Wang, Zhibo Liu, Haoxuan Wang, Dong Chen, Shuai Wang, and Cuiyun Gao. 2023. Cctest: Testing and repairing code completion systems. In2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 1238–1250

  49. [57]

    Puzhuo Liu, Chengnian Sun, Yaowen Zheng, Xuan Feng, Chuan Qin, Yuncheng Wang, Zhi Li, and Limin Sun. 2023. Harnessing the power of llm to support binary taint analysis.arXiv preprint arXiv:2310.08275(2023)

  50. [58]

    Yinhan Liu. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692364 (2019)

  51. [59]

    Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, et al . 2023. Starcoder: may the source be with you!arXiv preprint arXiv:2305.06161(2023)

  52. [60]

    Zhenhao Luo, Pengfei Wang, Baosheng Wang, Yong Tang, Wei Xie, Xu Zhou, Danjun Liu, and Kai Lu. 2023. VulHawk: Cross-architecture Vulnerability Detection with Entropy-based Binary Code Search.. InNDSS

  53. [61]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al

  54. [62]

    Niru Maheswaranathan, David Sussillo, Luke Metz, Ruoxi Sun, and Jascha Sohl- Dickstein. 2021. Reverse engineering learned optimizers reveals known and novel mechanisms.Advances in Neural Information Processing Systems34 (2021), 19910–19922

  55. [63]

    Zhongxin Liu, Zhijie Tang, Junwei Zhang, Xin Xia, and Xiaohu Yang. 2024. Pre-training by Predicting Program Dependencies for Vulnerability Analysis Tasks. InProceedings of the IEEE/ACM 46th International Conference on Software Engineering. 1–13

  56. [64]

    2024.Common Vulnerabilities and Exposures

    MITRE. 2024.Common Vulnerabilities and Exposures. https://cve.mitre.org/

  57. [65]

    2024.Common Weakness Enumeration

    MITRE. 2024.Common Weakness Enumeration. https://cwe.mitre.org/

  58. [66]

    2024.Juliet C/C++ 1.3 v1.3

    MITRE. 2024.Juliet C/C++ 1.3 v1.3. https://samate.nist.gov/SARD/test-suites/ 112

  59. [67]

    2024.The webiste of cve-2023-3699

    MITRE. 2024.The webiste of cve-2023-3699. https://cve.mitre.org/cgi-bin/ cvename.cgi?name=CVE-2023-3609

  60. [68]

    Yisroel Mirsky, George Macon, Michael Brown, Carter Yagemann, Matthew Pruett, Evan Downing, Sukarno Mertoguno, and Wenke Lee. 2023. {VulChecker}: Graph-based Vulnerability Localization in Source Code. In32nd USENIX Security Symposium (USENIX Security 23). 6557–6574

  61. [69]

    2024.The website of cwe-416

    MITRE. 2024.The website of cwe-416. https://cwe.mitre.org/data/definitions/ 416.html

  62. [70]

    Niklas Muennighoff, Qian Liu, Armel Zebaze, Qinkai Zheng, Binyuan Hui, Terry Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro Von Werra, and Shayne Longpre. 2023. Octopack: Instruction tuning code large language models.arXiv preprint arXiv:2308.07124(2023)

  63. [71]

    Daye Nam, Andrew Macvean, Vincent Hellendoorn, Bogdan Vasilescu, and Brad Myers. 2024. Using an llm to help with code understanding. InProceedings of the IEEE/ACM 46th International Conference on Software Engineering. 1–13

  64. [72]

    Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, et al

  65. [73]

    2024.The website of cve-2023-30772

    MITRE. 2024.The website of cve-2023-30772. https://cve.mitre.org/cgi-bin/ cvename.cgi?name=CVE-2023-30772

  66. [74]

    Vikram Nitin, Anthony Saieva, Baishakhi Ray, and Gail Kaiser. 2021. Direct: A transformer-based model for decompiled identifier renaming. InProceedings of the 1st Workshop on Natural Language Processing for Programming (NLP4Prog 2021). 48–57

  67. [75]

    Matt Noonan, Alexey Loginov, and David Cok. 2016. Polymorphic type inference for machine code. InProceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation. 27–41

  68. [76]

    Theo X Olausson, Jeevana Priya Inala, Chenglong Wang, Jianfeng Gao, and Armando Solar-Lezama. 2023. Demystifying gpt self-repair for code generation. arXiv preprint arXiv:2306.09896(2023)

  69. [77]

    2024.ChatGPT

    OpenAI. 2024.ChatGPT. https://openai.com/blog/chatgpt

  70. [78]

    Text and code embeddings by contrastive pre-training.arXiv preprint arXiv:2201.10005(2022). 11

  71. [79]

    Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Sil- vio Savarese, and Caiming Xiong. 2022. Codegen: An open large language model for code with multi-turn program synthesis.arXiv preprint arXiv:2203.13474 (2022)

  72. [80]

    Hammond Pearce, Benjamin Tan, Baleegh Ahmad, Ramesh Karri, and Bren- dan Dolan-Gavitt. 2023. Examining zero-shot vulnerability repair with large language models. In2023 IEEE Symposium on Security and Privacy (SP). IEEE, 2339–2356

  73. [81]

    Kexin Pei, Dongdong She, Michael Wang, Scott Geng, Zhou Xuan, Yaniv David, Junfeng Yang, Suman Jana, and Baishakhi Ray. 2022. NeuDep: neural binary memory dependence analysis. InProceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Fou...

  74. [82]

    Kexin Pei, Zhou Xuan, Junfeng Yang, Suman Jana, and Baishakhi Ray. 2022. Learning approximate execution semantics from traces for binary function similarity.IEEE Transactions on Software Engineering49, 4 (2022), 2776–2790

  75. [83]

    Tao Peng, Shixu Chen, Fei Zhu, Junwei Tang, Junping Liu, and Xinrong Hu

  76. [84]

    2024.GPT-4o

    OpenAI. 2024.GPT-4o. https://openai.com/index/hello-gpt-4o/

  77. [85]

    Chengbin Pang, Ruotong Yu, Yaohui Chen, Eric Koskinen, Georgios Portokalidis, Bing Mao, and Jun Xu. 2021. Sok: All you ever wanted to know about x86/x64 binary disassembly but were afraid to ask. In2021 IEEE symposium on security and privacy (SP). IEEE, 833–851

  78. [86]

    Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiao- qing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. 2023. Code llama: Open foundation models for code.arXiv preprint arXiv:2308.12950 (2023)

  79. [87]

    Jérémy Scheurer, Jon Ander Campos, Tomasz Korbak, Jun Shern Chan, Angelica Chen, Kyunghyun Cho, and Ethan Perez. 2023. Training language models with language feedback at scale.arXiv preprint arXiv:2303.16755(2023)

  80. [88]

    Yan Shoshitaishvili, Ruoyu Wang, Christopher Salls, Nick Stephens, Mario Polino, Andrew Dutcher, John Grosen, Siji Feng, Christophe Hauser, Christopher Kruegel, et al. 2016. Sok:(state of) the art of war: Offensive techniques in binary analysis. In2016 IEEE symposium on securi...

  81. [89]

    Benjamin Steenhoek, Hongyang Gao, and Wei Le. 2024. Dataflow analysis- inspired deep learning for efficient vulnerability detection. InProceedings of the 46th IEEE/ACM International Conference on Software Engineering. 1–13

  82. [90]

    Benjamin Steenhoek, Md Mahbubur Rahman, Richard Jiles, and Wei Le. 2023. An empirical study of deep learning models for vulnerability detection. In2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 2237–2248

  83. [91]

    Yun Peng, Shuzheng Gao, Cuiyun Gao, Yintong Huo, and Michael Lyu. 2024. Domain knowledge matters: Improving prompts with fix templates for repairing python type errors. InProceedings of the 46th IEEE/ACM International Conference on Software Engineering. 1–13

  84. [92]

    Alec Radford. 2018. Improving language understanding by generative pre- training. (2018)

  85. [93]

    Yashar Talebirad and Amirhossein Nadiri. 2023. Multi-agent collaboration: Harnessing the power of intelligent llm agents.arXiv preprint arXiv:2306.03314 (2023)

  86. [94]

    Hanzhuo Tan, Qi Luo, Jing Li, and Yuqun Zhang. 2024. LLM4Decompile: Decom- piling Binary Code with Large Language Models.arXiv preprint arXiv:2403.05286 (2024)

  87. [95]

    Wei Tang, Mingwei Tang, Minchao Ban, Ziguo Zhao, and Mingjun Feng. 2023. CSGVD: A deep learning approach combining sequence and graph embedding for source code vulnerability detection.Journal of Systems and Software199 (2023), 111623

  88. [96]

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. 2023. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805(2023)

  89. [97]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288(2023)

  90. [98]

    Zian Su, Xiangzhe Xu, Ziyang Huang, Kaiyuan Zhang, and Xiangyu Zhang. 2024. Source Code Foundation Models are Transferable Binary Analysis Knowledge Bases.arXiv preprint arXiv:2405.19581(2024)

  91. [99]

    Zian Su, Xiangzhe Xu, Ziyang Huang, Zhuo Zhang, Yapeng Ye, Jianjun Huang, and Xiangyu Zhang. 2024. Codeart: Better code models by attention regulariza- tion when symbols are lacking.Proceedings of the ACM on Software Engineering 1, FSE (2024), 562–585

  92. [100]

    A Vaswani. 2017. Attention is all you need.Advances in Neural Information Processing Systems(2017)

  93. [101]

    Fish Wang and Yan Shoshitaishvili. 2017. Angr-the next generation of binary analysis. In2017 IEEE Cybersecurity Development (SecDev). IEEE, 8–9

  94. [102]

    Hao Wang, Wenjie Qu, Gilad Katz, Wenyu Zhu, Zeyu Gao, Han Qiu, Jianwei Zhuge, and Chao Zhang. 2022. Jtrans: Jump-aware transformer for binary code similarity detection. InProceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis. 1–13

  95. [103]

    Junzhe Wang, Matthew Sharp, Chuxiong Wu, Qiang Zeng, and Lannan Luo. 2023. Can a Deep Learning Model for One Architecture Be Used for Others?{Retargeted-Architecture} Binary Code Analysis. In32nd USENIX Se- curity Symposium (USENIX Security 23). 7339–7356

  96. [104]

    Yue Wang, Weishi Wang, Shafiq Joty, and Steven CH Hoi. 2021. Codet5: Identifier-aware unified pre-trained encoder-decoder models for code under- standing and generation.arXiv preprint arXiv:2109.00859(2021)

  97. [105]

    Saad Ullah, Mingji Han, Saurabh Pujar, Hammond Pearce, Ayse Coskun, and Gianluca Stringhini. 2024. LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks. InIEEE Symposium on Security and Privacy

  98. [106]

    Jayakrishna Vadayath, Moritz Eckert, Kyle Zeng, Nicolaas Weideman, Gokulkr- ishna Praveen Menon, Yanick Fratantonio, Davide Balzarotti, Adam Doupé, Tiffany Bao, Ruoyu Wang, et al. 2022. Arbiter: Bridging the static and dynamic divide in vulnerability discovery on binary progra...

  99. [107]

    Chunqiu Steven Xia, Yuxiang Wei, and Lingming Zhang. 2023. Automated pro- gram repair in the era of large pre-trained language models. In2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 1482–1494

  100. [108]

    Danning Xie, Byungwoo Yoo, Nan Jiang, Mijung Kim, Lin Tan, Xiangyu Zhang, and Judy S Lee. 2023. Impact of large language models on generating software specifications.arXiv preprint arXiv:2306.03324(2023)

  101. [109]

    Danning Xie, Zhuo Zhang, Nan Jiang, Xiangzhe Xu, Lin Tan, and Xiangyu Zhang. 2024. ReSym: Harnessing LLMs to Recover Variable and Data Structure Symbols from Stripped Binaries. (2024)

  102. [110]

    Xiangzhe Xu, Shiwei Feng, Yapeng Ye, Guangyu Shen, Zian Su, Siyuan Cheng, Guanhong Tao, Qingkai Shi, Zhuo Zhang, and Xiangyu Zhang. 2023. Improving binary code similarity transformer models by semantics-driven instruction deemphasis. InProceedings of the 32nd ACM SIGSOFT Inter...

  103. [111]

    Shouguo Yang, Chaopeng Dong, Yang Xiao, Yiran Cheng, Zhiqiang Shi, Zhi Li, and Limin Sun. 2023. Asteria-Pro: Enhancing Deep Learning-based Binary Code Similarity Detection by Incorporating Domain Knowledge.ACM Transactions on Software Engineering and Methodology33, 1 (2023), 1–40

  104. [112]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits rea- soning in large language models.Advances in neural information processing systems35 (2022), 24824–24837

  105. [113]

    Yi Wu, Nan Jiang, Hung Viet Pham, Thibaud Lutellier, Jordan Davis, Lin Tan, Petr Babkin, and Sameena Shah. 2023. How effective are neural networks for fixing security vulnerabilities. InProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysi...

  106. [114]

    Jiawei Zhang, Haopeng Zhang, Congying Xia, and Li Sun. 2020. Graph-bert: Only attention is needed for learning graph representations.arXiv preprint arXiv:2001.05140(2020)

  107. [115]

    Zhuo Zhang, Yapeng Ye, Wei You, Guanhong Tao, Wen-chuan Lee, Yonghwi Kwon, Yousra Aafer, and Xiangyu Zhang. 2021. Osprey: Recovery of variable and data structure via probabilistic analysis for stripped binary. In2021 IEEE Symposium on Security and Privacy (SP). IEEE, 813–832

  108. [116]

    2023.{UVSCAN}: Detecting{Third-Party} Component Usage Violations in {IoT} Firmware

    Binbin Zhao, Shouling Ji, Xuhong Zhang, Yuan Tian, Qinying Wang, Yuwen Pu, Chenyang Lyu, and Raheem Beyah. 2023.{UVSCAN}: Detecting{Third-Party} Component Usage Violations in {IoT} Firmware. In32nd USENIX Security Symposium (USENIX Security 23). 3421–3438

  109. [117]

    Xin Zhou, Sicong Cao, Xiaobing Sun, and David Lo. 2024. Large Language Model for Vulnerability Detection and Repair: Literature Review and Roadmap. 12 arXiv preprint arXiv:2404.02525(2024)

  110. [118]

    Yaqin Zhou, Shangqing Liu, Jingkai Siow, Xiaoning Du, and Yang Liu. 2019. De- vign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks.Advances in neural information processing systems32 (2019). 13

  111. [119]

    Wei You, Zhuo Zhang, Yonghwi Kwon, Yousra Aafer, Fei Peng, Yu Shi, Carson Harmon, and Xiangyu Zhang. 2020. Pmp: Cost-effective forced execution with probabilistic memory pre-planning. In2020 IEEE Symposium on Security and Privacy (SP). IEEE, 1121–1138

  112. [120]

    Jiahao Yu, Xingwei Lin, Zheng Yu, and Xinyu Xing. 2024. {LLM-Fuzzer}: Scaling Assessment of Large Language Model Jailbreaks. In33rd USENIX Security Symposium (USENIX Security 24). 4657–4674

  113. [2018]

    InProceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security

    Debin: Predicting debug information in stripped binaries. InProceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. 1667–1680

  114. [2021]

    Unified pre-training for program understanding and generation.arXiv preprint arXiv:2103.06333(2021)

  115. [2022]

    In2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P)

    Investigating graph embedding methods for cross-platform binary code similarity detection. In2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P). IEEE, 60–73

  116. [2023]

    In2023 IEEE 23rd International Working Conference on Source Code Analysis and Manipulation (SCAM)

    PTLVD: Program Slicing and Transformer-based Line-level Vulnerability Detection System. In2023 IEEE 23rd International Working Conference on Source Code Analysis and Manipulation (SCAM). IEEE, 162–173

  117. [2024]

    Self-refine: Iterative refinement with self-feedback.Advances in Neural Information Processing Systems36 (2024)

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.