Pith. sign in

REVIEW 3 major objections 6 minor 69 references

Fine-tuned code LLMs can flag security patches in binary software when patches are represented as decompiled pseudo-code, reaching 91.5% accuracy.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Fine-tuned code LLMs detect binary security patches accurately on decompiler pseudo-code, and adding source-code patches to the training data improves accuracy and F1 further.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A valuable first benchmark for binary SPD with LLMs, but the train/test split likely leaks optimization-level variants of the same patch, which could inflate the headline numbers; needs a re-split before I trust the results. the 3 major comments →

arxiv 2509.06052 v1 pith:TTUXFONS submitted 2025-09-07 cs.SE cs.AIcs.CR

Empirical Study of Code Large Language Models for Binary Security Patch Detection

Qingyuan Li , Binchang Li , Cuiyun Gao , Shuzheng Gao , Zongjie Li This is my paper
classification cs.SE cs.AIcs.CR
keywords binary security patch detectioncode large language modelspseudo-codedecompilationLoRA fine-tuningassembly codesource-code augmentationempirical study
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that code LLMs can be made to detect security patches in closed-source binaries, provided the binaries are decompiled into pseudo-code rather than inspected as assembly. Prompting off-the-shelf models barely beats chance and often fails to follow instructions; fine-tuning changes this. On a new 19,448-sample dataset of pre/post-patch binary functions, the best fine-tuned model reaches 91.5% accuracy, 89.7% F1, and a 5.8% false-positive rate, and adding source-code patch data pushes those to 93.3%, 91.9%, and 3.7%. The proposed reason is that pseudo-code sits much closer to source code than assembly does, both in embedding space and in naturalness statistics, so it better activates what code LLMs learned during pre-training. If true, this gives a practical recipe for triaging binary-only vendor patches.

Core claim

The central claim is that binary security patch detection becomes achievable for code LLMs when the input is a decompiled pseudo-code patch pair rather than raw assembly, and when the model is fine-tuned rather than prompted. The study builds a 19,448-sample dataset of pre- and post-patch binary functions from five open-source projects at five optimization levels, then evaluates 19 open-weight code LLMs. Prompting fails: vanilla models often cannot even answer 'yes' or 'no', and neither chain-of-thought nor self-correction fixes that. LoRA fine-tuning changes the picture; fine-tuning on pseudo-code beats fine-tuning on assembly by average margins of 0.173 accuracy, 0.239 F1, and 0.115 false-

What carries the argument

The carrying mechanism is the pseudo-code patch pair: each binary function before and after a patch is decompiled into C-like pseudo-code, so the LLM sees text in a form closer to its pre-training distribution. On top of that sits Low-Rank Adaptation (LoRA) fine-tuning, which injects binary-SPD domain knowledge into the model without retraining all weights. The paper's explanation for why pseudo-code wins is an alignment diagnosis measured two ways: embedding distances between code representations and code naturalness (cross-entropy under a pretrained code model). A third instrument is the dataset itself—19,448 compiled patch pairs spanning five optimization levels—since it is what makes the

Load-bearing premise

The load-bearing premise is that compiling just the pre- and post-patch source file of each commit and carrying over the commit's security label produces binary patch pairs that faithfully represent real closed-source binary patches.

What would settle it

Run the best fine-tuned pseudo-code detector on binary patches from a project whose security status is established by triggering the vulnerability on the pre-patch binary; if accuracy falls well below the reported 93.3%, the compiled-open-source proxy is inflating the result. A cheaper check: compile entire repositories rather than only the pre/post-patch files for a held-out split and re-run the same fine-tuning.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Binary-only vendors and consumers can use a decompile-then-fine-tune pipeline to flag security fixes without source access.
  • Pseudo-code is a better default input than assembly for LLM-based binary code classification tasks.
  • Source-code patch data is a cheap and effective augmentation for binary patch detection, especially for small models.
  • Performance on pseudo-code is stable across optimization levels O1 through Os, so deployment does not require knowing the compiler's optimization flag.
  • Small instruction-tuned models can outperform large proprietary API models after fine-tuning, making the approach feasible on constrained hardware.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the reported accuracies are plausible upper bounds for real closed-source triage, because the labels come from open-source commit metadata and only single changed files are compiled; multi-file patches or link-time effects could degrade the detector.
  • Editorial inference: the alignment argument—representations closer to source code are easier for code LLMs—could be reused as a cheap pre-fine-tuning selection test for other binary analysis tasks such as vulnerability discovery, binary diffing, or decompiler output ranking.
  • Editorial inference: a direct testable extension is to run the same pipeline on a corpus where security patches are confirmed by actually triggering the vulnerability on the pre-patch binary; if F1 drops materially, the gap comes from label transfer rather than representation choice.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper constructs the first large-scale binary security patch detection (SPD) dataset, containing 19,448 pre- and post-patch function pairs from five open-source projects compiled at five optimization levels, each represented as both assembly code and pseudo-code. It evaluates 19 open-source code LLMs and two foundation models under zero-shot, chain-of-thought, and self-correction prompting, and then under LoRA fine-tuning on assembly- and pseudo-code representations. The central empirical claim is that fine-tuning on pseudo-code yields substantially higher accuracy, F1, and lower FPR than fine-tuning on assembly code, with the best model (LLM4Decompile-9B-v2) reaching 0.915/0.897/0.058 on pseudo-code and 0.933/0.919/0.037 after source-code augmentation. The paper further proposes an explanatory analysis (RQ3) based on embedding distances and code naturalness, and a source-code augmentation method (RQ4) that consistently improves fine-tuned models. The dataset, code, and fine-tuned models are promised as open-source.

Significance. If the headline results survive stricter evaluation, this would be a useful benchmark and a practical recipe for binary SPD with code LLMs. The study is broad in model coverage, uses a held-out split, and reports consistent gains for pseudo-code over assembly across many model families, which is an encouraging sign of robustness. The authors also contribute a concrete augmentation technique that improves smaller models substantially, and they make the dataset and fine-tuning code public, which supports reproducibility. However, the validity of the benchmark depends on the split protocol and on the representativeness of the compiled open-source proxy for closed-source binary patches; both currently need strengthening before the central numerical claims can be taken at face value.

major comments (3)
  1. [Section 3.2, Algorithm 1, Figure 5] Cross-optimization-level leakage threatens the internal validity of Tables 2 and 3. Algorithm 1 compiles each pre- and post-patch source file at O0, O1, O2, O3, and Os, and Figure 5 shows each level supplies roughly 20% of the 19,448 samples. The post-processing performs an 8:1:1 split and then applies 'exact whole-word matching' to avoid identical items across splits. This only removes byte-identical strings; it does not remove near-duplicate variants of the same commit/function produced by different optimization levels, which share the same security label. A test sample can therefore be a near-duplicate of a training sample, allowing the model to recognize a commit seen at a different optimization level. This can inflate the headline accuracies and F1 values, and it can also inflate the pseudo-code-vs-assembly gap because pseudo-code is more invariant across optimization levels than as
  2. [Section 4.3, Fig. 8] The quantitative RQ3 evidence is computed in the wrong space. The claim that the pseudo-code/source-code centroid distance is only 0.03 while the assembly/source-code distance is 0.37 is based on a t-SNE projection of the embedding vectors. t-SNE is designed to preserve local neighborhoods, not global distances, and the location of centroids in the projected 2D plane has no reliable metric meaning. Please recompute these distances in the original embedding space, or in a globally distance-preserving projection such as PCA, and report the variability (e.g., bootstrap intervals over samples). This is needed to support the explanatory conclusion that pseudo-code is semantically closer to source code.
  3. [Section 3.2 and Section 6] The proxy pipeline limits the claim about closed-source binary SPD. The dataset is built by compiling only the pre- and post-patch source files and inheriting commit-level labels from ReposVul and PatchDB. Section 6 explicitly concedes: 'We resort to compiling open-source software as a substitute.' A fix often spans multiple files or changes link-time/build behavior, and a single-file compile may not preserve the patched semantics; the commit-level label may also not transfer to every function-level sample. The central claim about detecting security patches in closed-source deployments is therefore only as strong as this proxy. Please temper the framing or provide additional validation on held-out projects/build configurations, e.g., cross-project evaluation or testing with a different compiler and optimization-flag combination.
minor comments (6)
  1. [Table 3] Header typo: 'Accuary' should be 'Accuracy'.
  2. [Section 4.4] The text says the previous lowest false positive rate was '0.58'; from Table 2 it should be 0.058.
  3. [Section 5] Typo: 'remain challenges' should be 'remaining challenges'.
  4. [Section 4.2 and Table 2] The formatting of the best F1 score in the LLMCompiler-7B row ('0.6510.207') is garbled and should be separated.
  5. [Figures 6 and 7] The box plots at each optimization level do not report the number of samples or the statistical significance of the differences; adding a brief note on non-overlap or a rank-sum test would help the robustness claim for pseudo-code.
  6. [Section 6] The internal-threats paragraph does not mention the cross-optimization-level near-duplicate issue discussed in the major comments; adding it to the threat model would improve transparency.

Circularity Check

0 steps flagged

No significant circularity: held-out fine-tuning evaluation; RQ3/RQ4 analyses are explanatory and do not reduce to fitted inputs.

full rationale

This paper is an empirical benchmark study, not a derivation. The central claims—fine-tuned code LLMs perform well on binary SPD, pseudo-code representation helps, and source-code augmentation helps—are supported by held-out test-set evaluations (Tables 2 and 3) on a dataset split 8:1:1 with exact-match deduplication. No parameter is fitted to the test set and then reported as a prediction: LoRA hyperparameters are fixed, and the RQ4 strategy choice is made on preliminary models rather than by optimizing test metrics. RQ3's similarity analysis (embedding distance and code naturalness) is explanatory: it motivates why pseudo-code may help, but it is not used to compute the test outcomes, so no equation reduces to its own input. The dataset construction inherits labels from ReposVul/PatchDB, which are external data sources with independent validation; the overlap of one current author with ReposVul is not load-bearing because the labels and benchmark are not defined in terms of this paper's results. Section 6 explicitly concedes the closed-source substitute limitation ('We resort to compiling open-source software as a substitute'), but this is an external-validity concern, not circularity. The only notable internal threat is the near-duplicate optimization-level split (same commit compiled at O0–Os may appear in both training and test under exact-match dedup), but that is a measurement-validity/leakage concern, not a circularity: it does not make any claimed result true by definition. Self-citations in related work do not carry the argument. Therefore no circular step is present.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

No new physical or conceptual entities are postulated; the augmentation strategy is a data-processing method. The ledger is dominated by domain assumptions about the benchmark pipe: label transfer, single-file compilation as a closed-source proxy, decompiler fidelity, and the pretraining-distribution argument. The hand-chosen experimental settings (LoRA hyperparameters, length filters, prompts) are free parameters that affect all reported numbers.

free parameters (4)
  • LoRA fine-tuning hyperparameters = lora_rank=8, lora_alpha=16, learning_rate=1e-4, batch_size=2, cutoff_len=4096, num_train_epochs=3
    Chosen from LLaMAFactory recommendations and manual adjustment (Section 3.4, Section 6); they shape all fine-tuning results in Tables 2 and 3.
  • num_train_epochs = 3
    Set based on validation set performance (Section 6), i.e., tuned on the validation split and shared by all models.
  • Double-threshold function-length filter bounds = not reported
    Functions that are "excessively short or long" are removed (Section 3.2); the actual thresholds are not reported, yet they determine dataset composition, class balance, and the reported metrics.
  • Prompt templates for zero-shot, CoT, and self-correction = not shown in the paper
    The exact prompts are not included in the preprint, so RQ1 failure rates and accuracy are not independently reproducible.
axioms (5)
  • domain assumption Security labels transfer from commit-level source patches to compiled single-file, function-level binary pairs.
    The dataset inherits security/non-security labels from ReposVul and PatchDB (Section 3.2) without validation at function or binary granularity; this is load-bearing for every reported accuracy.
  • domain assumption Single-file compilation at O0-Os yields binaries representative of real closed-source patch artifacts.
    Algorithm 1 compiles only the changed source files, not whole projects; Section 6 acknowledges this is a substitute for closed-source binaries, which are unavailable.
  • domain assumption IDA Pro disassembly and decompilation preserve the semantic patch delta (function calls, argument values, control flow).
    The motivating example (Figure 1) is the evidence offered; decompiler fidelity at scale is not validated.
  • domain assumption Code LLMs are pretrained predominantly on source code and perform better on representations closer to that distribution.
    Stated explicitly by the authors as an assumption in Section 3.4 for RQ3; cited supports are pretraining-mixture and multilingual studies [31, 39].
  • domain assumption t-SNE projection preserves relative distances between code representation clusters.
    Used to compute centroid distances 0.03 versus 0.37 in Section 4.3; t-SNE does not preserve global distances, so this assumption is violated.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Empirical Study of Code Large Language Models for Binary Security Patch Detection." pith.science (2026). https://pith.science/paper/TTUXFONS

@misc{pith2026250906052,
  author       = {Pith},
  title        = {Pith review of: Empirical Study of Code Large Language Models for Binary Security Patch Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TTUXFONS}},
  note         = {Machine review of arXiv:2509.06052}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Security patch detection (SPD) is crucial for maintaining software security, as unpatched vulnerabilities can lead to severe security risks. In recent years, numerous learning-based SPD approaches have demonstrated promising results on source code. However, these approaches typically cannot be applied to closed-source applications and proprietary systems that constitute a significant portion of real-world software, as they release patches only with binary files, and the source code is inaccessible. Given the impressive performance of code large language models (LLMs) in code intelligence and binary analysis tasks such as decompilation and compilation optimization, their potential for detecting binary security patches remains unexplored, exposing a significant research gap between their demonstrated low-level code understanding capabilities and this critical security task. To address this gap, we construct a large-scale binary patch dataset containing \textbf{19,448} samples, with two levels of representation: assembly code and pseudo-code, and systematically evaluate \textbf{19} code LLMs of varying scales to investigate their capability in binary SPD tasks. Our initial exploration demonstrates that directly prompting vanilla code LLMs struggles to accurately identify security patches from binary patches, and even state-of-the-art prompting techniques fail to mitigate the lack of domain knowledge in binary SPD within vanilla models. Drawing on the initial findings, we further investigate the fine-tuning strategy for injecting binary SPD domain knowledge into code LLMs through two levels of representation. Experimental results demonstrate that fine-tuned LLMs achieve outstanding performance, with the best results obtained on the pseudo-code representation.

Figures

Figures reproduced from arXiv: 2509.06052 by Binchang Li, Cuiyun Gao, Qingyuan Li, Shuzheng Gao, Zongjie Li.

Figure 1
Figure 1. Figure 1: A motivating example from Linux Kernel, and the high [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the empirical study 3.1 Research Questions We investigate the effectiveness of code LLMs in detecting binary security patches by answering the following research questions (RQs). RQ1: Can code LLMs be directly applied to binary security patch detection via prompting? RQ2: Can code LLMs be effectively adapted to binary security patch detection via fine-tuning? RQ3: Why do fine-tuned code LLMs de… view at source ↗
Figure 3
Figure 3. Figure 3: Construction workflow of the binary SPD dataset [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Pie chart of source projects’ distribution [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Box plots of the metrics on the assembly-code dataset at different optimization levels [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Box plots of the metrics on the pseudo-code dataset at different optimization levels [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: T-SNE plot of different code embeddings 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 assembly code pseudo code source code [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Violin plot of different code naturalness, and samples differ [PITH_FULL_IMAGE:figures/full_fig_p015_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Venn diagram of undetected security patches [PITH_FULL_IMAGE:figures/full_fig_p017_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

69 extracted references · 47 canonical work pages · 2 internal anchors

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report.arXiv preprint arXiv:2303.08774(2023)

    Pith/arXiv arXiv 2023

  2. [2]

    Toufique Ahmed, Christian Bird, Premkumar Devanbu, and Saikat Chakraborty. 2024. Studying llm performance on closed-and open-source data. arXiv preprint arXiv:2402.15100(2024)

    Pith/arXiv arXiv 2024

  3. [3]

    Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou, and Weizhu Chen. 2024. Make your llm fully utilize the context.Advances in Neural Information Processing Systems (NeurIPS)37 (2024), 62160–62188

    work page 2024

  4. [4]

    Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, Huazuo Gao, Kaige Gao, Wenjun Gao, Ruiqi Ge, Kang Guan, Daya Guo, Jianzhong Guo, Guangbo Hao, Zhewen Hao, Ying He, Wenjie Hu, Panpan Huang, Erhang Li, Guowei Li, Jiashi Li, Yao Li, Y. K. Li, Wenfeng Liang, Fangyun Lin, Alex X. Liu, Bo Liu...

    Pith/arXiv arXiv 2024

  5. [5]

    Zimin Chen, Steve Kommrusch, and Martin Monperrus. 2022. Neural transfer learning for repairing security vulnerabilities in c code.IEEE Transactions on Software Engineering (TSE)49, 1 (2022), 147–165

    work page 2022

  6. [6]

    Chris Cummins, Volker Seeker, Dejan Grubisic, Baptiste Roziere, Jonas Gehring, Gabriel Synnaeve, and Hugh Leather. 2025. LLM Compiler: Foundation Language Models for Compiler Optimization. InProceedings of the 34th ACM SIGPLAN International Conference on Compiler Construction. 141–153

    work page 2025

  7. [7]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai D...

    Pith/arXiv arXiv 2025

  8. [8]

    DeepSeek-AI, Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, Runxin Xu, Y. Wu, Yukun Li, Huazuo Gao, Shirong Ma, Wangding Zeng, Xiao Bi, Zihui Gu, Hanwei Xu, Damai Dai, Kai Dong, Liyue Zhang, Yishi Piao, Zhibin Gou, Zhenda Xie, Zhewen Hao, Bingxuan Wang, Junxiao Song, Deli Chen, Xin Xie, Kang Guan, Yuxiang You, Aixin Liu, Qiushi Du, Wenjun Gao...

    Pith/arXiv arXiv 2024

  9. [9]

    Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, et al. 2023. Parameter-efficient fine-tuning of large-scale pre-trained language models.Nature Machine Intelligence5, 3 (2023), 220–235

    2023

  10. [10]

    Nicholas Economides and Evangelos Katsamakas. 2006. Two-sided competition of proprietary vs. open source technology platforms and the implications for the software industry.Management science52, 7 (2006), 1057–1071

    work page 2006

  11. [11]

    C Frischknecht. 2017. $1.1 Trillion Impacted by Software Defects: A Testing Fail

    work page 2017

  12. [12]

    Michael Fu, Chakkrit Tantithamthavorn, Trung Le, Van Nguyen, and Dinh Phung. 2022. VulRepair: a T5-based automated software vulnerability repair. InProceedings of the 30th ACM joint european software engineering conference and symposium on the foundations of software engineering (ESEC/FSE). 935–947

    work page 2022

  13. [13]

    Xu He, Shu Wang, Pengbin Feng, Xinda Wang, Shiyu Sun, Qi Li, and Kun Sun. 2024. Bingo: Identifying security patches in binary code with graph representation learning. InProceedings of the 19th ACM Asia Conference on Computer and Communications Security. 1186–1199

    work page 2024

  14. [14]

    Abram Hindle, Earl T Barr, Mark Gabel, Zhendong Su, and Premkumar Devanbu. 2016. On the naturalness of software.Commun. ACM59, 5 (2016), 122–131

    work page 2016

  15. [15]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. Lora: Low-rank adaptation of large language models.ICLR1, 2 (2022), 3

    2022

  16. [16]

    Siming Huang, Tianhao Cheng, J. K. Liu, Jiaran Hao, Liuyihan Song, Yang Xu, J. Yang, J. H. Liu, Chenchen Zhang, Linzheng Chai, Ruifeng Yuan, Zhaoxiang Zhang, Jie Fu, Qian Liu, Ge Zhang, Zili Wang, Yuan Qi, Yinghui Xu, and Wei Chu. 2024. OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models.CoRRabs/2411.04905 (2024)

    Pith/arXiv arXiv 2024

  17. [17]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, An Yang, Rui Men, Fei Huang, Xingzhang Ren, Xuancheng Ren, Jingren Zhou, and Junyang Lin. 2024. Qwen2.5-Coder Technical Report.CoRRabs/2409.12186 (2024)

    Pith/arXiv arXiv 2024

  18. [18]

    Nan Jiang, Chengxiao Wang, Kevin Liu, Xiangzhe Xu, Lin Tan, Xiangyu Zhang, and Petr Babkin. 2025. Nova: Generative Language Models for Assembly Code with Hierarchical Attention and Contrastive Learning. InThe Thirteenth International Conference on Learning Representations

    work page 2025

  19. [19]

    Siyuan Jiang, Jia Li, He Zong, Huanyu Liu, Hao Zhu, Shukai Hu, Erlu Li, Jiazheng Ding, Yu Han, Wei Ning, et al. 2024. aiXcoder-7B: A Lightweight and Effective Large Language Model for Code Processing.arXiv preprint arXiv:2410.13187(2024)

    Pith/arXiv arXiv 2024

  20. [20]

    Hongye Jin, Xiaotian Han, Jingfeng Yang, Zhimeng Jiang, Zirui Liu, Chia-Yuan Chang, Huiyuan Chen, and Xia Hu. 2024. LLM Maybe LongLM: SelfExtend LLM context window without tuning. InProceedings of the 41st International Conference on Machine Learning. 22099–22114

    work page 2024

  21. [21]

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models.arXiv preprint arXiv:2001.08361(2020)

    Pith/arXiv arXiv 2020

  22. [22]

    Kipf and Max Welling

    Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings

    work page 2017

  23. [23]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems (NeurIPS)35 (2022), 22199–22213

    work page 2022

  24. [24]

    Zhe Lang, Shouguo Yang, Yiran Cheng, Xiaoling Zhang, Zhiqiang Shi, and Limin Sun. 2021. PMatch: Semantic-based patch detection for binary programs. In2021 IEEE International Performance, Computing, and Communications Conference (IPCCC). IEEE, 1–10

    work page 2021

  25. [25]

    Sang-Yong Tom Lee and Zhaoli Meng. 2005. Open source vs. proprietary software: competition and compatibility.Proprietary Software: Competition and Compatibility(2005)

    work page 2005

  26. [26]

    Frank Li and Vern Paxson. 2017. A large-scale empirical study of security patches. InProceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS). 2201–2215

    work page 2017

  27. [27]

    Raymond Li, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, LI Jia, Jenny Chim, Qian Liu, et al

    work page

  28. [28]

    Tsz-On Li, Wenxi Zong, Yibo Wang, Haoye Tian, Ying Wang, Shing-Chi Cheung, and Jeff Kramer. 2023. Nuances are the key: Unlocking chatgpt to find failure-inducing tests with differential prompting. In2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 14–26. Manuscript submitted to ACM 22 Qingyuan Li, Binchang Li, Cui...

    work page 2023

  29. [29]

    Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard S. Zemel. 2016. Gated Graph Sequence Neural Networks. In4th International Conference on Learning Representations, ICLR 2016

    work page 2016

  30. [30]

    Zhuo Li, Hengyi Li, and Lin Meng. 2023. Model compression for deep neural networks: A survey.Computers12, 3 (2023), 60

    work page 2023

  31. [31]

    Zihao Li, Yucheng Shi, Zirui Liu, Fan Yang, Ninghao Liu, and Mengnan Du. 2024. Quantifying multilingual performance of large language models across languages.CoRR(2024)

    work page 2024

  32. [32]

    Ruyan Lin, Yulong Fu, Wei Yi, Jincheng Yang, Jin Cao, Zhiqiang Dong, Fei Xie, and Hui Li. 2024. Vulnerabilities and Security Patches Detection in OSS: A Survey.Comput. Surveys57, 1 (2024), 1–37

    work page 2024

  33. [33]

    Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy-Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, et al. 2024. Starcoder 2 and the stack v2: The next generation.arXiv preprint arXiv:2402.19173(2024)

    Pith/arXiv arXiv 2024

  34. [34]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2023. Self-refine: Iterative refinement with self-feedback.Advances in Neural Information Processing Systems (NeurIPS)36 (2023), 46534–46594

    work page 2023

  35. [35]

    Antonio Mastropaolo, Emad Aghajani, Luca Pascarella, and Gabriele Bavota. 2021. An empirical study on code comment completion. In2021 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 159–170

    work page 2021

  36. [36]

    Samim Mirhosseini and Chris Parnin. 2017. Can automated pull requests encourage software developers to upgrade out-of-date dependencies?. In 2017 32nd IEEE/ACM international conference on automated software engineering (ASE). IEEE, 84–94

    work page 2017

  37. [37]

    Jialing Pan, Adrien Sadé, Jin Kim, Eric Soriano, Guillem Sole, and Sylvain Flamant. 2023. SteloCoder: a decoder-only LLM for multi-language to Python code translation.arXiv preprint arXiv:2310.15539(2023)

    Pith/arXiv arXiv 2023

  38. [38]

    Jiaqi Peng, Feng Li, Bingchang Liu, Lili Xu, Binghong Liu, Kai Chen, and Wei Huo. 2019. 1dvul: Discovering 1-day vulnerabilities through binary patches. In2019 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). IEEE, 605–616

    work page 2019

  39. [39]

    Jackson Petty, Sjoerd van Steenkiste, and Tal Linzen. 2025. How Does Code Pretraining Affect Language Model Task Performance?Transactions on Machine Learning Research (TMLR)2025 (2025)

    work page 2025

  40. [40]

    naturalness

    Baishakhi Ray, Vincent Hellendoorn, Saheel Godhane, Zhaopeng Tu, Alberto Bacchelli, and Premkumar Devanbu. 2016. On the" naturalness" of buggy code. InProceedings of the 38th International Conference on Software Engineering (ICSE). 428–439

    work page 2016

  41. [42]

    Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al

    work page

  42. [43]

    Wojciech Samek, Grégoire Montavon, Sebastian Lapuschkin, Christopher J Anders, and Klaus-Robert Müller. 2021. Explaining deep neural networks and beyond: A review of methods and applications.Proc. IEEE109, 3 (2021), 247–278

    work page 2021

  43. [44]

    Code llama: Open foundation models for code.arXiv preprint arXiv:2308.12950(2023)

    Pith/arXiv arXiv 2023

  44. [45]

    Hanzhuo Tan, Qi Luo, Jing Li, and Yuqun Zhang. 2024. LLM4Decompile: Decompiling Binary Code with Large Language Models. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP). 3473–3487

    work page 2024

  45. [46]

    André Silva, Sen Fang, and Martin Monperrus. 2023. Repairllama: Efficient representations and fine-tuned adapters for program repair.arXiv preprint arXiv:2312.15698(2023)

    Pith/arXiv arXiv 2023

  46. [47]

    Chandra Thapa, Seung Ick Jang, Muhammad Ejaz Ahmed, Seyit Camtepe, Josef Pieprzyk, and Surya Nepal. 2022. Transformer-based language models for software vulnerability detection. InProceedings of the 38th Annual Computer Security Applications Conference. 481–496

    work page 2022

  47. [48]

    Xunzhu Tang, Zhenghan Chen, Kisub Kim, Haoye Tian, Saad Ezzini, and Jacques Klein. 2023. Just-in-Time Security Patch Detection–LLM At the Rescue for Data Augmentation.arXiv preprint arXiv:2312.01241(2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  48. [49]

    Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE.Journal of machine learning research9, 11 (2008)

    2008

  49. [50]

    Saiteja Utpala, Alex Gu, and Pin Yu Chen. 2023. Language Agnostic Code Embeddings.arXiv preprint arXiv:2310.16803(2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  50. [51]

    Xinchen Wang, Ruida Hu, Cuiyun Gao, Xin-Cheng Wen, Yujia Chen, and Qing Liao. 2024. Reposvul: A repository-level high-quality vulnerability dataset. InProceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering (ICSE): Companion Proceedings. 472–483

    work page 2024

  51. [52]

    Shu Wang, Xinda Wang, Kun Sun, Sushil Jajodia, Haining Wang, and Qi Li. 2023. GraphSPD: Graph-based security patch detection with enriched code semantics. In2023 IEEE Symposium on Security and Privacy (SP). IEEE, 2409–2426

    work page 2023

  52. [53]

    Xinda Wang, Shu Wang, Pengbin Feng, Kun Sun, Sushil Jajodia, Sanae Benchaaboun, and Frank Geck. 2021. Patchrnn: A deep learning-based system for security patch identification. InMILCOM 2021-2021 IEEE Military Communications Conference (MILCOM). IEEE, 595–600

    work page 2021

  53. [54]

    Xinda Wang, Shu Wang, Pengbin Feng, Kun Sun, and Sushil Jajodia. 2021. Patchdb: A large-scale security patch dataset. In2021 51st Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). IEEE, 149–160

    work page 2021

  54. [55]

    Jônatas Wehrmann, Maurício Armani Lopes, Douglas Matos de Souza, and Rodrigo Coelho Barros. 2019. Language-Agnostic Visual-Semantic Embeddings. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    work page 2019

  55. [56]

    Yue Wang, Weishi Wang, Shafiq Joty, and Steven CH Hoi. 2021. CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP). 8696–8708

    work page 2021

  56. [57]

    Martin Weyssow, Xin Zhou, Kisub Kim, David Lo, and Houari Sahraoui. 2023. Exploring parameter-efficient fine-tuning techniques for code generation with large language models.ACM Transactions on Software Engineering and Methodology (TOSEM)(2023)

    work page 2023

  57. [58]

    Xin-Cheng Wen, Zirui Lin, Cuiyun Gao, Hongyu Zhang, Yong Wang, and Qing Liao. 2024. Repository-Level Graph Representation Learning for Enhanced Security Patch Detection.arXiv preprint arXiv:2412.08068(2024). Manuscript submitted to ACM Empirical Study of Code Large Language Models for Binary Security Patch Detection 23

    Pith/arXiv arXiv 2024

  58. [59]

    Chunqiu Steven Xia and Lingming Zhang. 2022. Less training, more repairing please: revisiting automated program repair via zero-shot learning. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 959–971

    2022

  59. [60]

    Chunqiu Steven Xia, Yuxiang Wei, and Lingming Zhang. 2023. Automated program repair in the era of large pre-trained language models. In2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 1482–1494

    2023

  60. [61]

    Chen Yang, Junjie Chen, Jiajun Jiang, and Yuliang Huang. 2024. Dependency-aware code naturalness.Proceedings of the ACM on Programming Languages8, OOPSLA2 (2024), 2355–2377

    work page 2024

  61. [62]

    Zhengzi Xu, Bihuan Chen, Mahinthan Chandramohan, Yang Liu, and Fu Song. 2017. Spain: security patch analysis for binaries towards understanding the pain and pills. In2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE). IEEE, 462–472

    work page 2017

  62. [63]

    Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Heng Li, Jiangcheng Zhu, Jianqun Chen, Jing Chang, Kaidong Yu, Peng Liu, Qiang Liu, Shawn Yue, Senbin Yang, Shiming Yang, Tao Yu, Wen Xie, Wenhao Huang, Xiaohui Hu, Xiaoyi Ren, Xinyao Niu, Pengcheng Nie, Yuchi Xu, Yudong Liu, Yue Wang, Yuxuan Cai, Zhenyu Gu, Zhiyuan Liu, and Zonghong D...

    Pith/arXiv arXiv 2024

  63. [64]

    Jun Yang, Yuehan Wang, Yiling Lou, Ming Wen, and Lingming Zhang. 2023. A large-scale empirical review of patch correctness checking approaches. InProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE). 1203–1215

    work page 2023

  64. [65]

    Jian Zhang, Chong Wang, Anran Li, Wenhan Wang, Tianlin Li, and Yang Liu. 2024. VulAdvisor: Natural Language Suggestion Generation for Software Vulnerability Repair. InProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering (ASE). 1932–1944

    work page 2024

  65. [66]

    Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals. 2014. Recurrent neural network regularization.arXiv preprint arXiv:1409.2329(2014)

    Pith/arXiv arXiv 2014

  66. [67]

    Xuandong Zhao, Siqi Ouyang, Zhiguo Yu, Ming Wu, and Lei Li. 2023. Pre-trained Language Models Can be Fully Zero-Shot Learners. InThe 61st Annual Meeting Of The Association For Computational Linguistics

    work page 2023

  67. [68]

    Quanjun Zhang, Chunrong Fang, Tongke Zhang, Bowen Yu, Weisong Sun, and Zhenyu Chen. 2023. Gamma: Revisiting template-based automated program repair via mask prediction. In2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 535–547

    2023

  68. [70]

    Xin Zhou, Kisub Kim, Bowen Xu, DongGyun Han, and David Lo. 2024. Out of Sight, Out of Mind: Better Automatic Vulnerability Repair by Broadening Input Ranges and Sources. InProceedings of the IEEE/ACM 46th International Conference on Software Engineering (ICSE). 1–13. Received 20 February 2007; revised 12 March 2009; accepted 5 June 2009 Manuscript submitt...

    work page 2024

  69. [2023]

    StarCoder: may the source be with you!Transactions on Machine Learning Research(2023)

    work page 2023

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.